# Cause-and-Effect Analysis of ADAS: A Comparison Study between Literature
Review and Complaint Data

Jackie Ayoub<EMAIL_ADDRESS>University of Michigan Dearborn4901 Evergreen
RoadDearbornMI48128USA ,  Zifei Wang<EMAIL_ADDRESS>University of Michigan
Dearborn4901 Evergreen RoadDearbornMI48128USA ,  Meitang Li<EMAIL_ADDRESS>University of Michigan Transportation Research Institute2901 Baxter RdAnn
ArborMI48109USA ,  Huizhong Guo<EMAIL_ADDRESS>University of Michigan
Transportation Research Institute2901 Baxter Rd.Ann ArborMI48109USA ,  Rini
Sherony<EMAIL_ADDRESS>Toyota Motor North America1555 Woodridge
AveAnn ArborMI48105USA ,  Shan Bao<EMAIL_ADDRESS>University of Michigan
Dearborn; University of Michigan Transportation Research Institute4901
Evergreen RoadDearbornMI48128USA and  Feng Zhou<EMAIL_ADDRESS>University
of Michigan Dearborn4901 Evergreen RoadDearbornMI48128USA

(2022)

###### Abstract.

Advanced driver assistance systems (ADAS) are designed to improve vehicle
safety. However, it is difficult to achieve such benefits without
understanding the causes and limitations of the current ADAS and their
possible solutions. This study 1) investigated the limitations and solutions
of ADAS through a literature review, 2) identified the causes and effects of
ADAS through consumer complaints using natural language processing models, and
3) compared the major differences between the two. These two lines of research
identified similar categories of ADAS causes, including human factors,
environmental factors, and vehicle factors. However, academic research focused
more on human factors of ADAS issues and proposed advanced algorithms to
mitigate such issues while drivers complained more of vehicle factors of ADAS
failures, which led to associated top consequences. The findings from these
two sources tend to complement each other and provide important implications
for the improvement of ADAS in the future.

ADAS, cause and effect, natural language processing, literature review,
complaint data

††journalyear: 2022††copyright: acmcopyright††conference: 14th International
Conference on Automotive User Interfaces and Interactive Vehicular
Applications; September 17–20, 2022; Seoul, Republic of Korea††booktitle: 14th
International Conference on Automotive User Interfaces and Interactive
Vehicular Applications (AutomotiveUI ’22), September 17–20, 2022, Seoul,
Republic of Korea††price: 15.00††doi: 10.1145/3543174.3547117††isbn:
978-1-4503-9415-4/22/09

##  1\. Introduction

Advanced driver assistance systems (ADAS) provide warnings to drivers of an
impending collision and/or temporal controls of the vehicle’s speed and/or
direction under certain circumstances to assist the driver (27). Examples of
ADAS include but are not limited to Forward Collision Warning (FCW), Automatic
Emergency Braking (AEB), Lane Departure Warning (LDW), Lane Keeping Assistance
(LKA), Blind Spot Warning (BSW), and Adaptive Cruise Control (ACC). These ADAS
are designed to take over drivers’ responsibilities and/or extend their
capabilities. For example, an AEB system can brake automatically if the in-
vehicle sensor detects that the vehicle is going to collide with another
object in front of it and a BSW system can provide warnings if another vehicle
is driving in the adjacent lane in the drivers’ blind spot. Thus, the safety
benefits of ADAS, when it is implemented and used properly, are considerable
because it can significantly decrease human error, economic cost, pollution
(67). Previous studies (12; 16; 36; 37; 59) reported that vehicles with such
ADAS had significantly fewer crashes and lower insurance losses compared to
vehicles without ADAS.

However, the benefits of ADAS will not be reached if they are not used
properly or drivers do not have proper trust in them due to possible ADAS
failures (5; 7). Reagan and McCartt (50) found that under-trust and false
alarms might be the main causes that led to the disuse of ADAS. For example,
it was found that participants only had moderate levels of trust in LDW and
LKA. Some vehicles’ FCW systems provided excessive alarms or even false alarms
when the drivers did not perceive danger (31). Other studies found that there
were negative ADAS effects on driver behavior, including distrust or
overreliance on the system, wrong mental model used in the locus of control,
and lack of support for behavioral adaptation, situation awareness, or mental
workload (51). For example, vehicles with ADAS might cause drivers to take a
passive role in driving with less caution and attention, and thus more likely
to be involved in traffic accidents (56; 75; 77). Paul et al. (48) emphasized
that the sole purpose of ADAS was to support drivers rather than to replace
them. Therefore, it is imperative to examine the causes of system failures,
especially those related to human factors and system designs, to identify
solutions to improve ADAS design and safety.

The overall goal of this study is to investigate ADAS failures (its causes and
effects) and possible solutions through literature review and text data mining
to provide recommendations for improved safety and customer experience. The
overall goal of this work is divided into four specific tasks as follows:

1) Explore ADAS limitations and solutions by conducting a literature review;

2) Build a natural language processing model based on deep learning to extract
cause-and-effect relationships of customer perceptions in ADAS uses from the
National Highway Traffic Safety Administration (NHTSA) consumer complaints
database (46);

3) Build a taxonomy of the causes and effects related to ADAS uses;

4) Discuss the differences between the findings above to help provide
recommendations for improving customer experience with ADAS.

##  2\. Literature Review

We conducted a literature review to examine the limitations and proposed
solutions of ADAS by searching journal articles, conference papers, and
technical reports published between January 1, 2016 and June 31, 2021 using a
combination of keywords in five academic databases, including Science Direct,
ACM Digital Library, Scopus, IEEE Xplore, and PsycINFO. A complimentary search
was performed in Google Scholar to accomplish a thorough result. The keywords
included ADAS (Advanced driver/driving assistant/assistance system), FCW
(forward collision warning), AEB (automatic/automated emergent/emergency
brake/braking), LDW (lane departure warning/alert/prevention), LKA (lane-
keeping assist/assistance), BSW (blind-spot warning/alert/monitor/assist), ACC
(adaptive cruise control), automatic braking, automatic steering, automated
driving, lane trace assist, supercruise, autopilot, specific names for
different OEMs, lane departure prevention, and rear cross-traffic alert system
with limitation, failure or improvement. In this process, we found 842 related
papers initially. The following criteria were defined to select eligible
papers:

1\. The paper identified one or more ADAS limitations/failures;

2\. The paper proposed one or more solutions for an existing ADAS problem.

The initial screening ended up with 72 papers. Then we carefully studied
through the eligible papers and selected 67 papers that were highly relevant
to the topic of our study regarding the limitations and solutions of ADAS
based on the topics, methodologies and main findings of the papers. Among the
selected papers, 26 of them were related to existing limitations of ADAS
(38.8%), while 41 of the papers were focused on the solutions proposed to
improve the performance of ADAS technologies (61.2%). We then summarized the
limitations and solutions of ADAS into three main categories, including human
factors, environmental factors, and vehicle factors. Within each main
category, subcategories were also defined to further explain the limitations.
For the studies related to ADAS limitations, most of them were about “Driver”
(40.7%), followed by “Software” (14.8%), “Adverse weather” (11.1%), “Hardware”
(11.1%), “Lane” (7.4%), and the rest were “Other driver/vehicle” (3.7%), “VRU”
(3.7%), “Light condition” (3.7%), and “Road” (3.7%). For the studies related
to ADAS solutions, the majority were about “Driver” (17.4%) and “Lane”
(17.4%), followed by “Hardware” (15.2%), “Other driver/vehicle” (10.9%),
“Light” (10.9%), “Software” (10.9%), and the rest solutions were about “VRU”
(8.7%), “Road” (6.5%), and “Adverse weather” (2.2%). Though having a more
detailed classification would provide further insight for understanding the
limitations and solutions of ADAS, the existing literature did not offer
enough information for further analysis.

###  2.1. ADAS Limitations

First, for ADAS limitations related to human factors, we identified three
subcategories, i.e., drivers, pedestrians, and other vehicles/drivers. The
first subcategory related to human factors was driver misuse or disuse of
ADAS, either intentional or unintentional, due to the trust level of the
driver in the system and/or the driver’s misunderstanding of the system.
Example studies (15; 3; 18) showed that drivers’ partial understanding or
misunderstanding of the system’s capabilities and limitations, led to system
misuse with possible serious consequences. A survey conducted by McDonald et
al. (42) showed that only 21% of the respondents correctly pointed out that
BSW was not designed for extremely high speed, and 62% believed that FCW would
automatically apply brakes. In the meantime, an appropriate level of trust in
the system was essential to fully benefit from ADAS. Overtrust caused
unintentional misuse as drivers believed that the technology was more capable
than it was. For example, drivers engaged in other activities while the ACC
was on and did not visually check the blind spot monitor (BSM) when
overtrusting the ADAS capability (42). The disuse of ADAS was an intentional
disengagement of the system when drivers have insufficient trust of the system
or when they were annoyed or distracted by system alarms (47; 27). For
example, Reagan and McCartt (50) showed that the LDW function was turned off
in 67.2% of the vehicles observed due to lack of trust and excessive false
alarms. The second subcategory of human factors is pedestrian. A study led by
AAA (4) revealed that the functioning of the AEB system was unsuccessful under
certain circumstances, such as children darting out, two pedestrians alongside
the roadway when their silhouette was unclear from the background, or
immediately after a right turn. The third category included other drivers and
vehicles cutting in the front, causing the ACC system to malfunction with
either late braking or excessive braking (11).

The second major limitation of ADAS is that its performance was largely
influenced by the environment. Such factors include adverse weather, lighting
conditions, and lane and road conditions. It was reported that when the
precipitation of rainfall was higher than 20mm/h, the sensors of ADAS would
lose effectiveness (22). Other adverse weather conditions, such as snow, fog,
ice, crosswind, could also block and disable the sensors of ADAS (52).
Extremely low or high temperature was another major cause of sensor failures
(9). Apart from adverse weather, lighting conditions could also offset the
benefits of ADAS. While darkness was a common problem, system failures could
also be caused by the glare of sunrise or sunset (64). Many lane related ADAS,
such as LKA and LDW might be ineffective under certain road conditions, e.g.,
when the lane divisions were not clear or covered by objects like snow,
leaves, or debris (9). Moreover, if the road was wet and/or slippery, some
ADAS functions, such as ACC and LKA, might be disabled due to the interactions
between the tires and the roads (64; 66).

The third type of ADAS limitations related to the vehicle can be further
divided into hardware and software issues. For the hardware part of the
system, the limitations of the sensors, such as radars, lidars, and cameras,
were the major causes of system malfunctions (64; 8). The performance of ADAS
sensors could also be reduced if vehicle modifications were not compatible
with the system. For example, the ADAS sensors might be damaged or obstructed
by the trailers attached to the vehicle (9). For the software issues,
cybersecurity was one of the main concerns regarding the ADAS for that ADAS
functions could be disturbed by cyberattacks, such as denial-of-service
attacks or theft of confidential information (43). Furthermore, the
reliability of algorithms related to sensor fusion, data merging, and other
functions was another challenge for ADAS. The information provided by multiple
sensors based on sensor fusion algorithms might be inconsistent or completely
opposite, which could cause inaccurate and unreliable predictions. (79; 28).

###  2.2. ADAS Solutions

Many reviewed studies proposed solutions for the identified ADAS problems,
with an overall goal to improve the performance of the system. This section
summarizes the solutions based on the same classifications used for
limitations. The solutions are categorized into three main categories, human
factors, environmental factors, and vehicle factors. We examined whether the
ADAS solutions covered all the ADAS limitations identified in the previous
studies. For the sub-categories “Light condition”, “Lane”, “Road” and
“Hardware”, all the topics described in the limitation-related studies were
covered in the solution-related studies, which indicates that the problems
were widely studied. However, for the sub-categories “Driver”, “VRU”, “Adverse
Weather” and “Software”, only part of the topics were found in the solution-
related studies. Among the studies related to human factors, the limitations
regarding driver’s trust required were in lack of solutions. And there was an
unsolved difficulty caused by silhouette issues of pedestrians walking
together. The solutions found to deal with environmental issues were more
focused on improving the detection accuracy under light-insufficient
conditions, however, the adverse weather related issues such as extreme
temperature, snow/ice blockage were also in need of solutions. In the area of
software-related limitations of ADAS, the malfunction caused by unreliable and
unstable algorithms was in lack of solutions apart from the cyber attack
problems. These limitations may need further explorations and investigation to
provide a more promising performance of ADAS.

First, the proposed solutions focused on the algorithms and methods for
detection and prediction improvement to solve human factor issues. In the
subcategory of ”Driver”, for the limitations related to driver’s unintentional
misuse caused by inaccurate understanding or overtrust, one solution was to
monitor the driver’s behaviour, identify, and correct the misbehavior in time.
Therefore, researchers proposed algorithms to detect and predict the
intentions of drivers to alert them of risky traffic situations and make
decisions about whether to take over the driver’s control of the vehicle (26;
19; 29). Other studies also examined the methods to improve driver’s behavior
modeling and detection, such as eye gaze detection, fatigue detection, and
other driving behavior detection for better monitoring performance (68; 71;
73; 65; 72; 74; 75). In terms of drivers’ intentional disuse of the system,
Meuller et al. (44) proposed a driver monitoring system based on driver
behavior tracking to detect deliberate disengagement and misuse of ADAS, the
system then used attention reminders and proactive methods to keep drivers
engaged in the driver-system interactions and maintain the system’s
functionalities. Pedestrian detection was another major issue for ADAS, and
previous studies focused on the framework of intention recognition (49). For
example, studies were proposed to deal with the problem that the pedestrians
on the roads were not sensitive enough for the sensors to detect them when
they were partially obstructed (10; 14). Other vehicles around the ego vehicle
also influenced the ADAS performance. Hence, previous studies were proposed to
estimate the location or velocity of surrounding vehicles using sensor fusion
(78; 34), detect vehicle cutting in, lane change intention, and overtaking to
avoid accidents (32; 41).

Second, to address the ADAS failures caused by environmental factors, some
researchers proposed machine learning models and algorithms to assist the
sensors to make reasonable predictions. For example, Tumas et al. (61)
proposed a deep learning model to fix sensor distortion under severe weather
conditions. Studies also aimed at classifying the road surface condition in
order to improve the performance of Electronic Stability Control (ESC), ACC,
and AEB systems (38; 53). Han et al. (23) developed a deep learning
segmentation algorithm to recognize irregular road areas. In order to improve
lane detection, many researchers proposed algorithms to deal with unclear or
missing lane markings (54; 69; 58; 30). Teo et al. (58) proposed a Gabor
filtering-based algorithm that efficiently detected fading lane boundaries.
Studies were also proposed to improve the robustness of lane detection systems
under various scenarios, such as rainy or poor lighting conditions (60; 13;
39). Another important stream of work was to improve algorithm performance in
poor light conditions, such as during nighttime or in areas with insufficient
light. Such deep learning algorithms and methods have been utilized to improve
the system performance for pedestrian detection, vehicle detection, rear lamp
tracking, and so on (45; 25; 57).

Third, the solutions proposed to deal with the ADAS problems related to
vehicle factors can be categorized into “Hardware” and “Software”. From the
hardware perspective, the literature were trying to improve the performance of
sensors by utilizing sensor fusion and other imaging techniques. Gao et al.
(21) proposed a radar-imaging technique (MIMO-SAR) to improve the resolution
problems from the side view. Another technique related to millimeter-wave
frequency modulated continuous wave radars was proposed with an adaptive
interference detection and suppression scheme, which could improve the signal
to noise ratio under multiple interference environments (62). Sensor fusion is
another commonly applied approach to achieve more accurate and reliable
performances of sensors. Kotur et al. (35) designed deep learning methods to
integrate LiDAR points with camera images to improve the accuracy of collision
detection and reduce the time for calculation. An coordinate calibration
algorithm was proposed by Kim et al (33). to fuse the radar data and vision
data, and a better object detection result could be achieved. From the
perspective of software, the goal of the solutions was focusing on improve the
ADAS performance by software methods and algorithms. Vermiglio et al. (63)
proposed an approach to reduce the computational cost in the system by using
reduced order modeling. To deal with possible Sybil attack faced by modern
vehicles equipped with ADAS sensors, Lim et al. (40) designed a scheme of
multi-step verification process to improve the performance of sybil attack
detection.

##  3\. Cause-and-Effect Analysis

###  3.1. Datasets

We used the dataset from the NHTSA Office of Defects Investigation (ODI)
Consumer Complaint database (46), which includes records of all safety-
relevant complaints about vehicles received by NHTSA since January 1, 1995.
Complaints included in this project were between October 27, 2015 and May 1,
2021. As for the manufacturer name, the complaints were related to the vehicle
manufacturers such as BMW, Mercedes, Porsche, GM, Ford, Honda, Toyota, KIA,
Hyundai, Nissan, Subaru, Chrysler, Volkswagen, and Mitsubishi. The total
number of complaints in the specified range was 440,999. The database
consisted of 49 variables detailing vehicle information, general vehicle
component conditions, complaint/incident description, and vehicle purchasing
history. The primary variable utilized in the study was complaint description,
which was an account of the complaint in the consumer’s own words. As some
complaints described multiple defective components, the database contained
duplicate observations only differed by the defective component description
variable.

###  3.2. ADAS Classification

To examine the cause-and-effect relationship in the the ADAS complaints, we
applied Bidirectional Encoder Representations from Transformers (BERT) (20).
To label the dataset used to train and test the classifier models, we firstly
identified a list of keywords based on the literature review and by manually
checking the complaints to identify the most occurring ADAS related keywords.
For instance, a complaint was classified as ADAS related if it contained any
of the keywords shown in Table 1. The number of ADAS complaints identified was
1141 and the number of non-ADAS complaints was 439,858. The percentage of
complaints containing the identified keywords is shown in Table 1. We used a
stratified 10-fold cross validation to train and test the classification
models.

Table 1. Percentage of complaints containing the identified keywords Keywords | Percentage
---|---
automatic braking, emergency braking system, intelligent braking system, forward emergency braking | 23.84
adaptive cruise control, super cruise, smart cruise control, dynamic cruise control | 16.56
lane keep assist | 16.56
automatic steering, steering assist | 12.80
autopilot, pilot assist | 12.36
forward collision mitigation system, forward collision avoidance, forward collision alert, rear cross traffic warning | 7.89
blind spot assist, blind spot collision warning | 1.49
pedestrian detection | 0.53
driver monitoring | 0.35
advanced driver assistance system | 0.18
automated driving | 0.18

BERT classifier: BERT achieved state-of-the-art results in text classification
tasks. It was pre-trained on a large number of unlabeled data extracted from
BooksCorpus (i.e., 800M words) and English Wikipedia (i.e., 2500M words) (20).
The BERT model consists of transformer encoders (i.e., read the text) and
decoders (i.e., produce a prediction). The general architecture of preparing
the input data for BERT transformer is summarized as follows (6): 1) the
complaint sentences were tokenized and passed through a token embedding layer
to transform them into a vector representation of fixed dimension (i.e.,
768-dimensional vector), 2) to help the model distinguish between two
sentences, extra classification [CLS] and separator [SEP] tokens were added to
the start and end of the tokenized sentence, 3) a sentence embedding was added
to each token, which helped in classifying a text given a pair of input texts,
4) a position embedding was added to each token to indicate its relative
position in a sentence using a sinusoidal function, 5) the final input
embedding was a summation of the three embeddings. The summed input was passed
to the transformer. In this study, we used the PyTorch-Pretrained-BERT library
to build the BERT model. Then, we fine-tuned its linear layer and the sigmoid
activation to obtain the predictions with the labeled dataset. During the
fine-tuning process, Adam optimizer was used with a learning rate of $3\times
10^{-6}$ and a batch size of 4. We fine-tuned the model on the collected
dataset for eight epoch.

For the assessment of the classification models, the performance was evaluated
with precision, recall, and F1-score. Precision represents the proportion of
positive samples that were correctly classified to the total number of
positive predicted samples, i.e.,
$Precision=\\#true\\_positive/(\\#true\\_positive+\\#false\\_positive)$.
Recall represents the positive correctly classified samples to the total
number of positive samples (i.e., ADAS data), i.e.,
$Recall=\\#true\\_positive/(\\#true\\_positive+\\#false\\_negative)$. F1-score
represents the harmonic mean of precision and recall, i.e., F1-score $=2\times
Precision\times Recall/(Precision+Recall)$ (76).

###  3.3. Causal Model Development

In this part of tasks, we annotated the cause-and-effect relationships about
ADAS by coding 542 sentences. Out of the selected sentences, 150 complaints
with the word “cause”, 150 complaints with the word “due to”, 42 complaints
with the words “trigger, lead, result”, and 200 sentences randomly picked. We
simplified the cause-and-effect tagging format (70), and the dataset was
labeled by three students to increase the reliability of the coding. The
inter-raters reliability was 91.5%. We formulated the problem of cause-and-
effect relationship extraction as a sequence labeling problem and we used
three labels to annotate a sentence, i.e., O: others, C: Cause, E: Effect (see
Figure 1) for an example.

Figure 1. Example of cause-and-effect annotation as a sequence labeling problem Table 2.  Tags distribution in the dataset Tags | Meaning | Number
---|---|---
E | Effect token | 7161
C | Cause token | 5601
O | Other token | 73788

In addition to the manually labeled data, we added two publicly available
datasets. One was retrieved from the California Department of Motor Vehicles
(CA DMV) (2) and it was labeled by Zhang et al. (70) to include cause and
effect. The second dataset was a task specific dataset (SemEval-2010 Task 8)
(24). The three datasets (2567 sentences) (see Table 4) were combined then
they were used as the ground truth to train the natural language processing
model to extract cause-and-effect relations.

Recent developments in natural language processing (NLP) have made substantial
progress by using a large amount of unlabeled pre-trained data with a small
number of labeled data for specific NLP tasks. In this work, we used the MPNet
model (55), which is based on permuted language modeling and it uses the
auxiliary position information to make the model see a full sentence. Results
showed that it performed substantially better than previous models (e.g., BERT
(20) and ELECTRA (17)) with the same computation budget. Therefore, we fine-
tuned the MPNet model with labeled data obtained from the previous step to
predict the remaining data. After we constructed the MPNet model, we compared
its performance with ELECTRA and BERT models using various performance
metrics, including precision, recall, F1 measure, evaluation loss, and run
time.

##  4\. Results

###  4.1. Classification Results

A summary of the obtained classification results is shown in Table 3. Since
the dataset was imbalanced, we varied the number of Non-ADAS complaints. In
the first trial, we used the same number of ADAS and Non-ADAS complaints. In
the second trial, we doubled the number of Non-ADAS complaints. And in the
third trial, we tripled the number of Non-ADAS complaints. A ten-fold-
stratified-cross-validation process was used to obtain the results as shown in
Table 3. The BERT model had slightly better performance when more non-ADAS
examples were included, with f1-score = 0.988.

Table 3.  Summary of BERT classifiers performance Sample size | Precision | Recall | F1-score
---|---|---|---
ADAS (1141), Non-ADAS (1141) | 0.985 | 0.985 | 0.985
ADAS (1141), Non-ADAS (2282) | 0.986 | 0.992 | 0.988
ADAS (1141), Non-ADAS (3423) | 0.985 | 0.992 | 0.988

###  4.2. Cause-and-Effect Model Performance and Results

The performance of the MPNet model was compared to BERT and XLNet since MPNet
was built based on these two models. The comparison results of MPNet, ELECTRA,
and BERT classification models, including precision, recall, F1 measure,
evaluation loss, and run time, are shown in Table 4 using a five-fold cross
validation strategy. Precision evaluates the fraction of correctly classified
instances among the one classified as positive; it changes from 0 to 1. Recall
evaluates the fraction of correctly classified instances among all possible
positive predictions and it changes from 0 to 1. F1-score is defined as the
harmonic mean between precision and recall. It is the most used metric when
learning from imbalanced data, it ranges from 0 to 1. For precision, recall,
and F1-score, a score of 1 represents a perfect model. Evaluation loss or
binary cross entropy loss measures how well the classification model performs,
it ranges from 0 to 1 with 0 being a perfect model. The MPNet model had a
slightly better performance across all metrics than the ELECTRA and BERT
models.

Table 4. Summary of model performance Models | Precision | Recall | F1-score | Loss | Run time
---|---|---|---|---|---
MPNet | 0.838 | 0.847 | 0.842 | 0.381 | 41 mins
Electra | 0.835 | 0.847 | 0.841 | 0.405 | 46 mins
BERT | 0.852 | 0.847 | 0.839 | 0.571 | 40 mins

One example of the predicted tags for the ADAS complaints is shown below: _…
tl* the contact owns a 2016 Tesla model s. the contact stated that while
driving at 75 mph, the auto pilot engaged on its own as a result, the contact
crashed into a barrier wall as he was able to place the vehicle back into
manual mode…_
Cause: “the auto pilot engaged on its own and the contact was unable to turn
it off ”.
Effect: “the contact crashed into a barrier wall”.

###  4.3. Taxonomy of Causes and Effects Related to ADAS

####  4.3.1. Categories of ADAS, Causes, and Effects

Based on the results from the causal model, we manually went through the
filtered dataset with 1141 complaints to identify the causes and effects of
different ADAS systems.

First, we identified eight major ADAS categories, including, 1) Emergency
braking system (24%), 2) Adaptive cruise control (23.5%), 3) Lane keep assist
(16.7%), 4) Automatic steering (12.8%), 5) Autopilot (12.7%), 6) Forward
collision avoidance (87%), 7) Blind spot assist (1.5%), and 8) Others (1.2%).
Among them, others included pedestrian detection (0.4%), driver monitoring
(0.2%), advanced driver assistance system (0.2%, which did not specify the
exact ADAS type), rear cross traffic warning (0.3%), and super cruise (0.2%).

In the second step, we identified different categories of causes below: 1)
Vehicle factors (71.2%, including battery, powertrain, wiring issue, data
storage issue, over correction, false alarm, windshield crack, automatic
engagement, suspension failure, infotainment defect, automatic disengagement,
brake failure, over sensitive, main control unit issue, others, recognition
error, inattentive driving, electric system issue, steering issue, software
issue), 2) Unknown causes (22.6%), 3) Environment (5.0%, road conditions and
weather), and 4) human factors (1.2%, false advertising, other vehicles).

We then examined the detailed categories related to these major categories as
shown in Figure 2a. The top five causes were unknown (22.6%), false alarm
(17.3%), fail to respond (13.1%), recall (6.2%), and sensor issue (5.5%). Note
when we labeled these categories, we showed the main differences between them
below.

  1. (1)

False alarm means that there was nothing on the road, but the vehicle suddenly
braked (e.g., _“on multiple occasions while using the adaptive cruise control
the vehicle will suddenly and unexpectedly apply the brakes when the path is
clear in front of the vehicle”_);

  2. (2)

Recognition error means that the vehicle recognizes A as B (e.g., “ _while
using adaptive cruise control the car will inexplicably break when it sees
shadows on the road…_ ”);

  3. (3)

Sensor issues mainly include sensor (camera or radar) obstruction or general
sensor issues without specification (e.g., “ _the adaptive cruise control
radar sensor was found crack after being taken to the dealership for error
code_ ”);

  4. (4)

Fail to respond means that when there were objects with risks to the vehicle,
the vehicle’s ADAS did not respond (e.g., “ _the forward collision avoidance
system was inoperable. the system does not respond to speed limits, does not
alarm, does not display pictures, and does not apply any braking force to
avoid a crash…_ ”);

  5. (5)

Other vehicles include the driving behavior of others causing the issues
(e.g., “ _i was driving my 2018 yukon denali on a 3 lane interstate with gm’s
new adaptive cruise control turned on.. another driver cut in front of me from
the right. adaptive cruise slammed on the brakes almost causing a rear-end
accident by overcompensating…_ ”);

  6. (6)

Data storage issues mostly associated with issues of embedded MultiMediaCard
(emmc) (e.g., “ _the emmc flash memory is failing, leading to the main screen
blacking out for several minutes both while driving and when at stop and when
the car is booting up. this prevents adjustment of several components
including climate (fans, defog, ac), manual lighting, cruise
control/autopilot, etc.”)_ ”);

  7. (7)

Inattentive driving includes mostly handoff driving from Tesla vehicles (e.g.,
“ _he is not taking the advice to hold the wheel seriously and his placing
lives at risk by his reckless behavior…_ ”);

  8. (8)

Others include causes that were less than 5 cases, including engine, fuel
pump, leaking-control box, leaking-cylinder, leaking-oil, mirror issue, remote
start, security design, speed adjuster, switch, transmission problem,
unintended acceleration, unintended deceleration, and unintended lane change.

Thirdly, we identified different categories of effects of these causes as
shown in Figure 2b. The top five effects were 1) ADAS failure (18.8%), hard
braking (13.0%), warning alert (10.5%), collision (9.2%), and near collision
(5.9%). We also explained the main differences among categories when we
labeled them as follows:

(a)  (b)

Figure 2. Percentages of (a) causes and (b) effects of ADAS failures.

  1. (1)

Collision was less severe than crash, such as “ _i had to merge into traffic
from the hov lane disengaging the cruise control. i then used the auto
transmission to shift to a lower gear to gain more torque in changing lanes
when the car went into a stall and lost all power. consequently we were rear-
ended by another car._ ”

  2. (2)

Crash was more severe than collision, such as “ _he credits the tesla with
saving his life but does not concede that the crash might have been caused by
a vehicle fault… both front and rear wheels have been torn off the car which
may have been the result of the crash …_ ”

  3. (3)

Crash with injury or death was the most severe, such as “ _…lane assist jerked
the wheel as i was approaching the car in front of me and caused me to go into
the other lane and and then it resumed speed because it did not realize a
vehicle was coming the other way. at this time i had no control of the car. by
the time i could get back over and got control of the vehicle it was to late
and i hit someone and they died._ ”

  4. (4)

Loss of control is more general from the description than hard braking or
swerving which were explicitly mentioned, such as “ _… this even happens when
” the lane keep assist” is switched off. it is a constant struggle to keep it
from drifting into the right lane and other cars… the pull is so strong that
it makes the lane keep assist system (lkas) unusable. when using lkas on
cruise control the vehicle keeps drifting into the shoulder of the road and
even crosses the line._ ” One example of hard braking was, “ _i was driving
home… a message on the display stated that a crash was imminent. then the
brakes engaged suddenly, and the car almost came to a complete stop in the
middle of the street. the brakes engaged so hard and so quickly, that it threw
me forward in the seat._ ” One example of swerving was “ _when this happened
on highways while using autopilot (experienced at least three times), the car
swerved left and right erratically (after changing lanes) in an effort to re-
center on the lane without success, and i had to take over manually in over to
avoid from flipping over or an accident._ ”

  5. (5)

Stalling (not drivable but with infotainment on) was different from vehicle
complete failure which lost all power of the vehicle. One example of stalling
was “ _vehicle is stalling at lights. vehicle has rough idle. vehicle
automatic braking activate without warning._ ” One example of vehicle complete
failure was “ _2017 honda crv parked in garage, when attempting to start
engine dash lights up warnings on braking system, cruise control, lane assist,
battery and many other areas. car would not start and goes dead with 3rd
attempt to start._ ”

  6. (6)

Safety hazard is used when it was explicitly mentioned while unknown did not
mention any results of the complaint. One example of safety hazard was “ _van
has just over 5k miles… primary warning was ’low brake power’ and braking
became extremely touchy as it seemed to utilize the emergency brake as a
backup. this extreme braking is hazardous and could result in an accident
caused by braking too quickly - with a possible rear-end collision by traffic
to the rear._ ”

####  4.3.2. Causes to ADAS issues

By examining the cause and effect pairs associated with each ADAS issue, we
were able to identify the most important causes to specific ADAS issues and
their associated effects.Table 6 shows the top five causes and top five
effects to each ADAS category. For example, the top five causes of forward
collision avoidance were recall (41.4%), false alarm (18.4%), unknown (14.9%),
fail to respond (11.5%) and sensor issue (5.7%), which accounted for 91.9% of
the cases. For example, the top five effects of forward collision avoidance
were repair unavailable (27.6%), hard braking (18.4%), ADAS failure (11.5%),
warning alert (11.5%) and unintended braking (8.0%), which accounted for 77.0%
of the cases. We were also able to identify the top 10 cause-effect pairs by
aggregating different ADAS categories in Table 5.

Table 5. Top 10 causes to effects by aggregating different ADAS categories  Cause | Effect | #Complaints | %Complaints
---|---|---|---
unknown | ADAS failure | 87 | 7.6%
false alarm | hard braking | 85 | 7.4%
fail to respond | collision | 46 | 4%
unknown | warning alert | 40 | 3.5%
recall | repair unavailable | 24 | 2.1%
false alarm | unintended braking | 22 | 1.9%
false alarm | warning alert | 21 | 1.8%
inattentive driving | collision | 19 | 1.7%
unknown | error message | 18 | 1.6%
sensor issue | ADAS failure | 17 | 1.5%
Table 6. Top 5 causes and effects of the identified ADAS categories with their
percentages

ADAS category | Top five causes | Top five effects
---|---|---
Forward collision Avoidance |  Recall (41.4), false alarm (18.4), unknown (14.9), fail to respond (11.5), sensor issue (5.7) |  Repair unavailable (27.6), hard braking (18.4), ADAS failure (11.5), warning alert (11.5), unintended braking (8)
Emergency braking system |  False alarm (45.3), fail to respond (13.5), sensor issue (12), unknown (10.9), over sensitive (7.3) |  Hard braking (28.9), warning alert (15.7), collision (11.7), ADAS failure (8.4), unintended braking (7.3)
Automatic steering |  Unknown (45.9), recall (15.8), steering issue (11), electric system issue (6.8), weather (5.5) |  ADAS failure (40.4), error message (20.5), warning alert (11.6), loss of control (6.8), ADAS unavailable (4.8)
Autopilot |  Fail to respond (21.4), inattentive driving (19.3), unknown (13.1), main control unit issue (11.7), software issue (8.3) |  Collision (31), ADAS unavailable (13.1), ADAS failure (9.7), loss of control (6.9), hard braking (6.2)
Blind spot assist |  Unknown (23.5), others (17.6), battery (11.8), automatic disengage (5.9), electric system issue (5.9) |  ADAS failure (41.2), warning alert (17.6), safety hazard (11.8), collision (5.9), error message (5.9)
Lane keep assist |  Unknown (24.7), fail to respond (11.6), recognition error (8.9), steering issue (7.9), false alarm (5.8) |  ADAS failure (21.6), loss of control (14.7), swerving (12.1), warning alert (11.6), distrust (6.8)

##  5\. Discussions

This study investigated the issues and solutions of ADAS failures in the
literature and identified the causes and effects from a consumer complaint
dataset. The key findings of both efforts are highlighted below.

Major ADAS issues. We found three main categories of factors that were
associated with ADAS issues in both efforts of this study, including human
factors, environmental, and vehicle factors. In the literature review, we
found that ADAS issues were mostly related to human factors (48.1%), followed
by vehicle factors (26.0%) and environmental factors (25.9%). Thus, many
solutions were proposed with regard to human factors solutions. However, in
the consumer complaint dataset, we found that 71.2% of the complaints were
associated with vehicle, followed by unknown factors (22.6%), environmental
factors (5.0%), and human factors (1.2%). The differences in terms of the
distributions of causes showed different possible biases in self-reported
complaint data that humans tend to complain more about external factors rather
than internal factors (i.e., human factors). Even though, human factors issues
were reported in the consumer complaint dataset, they were related to false
advertising and other vehicles. Nevertheless, we were able to identify a
variety of causes of ADAS related to vehicles in the consumer complaint
dataset (see Figure 2a) with top five causes: false alarm (17.3%), fail to
respond (13.1%), recall (6.2%), and sensor issue (5.5%), and software issues
(3.2%). We found the environmental factors played key roles in leading to ADAS
issues, including road and lane conditions, weather and lighting, which seemed
to be consistent between the two sources. At the same time, we were able to
identify the leading causes of individual ADAS systems through the consumer
complaint dataset (see Table 6), which were not available in the literature.
Such findings helped provide more information to understand the possible
solutions to improve ADAS more specifically in reality. In this sense, these
two lines of research tend to complement each other with a better
understanding of the the causes of ADAS.

ADAS solutions and effects. In the literature, researchers were able to
provide a variety number of solutions based on the issues identified. A larger
number of studies focused on developing advanced algorithms to improve the
performance of ADAS that were able to improve driver trust and less vulnerable
to different weather and road conditions (e.g., (26; 19; 29)). Another
important finding was that researchers were also interested in modeling and
monitoring driver behavior to help improve the joint performance of the human-
vehicle system (e.g., (68; 71; 73). However, in the consumer complaint
dataset, we emphasized the possible consequences of the ADAS issues despite
the fact that there were a small number of solutions to ADAS issues provided
by dealers, mechanics, or automotive companies, but these might be temporary
solutions since many ADAS problems were still persistent (e.g., _tl* the
contact owns a 2019 honda cr-v. while the contact was driving approximately 45
mph, another vehicle pausing on the passenger side of the contact’s vehicle
caused the automatic braking system to activate. as a result, the contact’s
vehicle independently and abruptly stopped… the vehicle was taken to first
team Honda where it was diagnosed that the forward collision avoidance sensor
lens was contaminated with road debris. the lens was cleaned and the vehicle
was returned to the contact. the contact stated that the failure recurred
within thirty minutes…_). The effects of ADAS issues identified were either
associated with specific ADAS or specific causes. For example, Table 6 shows
the top effects related to specific ADAS while Table 5 shows the most
frequently cause and effect pairs in the dataset. These insights provided
guideline in prioritization in terms of fixing different issues associated
with ADAS. For example, distrust was one of the top 5 effects associated with
lane keep assist, which indicated an urgent need to improve the performance of
such ADAS in different scenarios.

Natural language processing models. In this study, we made use of the state-
of-the-art pre-trained language models to perform two important tasks, i.e.,
ADAS classification and cause-and-effect relation extraction. We used BERT to
conduct ADAS classification. The model performed well for this task, which
helped us identify the ADAS related consumer complaints. Then we used three
different types of language models for cause-and-effect relation extraction
and it was found that MPNet performed slightly better than ELECTRA and BERT.
This task was much more difficult than ADAS classification. We used labeled
dataset for fine-tuning the language models. However, the labeled datasets
were from different domains. More research is needed to further improve the
performance of the cause-and-effect extraction.

Limitations and future work. The study has some limitations that can be
addressed in future studies. First, the findings from the literature review
and the consumer complaint dataset might be biased toward specific causes of
ADAS issues. The literature review was fairly objective by focusing the
current major ADAS limitations and their potential solutions from research
point of view, but might ignore other issues that would happen in our daily
life. The consumer complaint dataset was more subjective and tended to be
biased toward vehicle factors associated with ADAS causes. Thanks to the
nature of these two sources, the results were complementary to each other.
Future research can be conducted to examine other resources to identify the
causes of ADAS issues and solutions more comprehensively, such as patents.
Second, due to the cost associated with labeling cause-and-effect relations,
we only had a small labeled dataset to fine-tune the pre-trained language
models to extract causes and effects of ADAS in the consumer complaint
dataset. Although our MPNet was able to identify the causes and effects with
f1-score = 0.842, more work is needed to further improve the performance of
cause and effect extraction.

##  6\. Conclusion

In this study, we conducted a comprehensive literature review on the causes
and solutions of ADAS failure and we found that both the causes and solutions
of ADAS can be categorized into human factors, environmental factors, and
vehicle factors. Then we conducted a cause-effect analysis with an NHTSA ODI
Consumer Complaint database using natural language processing techniques based
on deep learning models. Our model was able to identify causes and effects
with f1-score = 84.1%, based on which we identified the causes and effects
associated with different ADAS categories. At the same time, we identified 1)
top 5 causes of each ADAS, 2) top 5 effects of each ADAS, and 3) top 10 most
frequent cause and effect pairs. Then the findings from these two efforts were
compared and discussed to provide possible recommendations to improve the
safety of ADAS.

###### Acknowledgements.

This work was supported by TOYOTA MOTOR ENGINEERING and MANUFACTURING, NORTH
AMERICA, INC.

## References

  * (1)
  * cal (2022) 2022\.  Autonomous vehicles.  https://www.dmv.ca.gov/portal/vehicle-industry-services/autonomous-vehicles/
  * Abraham et al. (2017) Hillary Abraham, Chaiwoo Lee, Samantha Brady, Craig Fitzgerald, Bruce Mehler, Bryan Reimer, and Joseph F Coughlin. 2017.  Autonomous vehicles and alternatives to driving: trust, preferences, and effects of age. In _Proceedings of the transportation research board 96th annual meeting (TRB’17)_.
  * Association et al. (2019) American Automobile Association et al. 2019\.  Automatic emergency braking with pedestrian detection.  _Retrieved January_ 20 (2019), 2020.  https://www.aaa.com/AAA/common/aar/files/Research-Report-Pedestrian-Detection.pdf
  * Ayoub et al. (2021a) Jackie Ayoub, Lilit Avetisyan, Mustapha Makki, and Feng Zhou. 2021a.  An Investigation of Drivers’ Dynamic Situational Trust in Conditionally Automated Driving.  _IEEE Transactions on Human-Machine Systems_ (2021).
  * Ayoub et al. (2021b) Jackie Ayoub, X Jessie Yang, and Feng Zhou. 2021b.  Combat COVID-19 infodemic using explainable natural language processing models.  _Information Processing & Management_ 58, 4 (2021), 102569\.  https://doi.org/10.1016/j.ipm.2021.102569
  * Ayoub et al. (2021c) Jackie Ayoub, X Jessie Yang, and Feng Zhou. 2021c.  Modeling dispositional and initial learned trust in automated vehicles with predictability and explainability.  _Transportation research part F: traffic psychology and behaviour_ 77 (2021), 102–116.
  * BMW ([n.d.]) BMW. [n.d.].  _BMW X3 — Owner’s Manual_.  BMW.
  * Capallera et al. (2019) Marine Capallera, Quentin Meteier, Emmanuel de Salis, Leonardo Angelini, Stefano Carrino, Omar Abou Khaled, and Elena Mugellini. 2019\.  Owner manuals review and taxonomy of ADAS limitations in partially automated vehicles. In _Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications_. 156–164.  https://doi.org/10.1145/3342197.3344530
  * Castelino et al. (2020) Redge Melroy Castelino, Gabriel Passos Moreira Pinheiro, Bruno Justino Garcia Praciano, Giovanni Almeida Santos, Lothar Weichenberger, and Rafael Timóteo De Sousa Júnior. 2020.  Improving the Accuracy of Pedestrian Detection in Partially Occluded or Obstructed Scenarios. In _2020 10th International Conference on Advanced Computer Information Technologies (ACIT)_. IEEE, 834–838.  https://doi.org/10.1109/ACIT49673.2020.9208877
  * Chen et al. (2021) Chongpu Chen, Jianhua Guo, Chong Guo, Chaoyi Chen, Yao Zhang, and Jiawei Wang. 2021\.  Adaptive Cruise Control for Cut-In Scenarios Based on Model Predictive Control Algorithm.  _Applied Sciences_ 11, 11 (2021), 5293.  https://doi.org/10.3390/app11115293
  * Chen et al. (2015) Qiang Chen, Miao Lin, Bing Dai, and Jiguang Chen. 2015\.  Typical pedestrian accident scenarios in China and crash severity mitigation by autonomous emergency braking systems. In _SAE 2015 World Congress Proceedings. SAE International_.  https://doi.org/10.4271/2015-01-1464.
  * Chen et al. (2019) Shizeng Chen, Bijun Li, Yuan Guo, and Jian Zhou. 2019\.  Lane Detection Based on Histogram of Oriented Vanishing Points. In _Asian Conference on Pattern Recognition_. Springer, 3–11.  https://doi.org/10.1007/978-981-15-3651-9_1
  * Chen et al. (2020) Wen-Hui Chen, Chi-Wei Kuan, and Chuan-Cho Chiang. 2020\.  Integrating Multiscale Deformable Part Models and Convolutional Networks for Pedestrian Detection.. In _VEHITS_. 515–521.
  * Cho and Heo (2020) Jiyong Cho and Jeongyun Heo. 2020.  The More You Know, The More You Can Trust: Drivers’ Understanding of the Advanced Driver Assistance System. In _International Conference on Human-Computer Interaction_. Springer, 230–248.  https://doi.org/10.1007/978-3-030-50523-3_16
  * Cicchino (2017) Jessica B Cicchino. 2017\.  Effectiveness of forward collision warning and autonomous emergency braking systems in reducing front-to-rear crash rates.  _Accident Analysis & Prevention_ 99 (2017), 142–152.  https://doi.org/10.1016/j.aap.2016.11.009
  * Clark et al. (2020) Kevin Clark, Minh-Thang Luong, Quoc V Le, and Christopher D Manning. 2020.  Electra: Pre-training text encoders as discriminators rather than generators.  _arXiv preprint arXiv:2003.10555_ (2020).  https://doi.org/10.48550/arXiv.2003.10555
  * Crump et al. (2016) Caroline Crump, David Cades, Benjamin Lester, Scott Reed, Brandon Barakat, Laurene Milan, and Douglas Young. 2016.  Differing perceptions of advanced driver assistance systems (ADAS). In _Proceedings of the Human Factors and Ergonomics Society Annual Meeting_ , Vol. 60. SAGE Publications Sage CA: Los Angeles, CA, 861–865.  https://doi.org/10.1177/1541931213601197
  * Deng et al. (2020) Qi Deng, Maryam Saleh, Foghor Tanshi, and Dirk Söffker. 2020\.  Online intention recognition applied to real simulated driving maneuvers. In _2020 IEEE Conference on Cognitive and Computational Aspects of Situation Management (CogSIMA)_. IEEE, 1–6.  https://doi.org/10.1109/CogSIMA49017.2020.9216115
  * Devlin et al. (2018) Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018.  Bert: Pre-training of deep bidirectional transformers for language understanding.  _arXiv preprint arXiv:1810.04805_ (2018).  https://doi.org/10.48550/arXiv.1810.04805
  * Gao et al. (2021) Xiangyu Gao, Sumit Roy, and Guanbin Xing. 2021.  MIMO-SAR: A Hierarchical High-Resolution Imaging Algorithm for mmWave FMCW Radar in Autonomous Driving.  _IEEE Transactions on Vehicular Technology_ 70, 8 (2021), 7322–7334.  https://doi.org/10.1109/TVT.2021.3092355
  * Goodin et al. (2019) Christopher Goodin, Daniel Carruth, Matthew Doude, and Christopher Hudson. 2019.  Predicting the Influence of Rain on LIDAR in ADAS.  _Electronics_ 8, 1 (2019), 89.  https://doi.org/10.3390/electronics8010089
  * Han et al. (2020) Hsiang-Yu Han, Yu-Chi Chen, Pei-Yung Hsiao, and Li-Chen Fu. 2020\.  Using channel-wise attention for deep CNN based real-time semantic segmentation with class-aware edge information.  _IEEE Transactions on Intelligent Transportation Systems_ 22, 2 (2020), 1041–1051.  https://doi.org/10.1109/TITS.2019.2962094
  * Hendrickx et al. (2019) Iris Hendrickx, Su Nam Kim, Zornitsa Kozareva, Preslav Nakov, Diarmuid O Séaghdha, Sebastian Padó, Marco Pennacchiotti, Lorenza Romano, and Stan Szpakowicz. 2019\.  Semeval-2010 task 8: Multi-way classification of semantic relations between pairs of nominals.  _arXiv preprint arXiv:1911.10422_ (2019).  https://doi.org/10.48550/arXiv.1911.10422
  * Huang et al. (2021b) Danyang Huang, Zhiheng Zhou, Ming Deng, and Zhihao Li. 2021b.  Nighttime vehicle detection based on direction attention network and bayes corner localization.  _Journal of Intelligent & Fuzzy Systems_ Preprint (2021), 1–19.  https://doi.org/10.3233/JIFS-202676
  * Huang et al. (2021a) Xin Huang, Stephen G McGill, Jonathan A DeCastro, Luke Fletcher, John J Leonard, Brian C Williams, and Guy Rosman. 2021a.  CARPAL: Confidence-aware intent recognition for parallel autonomy.  _IEEE Robotics and Automation Letters_ 6, 3 (2021), 4433–4440.  https://doi.org/10.1109/LRA.2021.3068894
  * Jin et al. (2019) Lydia Jin, Brian C Tefft, and William J Horrey. 2019.  Mining consumer complaints to identify unsuccessful interactions with advanced driver assistance systems. In _Proceedings of the 11th International Conference on Automotive User Interfaces and Interactive Vehicular Applications: Adjunct Proceedings_. 71–75.  https://doi.org/10.1145/3349263.3351313
  * Kang and Kum (2020) Daejun Kang and Dongsuk Kum. 2020.  Camera and radar sensor fusion for robust vehicle localization via vehicle part localization.  _IEEE Access_ 8 (2020), 75223–75236.  https://doi.org/10.1109/ACCESS.2020.2985075
  * Khairdoost et al. (2020) Nima Khairdoost, Mohsen Shirpour, Michael A Bauer, and Steven S Beauchemin. 2020.  Real-time driver maneuver prediction using LSTM.  _IEEE Transactions on Intelligent Vehicles_ 5, 4 (2020), 714–724.  https://doi.org/10.1109/TIV.2020.3003889
  * Khan et al. (2020) Hussam Ullah Khan, Afsheen Rafaqat Ali, Ali Hassan, Ahmed Ali, Wajahat Kazmi, and Aamer Zaheer. 2020.  Lane detection using lane boundary marker network with road geometry constraints. In _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_. 1834–1843.
  * Kidd et al. (2017) David G Kidd, Jessica B Cicchino, Ian J Reagan, and Laura B Kerfoot. 2017.  Driver trust in five driver assistance technologies following real-world use in four production vehicles.  _Traffic injury prevention_ 18, sup1 (2017), S44–S50.  https://doi.org/10.1080/15389588.2017.1297532
  * Kim et al. (2021) Dae Jung Kim, Jin Sung Kim, Jin Ho Yang, Seok Cheol Kee, and Chung Choo Chung. 2021.  Lane change intention classification of surrounding vehicles utilizing open set recognition.  _IEEE Access_ 9 (2021), 57589–57602.  https://doi.org/10.1109/ACCESS.2021.3072413
  * Kim et al. (2018) Jihun Kim, Dong Seog Han, and Benaoumeur Senouci. 2018.  Radar and Vision Sensor Fusion for Object Detection in Autonomous Vehicle Surroundings. In _2018 Tenth International Conference on Ubiquitous and Future Networks (ICUFN)_. 76–78.  https://doi.org/10.1109/ICUFN.2018.8436959
  * Kotur et al. (2021a) Mila Kotur, Nemanja Lukić, Momčilo Krunić, and Željko Lukač. 2021a.  Camera and LiDAR Sensor Fusion for 3D Object Tracking in a Collision Avoidance System. In _2021 Zooming Innovation in Consumer Technologies Conference (ZINC)_. IEEE, 198–202.  https://doi.org/10.1109/ZINC52049.2021.9499281
  * Kotur et al. (2021b) Mila Kotur, Nemanja Lukić, Momčilo Krunić, and Željko Lukač. 2021b.  Camera and LiDAR Sensor Fusion for 3D Object Tracking in a Collision Avoidance System. In _2021 Zooming Innovation in Consumer Technologies Conference (ZINC)_. 198–202.  https://doi.org/10.1109/ZINC52049.2021.9499281
  * Kusano and Gabler (2012) Kristofer D Kusano and Hampton C Gabler. 2012.  Safety benefits of forward collision warning, brake assist, and autonomous braking systems in rear-end collisions.  _IEEE Transactions on Intelligent Transportation Systems_ 13, 4 (2012), 1546–1555.  https://doi.org/10.1109/TITS.2012.2191542
  * Kusano and Gabler (2015) Kristofer D Kusano and Hampton C Gabler. 2015.  Comparison of expected crash and injury reduction from production forward collision and lane departure warning systems.  _Traffic injury prevention_ 16, sup2 (2015), S109–S114.  https://doi.org/10.1080/15389588.2015.1063619
  * Lee et al. (2021) Dongwook Lee, Ji-Chul Kim, Mingeuk Kim, and Hanmin Lee. 2021\.  Intelligent tire sensor-based real-time road surface classification using an artificial neural network.  _Sensors_ 21, 9 (2021), 3233.  https://doi.org/10.3390/s21093233
  * Li et al. (2020) Wenhui Li, Feng Qu, Jialun Liu, Fengdong Sun, and Ying Wang. 2020.  A lane detection network based on IBN and attention.  _Multimedia Tools and Applications_ 79, 23 (2020), 16473–16486.  https://doi.org/10.1007/s11042-019-7475-x
  * Lim et al. (2020) Kiho Lim, Tariqul Islam, Hyunbum Kim, and Jingon Joung. 2020\.  A Sybil Attack Detection Scheme based on ADAS Sensors for Vehicular Networks. In _2020 IEEE 17th Annual Consumer Communications & Networking Conference (CCNC)_. 1–5.  https://doi.org/10.1109/CCNC46108.2020.9045356
  * Lin et al. (2020) Huei-Yung Lin, Jyun-Min Dai, Lu-Ting Wu, and Li-Qi Chen. 2020\.  A vision-based driver assistance system with forward collision and overtaking detection.  _Sensors_ 20, 18 (2020), 5139.  https://doi.org/10.3390/s20185139
  * McDonald et al. (2018) Ashley McDonald, Cher Carney, and Daniel V McGehee. 2018\.  Vehicle owners’ experiences with and reactions to advanced driver assistance systems.  (2018).
  * Möller et al. (2018) Dietmar PF Möller, Isabell A Jehle, and Roland E Haas. 2018\.  Challenges for vehicular cybersecurity. In _2018 IEEE International Conference on Electro/Information Technology (EIT)_. IEEE, 0428–0433.  https://doi.org/10.1109/EIT.2018.8500208
  * Mueller et al. (2021) Alexandra S Mueller, Ian J Reagan, and Jessica B Cicchino. 2021\.  Addressing Driver Disengagement and Proper System Use: Human Factors Recommendations for Level 2 Driving Automation Design.  _Journal of Cognitive Engineering and Decision Making_ 15, 1 (2021), 3–27.  https://doi.org/10.1177/1555343420983126
  * Nataprawira et al. (2021) Jason Nataprawira, Yanlei Gu, Igor Goncharenko, and Shunsuke Kamijo. 2021.  Pedestrian detection using multispectral images and a deep neural network.  _Sensors_ 21, 7 (2021), 2536.  https://doi.org/10.3390/s21072536
  * NHTSA (2011) NHTSA. 2011.  _Vehicle Owner’s Complaint Database_.  Retrieved October 7, 2011 from http://www-odi.nhtsa.dot.gov/downloads/
  * Parasuraman and Riley (1997) Raja Parasuraman and Victor Riley. 1997.  Humans and automation: Use, misuse, disuse, abuse.  _Human factors_ 39, 2 (1997), 230–253.  https://doi.org/10.1518/001872097778543886
  * Paul et al. ([n.d.]) A Paul, R Chauhan, R Srivastava, and M Baruah. [n.d.].  Advanced driver assistance systems. 2016.  _SAE Technical Paper_ ([n. d.]).
  * Piccoli et al. (2020) Francesco Piccoli, Rajarathnam Balakrishnan, Maria Jesus Perez, Moraldeepsingh Sachdeo, Carlos Nunez, Matthew Tang, Kajsa Andreasson, Kalle Bjurek, Ria Dass Raj, Ebba Davidsson, et al. 2020\.  Fussi-net: Fusion of spatio-temporal skeletons for intention prediction network. In _2020 54th Asilomar Conference on Signals, Systems, and Computers_. IEEE, 68–72.  https://doi.org/10.1109/IEEECONF51394.2020.9443552
  * Reagan and McCartt (2016) Ian J Reagan and Anne T McCartt. 2016.  Observed activation status of lane departure warning and forward collision warning of Honda vehicles at dealership service centers.  _Traffic injury prevention_ 17, 8 (2016), 827–832.  https://doi.org/10.1080/15389588.2016.1149698
  * Richardson (2001) JH Richardson. 2001\.  Human factors research priorities for ADAS systems: a UK perspective. In _2001 ADAS. International Conference on Advanced Driver Assistance Systems,(IEE Conf. Publ. No. 483)_. IET, 20–24.
  * Roh et al. (2020) Chang-Gyun Roh, Jisoo Kim, and I-Jeong Im. 2020.  Analysis of impact of rain conditions on ADAS.  _Sensors_ 20, 23 (2020), 6720.  https://doi.org/10.3390/s20236720
  * Šabanovič et al. (2020) Eldar Šabanovič, Vidas Žuraulis, Olegas Prentkovskis, and Viktor Skrickij. 2020\.  Identification of road-surface type using deep neural networks for friction coefficient estimation.  _Sensors_ 20, 3 (2020), 612.  https://doi.org/10.3390/s20030612
  * Seo et al. (2021) J Seo, S Oh, and YK Kim. 2021.  A study of curved lane detection based on dual sensor monitoring of LiDAR and camera.  _Transaction of the Korean Society of Automotive Engineers_ 29, 2 (2021), 197–204.
  * Song et al. (2020) Kaitao Song, Xu Tan, Tao Qin, Jianfeng Lu, and Tie-Yan Liu. 2020.  Mpnet: Masked and permuted pre-training for language understanding.  _Advances in Neural Information Processing Systems_ 33 (2020), 16857–16867.
  * Stanton and Young (2005) Neville A Stanton and Mark S Young. 2005.  Driver behaviour with adaptive cruise control.  _Ergonomics_ 48, 10 (2005), 1294–1313.  https://doi.org/10.1080/00140130500252990
  * Sun et al. (2021) Haitian Sun, Takumi Nakane, Naidan Zhang, and Chao Zhang. 2021\.  Evolutionary Rear-Lamp Tracking at Nighttime.  _IEEE Access_ 9 (2021), 86667–86676.  https://doi.org/10.1109/ACCESS.2021.3087238
  * Teo et al. (2021) Ting Yau Teo, Ricky Sutopo, Joanne Mun-Yee Lim, and KokSheik Wong. 2021.  Innovative lane detection method to increase the accuracy of lane departure warning system.  _Multimedia Tools and Applications_ 80, 2 (2021), 2063–2080.  https://doi.org/10.1007/s11042-020-09819-0
  * Teoh (2021) Eric R Teoh. 2021\.  Effectiveness of front crash prevention systems in reducing large truck real-world crash rates.  _Traffic injury prevention_ 22, 4 (2021), 284–289.  https://doi.org/10.1080/15389588.2021.1893700
  * Tian et al. (2021) Jun Tian, Shiwang Liu, Xunyu Zhong, and Jianping Zeng. 2021\.  LSD-based adaptive lane detection and tracking for ADAS in structured road environment.  _Soft Computing_ 25, 7 (2021), 5709–5722.  https://doi.org/10.1007/s00500-020-05566-4
  * Tumas et al. (2021) Paulius Tumas, Artūras Serackis, and Adam Nowosielski. 2021\.  Augmentation of Severe Weather Impact to Far-Infrared Sensor Images to Improve Pedestrian Detection System.  _Electronics_ 10, 8 (2021), 934.  https://doi.org/10.3390/electronics10080934
  * Umehira et al. (2020) Masahiro Umehira, Takeo Okuda, Xiaoyan Wang, Shigeki Takeda, and Hiroshi Kuroda. 2020.  An Adaptive Interference Detection and Suppression Scheme Using Iterative Processing for Automotive FMCW Radars. In _2020 IEEE Radar Conference (RadarConf20)_. 1–5.  https://doi.org/10.1109/RadarConf2043947.2020.9266712
  * Vermiglio et al. (2020) Simona Vermiglio, Victor Champaney, Abel Sancarlos, Fatima Daim, Jean Claude Kedzia, Jean Louis Duval, Pedro Diez, and Francisco Chinesta. 2020\.  Parametric Electromagnetic Analysis of Radar-Based Advanced Driver Assistant Systems.  _Sensors_ 20, 19 (Oct 2020), 5686\.  https://doi.org/10.3390/s20195686
  * Wang et al. (2018) Yao Wang, Fangfa Fu, Fengchang Lai, Weizhe Xu, Jinjin Shi, and Jinxiang Wang. 2018\.  Efficient road specular reflection removal based on gradient properties.  _Multimedia Tools and Applications_ 77, 23 (2018), 30615–30631.  https://doi.org/10.1007/s11042-018-6156-5
  * Wang et al. (2020) Ziran Wang, Xishun Liao, Chao Wang, David Oswald, Guoyuan Wu, Kanok Boriboonsomsin, Matthew J Barth, Kyungtae Han, BaekGyu Kim, and Prashant Tiwari. 2020\.  Driver behavior modeling using game engine and real vehicle: A learning-based approach.  _IEEE Transactions on Intelligent Vehicles_ 5, 4 (2020), 738–749.  https://doi.org/10.1109/TIV.2020.2991948
  * Wielitzka et al. (2017) Mark Wielitzka, Matthias Dagen, and Tobias Ortmaier. 2017\.  State and maximum friction coefficient estimation in vehicle dynamics using UKF. In _2017 American Control Conference (ACC)_. IEEE, 4322–4327.  https://doi.org/10.23919/ACC.2017.7963620
  * Winner et al. (2014) Hermann Winner, Stephan Hakuli, Felix Lotz, and Christina Singer. 2014.  _Handbook of driver assistance systems_.  Springer International Publishing Amsterdam, The Netherlands:.
  * Yuan et al. (2021) Guoliang Yuan, Yafei Wang, Jinjia Peng, and Xianping Fu. 2021\.  A Novel Driving Behavior Learning and Visualization Method With Natural Gaze Prediction.  _IEEE Access_ 9 (2021), 18560–18568.  https://doi.org/10.1109/ACCESS.2021.3054951
  * Zhang et al. (2021a) Ronghui Zhang, Yueying Wu, Wanting Gou, and Junzhou Chen. 2021a.  Rs-Lane: a robust lane detection method based on ResNeSt and self-attention distillation for challenging traffic situations.  _Journal of advanced transportation_ 2021 (2021).  https://doi.org/10.1155/2021/7544355
  * Zhang et al. (2021b) Yangtao Zhang, X Jessie Yang, and Feng Zhou. 2021b.  Disengagement Cause-and-Effect Relationships Extraction Using an NLP Pipeline.  _arXiv preprint arXiv:2111.03511_ (2021).  https://doi.org/10.48550/arXiv.2111.03511
  * Zhao et al. (2022) Guangzhe Zhao, Yanqing He, Hanting Yang, and Yong Tao. 2022\.  Research on fatigue detection based on visual features.  _IET Image Processing_ 16, 4 (2022), 1044–1053.  https://doi.org/10.1049/ipr2.12207
  * Zheng and Zhao (2020) Hongyu Zheng and Mingxin Zhao. 2020.  An investigation on coordination of lane departure warning based on driver behaviour characteristics.  _International Journal of Vehicle Autonomous Systems_ 15, 1 (2020), 77–99.
  * Zhou et al. (2020b) Dong Zhou, Hongyi Liu, Huimin Ma, Xiang Wang, Xiaoqin Zhang, and Yuhan Dong. 2020b.  Driving behavior prediction considering cognitive prior and driving context.  _IEEE Transactions on Intelligent Transportation Systems_ 22, 5 (2020), 2669–2678.  https://doi.org/10.1109/TITS.2020.2973751
  * Zhou et al. (2022) Feng Zhou, Areen Alsaid, Mike Blommer, Reates Curry, Radhakrishnan Swaminathan, Dev Kochhar, Walter Talamonti, and Louis Tijerina. 2022\.  Predicting Driver Fatigue in Monotonous Automated Driving with Explanation using GPBoost and SHAP.  _International Journal of Human–Computer Interaction_ 38, 8 (2022), 719–729.  https://doi.org/10.1080/10447318.2021.1965774
  * Zhou et al. (2020a) Feng Zhou, Areen Alsaid, Mike Blommer, Reates Curry, Radhakrishnan Swaminathan, Dev Kochhar, Walter Talamonti, Louis Tijerina, and Baiying Lei. 2020a.  Driver fatigue transition prediction in highly automated driving using physiological features.  _Expert Systems with Applications_ 147 (2020), 113204.  https://doi.org/10.1016/j.eswa.2020.113204
  * Zhou et al. (2011) Feng Zhou, Xingda Qu, Martin G Helander, and Jianxin Roger Jiao. 2011.  Affect prediction from physiological measures via visual stimuli.  _International Journal of Human-Computer Studies_ 69, 12 (2011), 801–819.  https://doi.org/10.1016/j.ijhcs.2011.07.005
  * Zhou et al. (2020c) Feng Zhou, X Jessie Yang, and Xin Zhang. 2020c.  Takeover transition in autonomous vehicles: a YouTube study.  _International Journal of Human–Computer Interaction_ 36, 3 (2020), 295–306.  https://doi.org/10.1080/10447318.2019.1634317
  * Zhu et al. (2019) Bin Zhu, Augusto Ferrante, Johan Karlsson, and Mattia Zorzi. 2019.  Fusion of sensors data in automotive radar systems: A spectral estimation approach. In _2019 IEEE 58th Conference on Decision and Control (CDC)_. IEEE, 5088–5093.  https://doi.org/10.1109/CDC40024.2019.9029655
  * Ziebinski et al. (2016) Adam Ziebinski, Rafal Cupek, Hueseyin Erdogan, and Sonja Waechter. 2016.  A survey of ADAS technologies for the future perspective of sensor fusion. In _International Conference on Computational Collective Intelligence_. Springer, 135–146.  https://doi.org/10.1007/978-3-319-45246-3_13