To avoid accidents, CMVs need greater distance and time to come to a complete stop compared to passenger cars. The braking distance of a truck is even longer when it is hauling a heavy load and/or there are adverse road conditions such as snow, ice or rain, as demonstrated in chapter 4. At the same time, the AEB function requires to operate in the robust detection ranges, so the TTCE1 fot curve saturates at higher velocities of E1 x above 50[km/h] to keep the reaction distance increasing linearly with the rel x and thus optimize the sensitivity and specificity of the AEB. The TTCE1 ref is calculated by the HiL test platform based on synthetic sensor models, whereby the simulation environment is communicating with the embedded ECU for AEB function under real-time conditions. The d E1 TTC[s] represents the subtraction between TTCE1 ref [s] and TTCE1 fot [s] values and is expressed as a numerical value denoting a quantifiable error indicator.

### **6.5 Sensitivity Analysis**

Saltelli et al. [S +08] defines the sensitivity analysis as a study of how the uncertainty in the output of a model or system can be apportioned and allocated to different sources of variation in its inputs. After defining the input variables and model responses, the sensitivity analysis then scans the parameter space with Design of Experiments (DoE) or sampling methods. Ebert et al. [EGK+20] applies the sensitivity analysis to reduce the dimension of the DoE to the most important factors as a part of the engine calibration process within the Real Driving Emissions (RDE) test cycle. Accordingly, the sensitivity analysis can be used to analyze the contribution of each input variable to the scatter of each model response. Each scattering input specifies a distribution type including mean value and variance, whereby the input variables are defined as random variables. The dependencies between the input variables can be formulated in terms of linear correlations. The meta-models are also used to represent the model responses by surrogate functions in terms of the model inputs [BSH17]. Meta-model approaches such as Kriging approximation, Metamodel of Optimal Prognosis (MOP) approximation, Support Vector Regression (SVR) and Artifical Neural Network (ANN) provide an automatic reduction of the variable space. The meta-model applied in this work is based on the anisotropic kriging approach using the Metamodel of Optimal Prognosis (MOP) solver [MW08].

Most et al. defines the coefficient of prognosis as a sensitivity measure to determine the optimal input variable subspace together with the optimal metamodel approach. The sensitivity analysis is applied in the example of the cluster C 1 s at the escalation level E 1 s . The sensitivity measures are determined in step (V) using the sensitivity analysis, whereby the input variables of the approximation model are ( E1 x [km/h],d E1 x [m], d E1 y [m], E1 ego[1/km] and d E1 TTC[s]) and the approximation model output is TTCE1 min [s]. The scalar random variables can be used to model the scattering inputs. The MOP solver reduced the input variables from [ E1 x ,d E1 x ,d E1 y , E1 ego,d E1 TTC] into [ E1 x ,d E1 TTC]. While it is important to note that the variable E1 ego was eliminated by the MOP solver, although curvature could play a role in determining TTCE1 min for such FP events — as shown in the figure 6.4 — , it did not, arguing that correlation does not imply causation.

**Definition 6.1** (**minimum** TTC)**:**It describes the lowest time-to-collision in which an environmental perception sensor reports a tracked object list with objects classified as relevant for the ADF. The minimum TTC relates to the error-object classification in an inverse-proportional sense; the lower the value of TTCE1 min, the more prominent the error in classifying the object for such FP events in the cluster C 1 s [BES+21].

### **6.6 Exploration of Parameter Space**

In general, the mean value and the standard deviation are used to describe the variation of a random variable. However, the shape of the distribution function can have a significant influence on the results of a stochastic analysis. Accordingly, the parameter space exploration requires the definition of distribution types including mean value and standard deviation for all random input variables. The truncated normal distribution is chosen as a suitable distribution type for the measurements of Ego-vehicle curvature E1 ego[1/km], lateral deviation to the stationary object d E1 y [m] and error indicator of environment perception d E1 TTC[s] at the escalation level E 1 s of the cluster C 1 s . The truncated normal distribution is a normal Gaussian distribution that is restricted to lie within a finite range, i.e. [ min ego ≤ E1 ego ≤ max ego ]. The truncated normal distribution can be expressed in terms of the normal Gaussian distribution as follows:

$$f\_{\rm PDF}(\kappa\_{\rm ego}^{\rm C1}, \kappa\_{\rm ego}^{\rm min}, \kappa\_{\rm ego}^{\rm max}, \mu, \sigma) = \frac{1}{\sigma} \frac{\Omega\left(\frac{\kappa\_{\rm ego}^{\rm C1} - \mu}{\sigma}\right)}{\Phi\left(\frac{\kappa\_{\rm ego}^{\rm max} - \mu}{\sigma}\right) - \Phi\left(\frac{\kappa\_{\rm ego}^{\rm min} - \mu}{\sigma}\right)}, \kappa\_{\rm ego}^{\rm min} \le \kappa\_{\rm ego}^{\rm C1} \le \kappa\_{\rm ego}^{\rm max} \tag{6.13}$$

$$f\_{\rm CDF}(\kappa\_{\rm ego}^{\mathcal{E}1};\kappa\_{\rm ego}^{\min},\kappa\_{\rm ego}^{\max},\mu,\sigma) = \frac{\Phi(\frac{\kappa\_{\rm ego}^{\mathcal{E}1}-\mu}{\sigma}) - \Phi(\frac{\kappa\_{\rm ego}^{\min}-\mu}{\sigma})}{\Phi(\frac{\kappa\_{\rm ego}^{\max}-\mu}{\sigma}) - \Phi(\frac{\kappa\_{\rm ego}^{\min}-\mu}{\sigma})},\kappa\_{\rm ego}^{\min} \le \kappa\_{\rm ego}^{\mathcal{E}1} \le \kappa\_{\rm ego}^{\max} \tag{6.14}$$

where and denote the mean and standard deviation of the parent normal distribution and min ego= −4.5[1/km] and max ego = 3.9[1/km] as the lower and upper truncation points respectively.

The Ω and Φ designate the Probability Distribution Function (PDF) and the Cumulative Distribution Function (CDF) for the normal Gaussian distribution respectively. Figure 6.6 illustrates the PDF and CDF estimations using the truncated normal distribution. Equations 6.15 and 6.16 estimate the mean and standard deviation of the truncated normal distribution.

Figure 6.6: Estimation of the PDF and CDF of the Ego-vehicle curvature E1 ego[1/km] (top) and the lateral deviation to the stationary object d E1 y [m] (bottom) at the escalation level E 1 s of the cluster C 1 s .

$$\mu\_{\kappa} = \mu + \sigma \ast \frac{\Omega(\frac{\kappa\_{\rm cgo}^{\rm min} - \mu}{\sigma}) - \Omega(\frac{\kappa\_{\rm cgo}^{\rm max} - \mu}{\sigma})}{\Phi(\frac{\kappa\_{\rm cgo}^{\rm max} - \mu}{\sigma}) - \Phi(\frac{\kappa\_{\rm cgo}^{\rm min} - \mu}{\sigma})} \tag{6.15}$$

$$
\sigma\_{\kappa} = \sigma \ast \sqrt{1 + \frac{(\frac{\kappa\_{\rm{cph}}^{\rm{min}} - \mu}{\sigma}) \mathfrak{Q} (\frac{\kappa\_{\rm{cph}}^{\rm{min}} - \mu}{\sigma}) - (\frac{\kappa\_{\rm{cph}}^{\rm{max}} - \mu}{\sigma}) \mathfrak{Q} (\frac{\kappa\_{\rm{cph}}^{\rm{max}} - \mu}{\sigma})}{\Phi (\frac{\kappa\_{\rm{cph}}^{\rm{min}} - \mu}{\sigma}) - \Phi (\frac{\kappa\_{\rm{cph}}^{\rm{min}} - \mu}{\sigma})} - \left[ \frac{\mathfrak{Q} (\frac{\kappa\_{\rm{cph}}^{\rm{min}} - \mu}{\sigma}) - \mathfrak{Q} (\frac{\kappa\_{\rm{cph}}^{\rm{max}} - \mu}{\sigma})}{\Phi (\frac{\kappa\_{\rm{cph}}^{\rm{min}} - \mu}{\sigma}) - \Phi (\frac{\kappa\_{\rm{cph}}^{\rm{min}} - \mu}{\sigma})} \right]^2 \tag{6.16}
$$

The same calculations are applied for the lateral deviation to the stationary object d E1 y [m], where d min y = −2.6[m] and d max y = 0.8[m] as the lower and upper truncation points respectively. The false warnings from the cluster C 1 s are presented as an example for the sensitivity analysis. Accordingly, the sensitivity measures are analyzed to define the important input variables in relation to the response variable TTCE1 min[s].

The triangular distribution is chosen as a suitable distribution type for the measurements of the Ego-vehicle longitudinal velocity E1 x [km/h] and the minimum time to collision TTCE1 min[s] at the escalation level E 1 s of the cluster C 1 s . The following equations calculate the PDF, CDF, mean and standard deviation of the Ego-vehicle longitudinal velocity E1 x [km/h] based on the triangular distribution respectively, where min x = 7.9[km/h], max x = 83.4[km/h] and peak x = 59.6[km/h] as the lower limit, upper limit and mode respectively. Equations 6.19 and 6.20 estimate the mean and standard deviation of the triangular distribution of the E1 x [km/h].

$$f\_{\rm PDF}(\boldsymbol{\nu}\_{\rm X}^{\mathcal{E}1};\boldsymbol{\nu}\_{\rm X}^{\min},\boldsymbol{\nu}\_{\rm X}^{\max},\boldsymbol{\nu}\_{\rm X}^{\text{peak}}) = \begin{cases} \frac{2\left[\boldsymbol{\nu}\_{\rm X}^{\mathcal{E}1} - \boldsymbol{\nu}\_{\rm X}^{\min}\right]}{\left[\boldsymbol{\nu}\_{\rm X}^{\max} - \boldsymbol{\nu}\_{\rm X}^{\min}\right]\left[\boldsymbol{\nu}\_{\rm X}^{\text{peak}} - \boldsymbol{\nu}\_{\rm X}^{\min}\right]}, & \forall \boldsymbol{\nu}\_{\rm X}^{\mathcal{E}1} \le \boldsymbol{\nu}\_{\rm X}^{\mathcal{E}1} \le \boldsymbol{\nu}\_{\rm X}^{\text{peak}} \\\\ \frac{2\left[\boldsymbol{\nu}\_{\rm X}^{\max} - \boldsymbol{\nu}\_{\rm X}^{\mathcal{E}1}\right]}{\left[\boldsymbol{\nu}\_{\rm X}^{\max} - \boldsymbol{\nu}\_{\rm X}^{\min}\right]\left[\boldsymbol{\nu}\_{\rm X}^{\max} - \boldsymbol{\nu}\_{\rm X}^{\text{peak}}\right]}, & \forall \boldsymbol{\nu}\_{\rm X}^{\mathcal{E}1} < \boldsymbol{\nu}\_{\rm X}^{\mathcal{E}1} \le \boldsymbol{\nu}\_{\rm X}^{\max} \end{cases} \tag{6.17}$$

$$f\_{\rm CDF}(\boldsymbol{\nu}\_{\rm X}^{\mathcal{E}1};\boldsymbol{\nu}\_{\rm X}^{\min},\boldsymbol{\nu}\_{\rm X}^{\max},\boldsymbol{\nu}\_{\rm X}^{\text{peak}}) = \begin{cases} \frac{\left\{\boldsymbol{\nu}\_{\rm X}^{\mathcal{E}1} - \boldsymbol{\nu}\_{\rm X}^{\min}\right\}^{2}}{\left\{\boldsymbol{\nu}\_{\rm X}^{\max} - \boldsymbol{\nu}\_{\rm X}^{\min}\right\}\left[\boldsymbol{\nu}\_{\rm X}^{\text{peak}} - \boldsymbol{\nu}\_{\rm X}^{\min}\right]}, & \forall \boldsymbol{\nu}\_{\rm X}^{\mathcal{E}1} \le \boldsymbol{\nu}\_{\rm X}^{\mathcal{E}1} \le \boldsymbol{\nu}\_{\rm X}^{\text{peak}} \\\\ 1 - \frac{\left[\boldsymbol{\nu}\_{\rm X}^{\max} - \boldsymbol{\nu}\_{\rm X}^{\mathcal{E}1}\right]^{2}}{\left[\boldsymbol{\nu}\_{\rm X}^{\max} - \boldsymbol{\nu}\_{\rm X}^{\min}\right]\left[\boldsymbol{\nu}\_{\rm X}^{\max} - \boldsymbol{\nu}\_{\rm X}^{\text{peak}}\right]}, & \forall \boldsymbol{\nu}\_{\rm X}^{\mathcal{E}1} < \boldsymbol{\nu}\_{\rm X}^{\mathcal{E}1} \le \boldsymbol{\nu}\_{\rm X}^{\max} \end{cases} \tag{6.18}$$

$$
\mu\_{\nu} = \frac{1}{3} \left[ \nu\_{\text{x}}^{\text{min}} + \nu\_{\text{x}}^{\text{max}} + \nu\_{\text{x}}^{\text{peak}} \right] \tag{6.19}
$$

$$
\sigma\_{\nu} = \sqrt{\frac{[\nu\_{\text{X}}^{\text{min}}]^2 + [\nu\_{\text{X}}^{\text{max}}]^2 + [\nu\_{\text{X}}^{\text{peak}}]^2 - [\nu\_{\text{X}}^{\text{min}} \ast \nu\_{\text{X}}^{\text{max}}] - [\nu\_{\text{X}}^{\text{min}} \ast \nu\_{\text{X}}^{\text{peak}}] - [\nu\_{\text{X}}^{\text{max}} \ast \nu\_{\text{X}}^{\text{peak}}]}{18} \tag{6.20}
$$

The same calculations are applied for the minimum time to collision TTCE1 min[s] on the basis of the triangular distribution function. Figure 6.7 shows the PDF and CDF estimations of the E1 x [km/h] using the triangular distribution and the d E1 TTC[s] using the truncated normal distribution. The calculations are applied for the error indicator of environment perception d E1 TTC[s], where the minimum value of d E1 TTC = −1.5[s] and the maximum value of d E1 TTC = 0.14[s] are the lower and upper truncation points respectively.

Figure 6.7: Estimation of the PDF and CDF of the error indicator of environment perception d E1 TTC[s] (top) and the Ego-vehicle longitudinal velocity E1 x [m] (bottom) at the escalation level E 1 s of the cluster C 1 s .

# **7 Reliability Analysis Using Sampling Methods**

The ADFs in a vehicle have to comply with the ISO 26262:2018 standard for functional safety and are not allowed to influence each other negatively. The Regulation No. 79 applies to the steering equipment of CMVs for lane change operations. Meanwhilst, the ACSF of Category E requires minimum monitoring ranges for the front, rear and side detection of automated vehicles [EST+17]. If a motorcycle approaches the Ego-vehicle from behind, the rear detection has to take place early enough to prevent the motorcycle from braking abruptly when the Ego-vehicle changes lanes. If the distance is below a critical range, the lane change request must be suppressed. At the same time, the CMVs include a variety of series and models with tractors, semi-tractors or trailer combinations. Consequently, the tractor-mounted environmental perception sensors cannot provide a full 360° surround-view when a trailer is coupled to the tractor. In Germany, according to Section 4 of the German road traffic regulations, the Ego-vehicle in front must not brake suddenly without a compelling reason. Therefore, compliance with ISO 26262:2018 is achieved if an ADF represents an acceptable residual risk for the subsequent traffic. A probabilistic safety analysis is needed to quantify the uncertainties and to enable a prospective risk assessment for the testing and further development of ADFs [BRW+17].

# **7.1 Probabilistic Safety Assessment**

Following the application of cluster and sensitivity analysis to the real traffic data-set — interpreted in chapters 5 and 6 — , the data is then used to determine the probability distributions of the input variables and the probability of occurrence of the response variables [Mul18].

In accordance with ISO 26262:2018, the reference values of the tolerable risk curve are derived in step (VI) as a confidence level for the optimization quality of ADFs. The measurable safety methodology aims at evaluating the SOTIFor OEDR-related capabilities of ADFs. Accordingly, the reliability analysis aims at identifying the probability of safety violation for each logical scenario [AKK+19]. The parameter space is searched with a suitable sampling method to determine the probability that a predefined safety margin is exceeded [Roo02]. The scattering input variables are considered as random variables within the framework of probabilistic safety analysis. Therefore, the probability of failure Pf is defined as the probability of the event that a random vector *Z* falls into the failure domain. The failure mode involves defining a failure criterion known as the Limit State Function (LSF) or safety margin. A direct formulation of the LSF in dependence of the input random variables is often not possible, but is implicitly given by the simulation results of the investigated system [Bay99]. The LSF forms the basis for the reliability calculations and denoted by g(*Z*). The function expresses Resistance-Load as a function of *Z*, where *Z* is a vector of all uncertainty variables describing the loads and resistances. The failure criterion is consequently defined as g(*Z*) ≤ , where denotes a predefined safety margin. For a given LSF with g(*Z*), the probability of failure Pf is defined as follows:

$$\mathbf{P}\_{\mathbf{f}} = P\{\mathbf{g}(Z) \le \boldsymbol{\gamma}\} = \int \underset{\{\boldsymbol{z} \in \mathbb{R}^{n} | \mathbf{g}(\boldsymbol{z}) \le \boldsymbol{\gamma}\}}{\text{\smalls}} \int f\mathbf{z}(\boldsymbol{z})d\boldsymbol{z} \tag{7.1}$$

where {*z* ∈ R n |g(*z*) ≤ } represents the failure domain, (*z*) designates the realization of *Z* in the variable space and *fZ* (*z*) denotes the joint density function for *Z*. Accordingly, the probability of failure is the integral of the joint PDF over the failure domain. While the probability of failure is the complementary of reliability, the failure criteria do not have to indicate a system breakdown. Such failure criteria can be defined by the violation of quality or safety requirements. The probability integral can be expressed as the expectation of the indicator function *I*(g(*z*)), where *I*(g(*z*))= 1 if g(*z*) ≤ and *I*(g(*z*))= 0 otherwise. The probability integral can be interpreted as the expected value E of the indicator *I*(g(*z*)) as follows:

$$\mathbf{P}\_{\mathbf{f}} = \mathbb{E}[I(\mathbf{g}(z))] = \int\_{-\infty}^{\infty} \cdots \int\_{-\infty}^{\infty} I(\mathbf{g}(z)) f\_{\mathbf{Z}}(z) dz \tag{7.2}$$

While the space of all random variables can be separated into a safe domain and a failure domain, the indicator's expected value E[*I*(g(*z*))] calculates the probability of severity violations within the prospective risk assessment. The reliability analysis identifies the failure region in the parameter space to predict the probability of failure for each logical scenario. While the analytical calculation of expected values E[*I*(g(*z*))] with the dimensional increase of input random variables is no longer suitable for practical use, the Monte Carlo integration approximates the indicator's expectation E[*I*(g(*z*))] by simulation-based approximation. The simulation of rare events approximates the integral solution using Monte Carlo Sampling (MCS) and Adaptive Importance Sampling (AIS) methods [WPZ19]. In compliance with the HARA of the ISO 26262:2018, the failure criteria present the different levels of severity [LPP11], whereby the TTCE1 min[s] defines the different severity levels in an exemplary way.

The TTCE1 min describes the lowest TTC in which an environmental perception sensor reports a tracked object list with objects classified as relevant for the AEB function of the cluster C 1 s at the escalation level E 1 s . The sensitivity analysis defines the variables E1 x [km/h] and d E1 TTC[s] as the most important factors influencing the variable TTCE1 min[s] as a response within the probabilistic safety assessment. The selection is determined by the variables with the greatest impact on the model output TTCE1 min[s]. Accordingly, the reliability analysis is applied to the inputs and response of the cluster C 1 s at the escalation level E 1 s . As discussed in section 5.3.3, the E 1 s events represent phantom warnings, where the situation was safe and a warning was erroneously signaled. The prospective risk assessment R C 1 s is defined as the combination of the probability of occurrence *P*(C1 s ) of harm and the severity of that harm. Table 7.1 shows the severity levels, which are divided into four regions as follows:

Table 7.1: Hypothetical severity levels for object classification error based on the criticality indicator TTCE1 min[s].

Equation 7.3 represents the prospective risk for the cluster C 1 s with the combined curve to the left. In step (VII), the tolerable risk curve is generated as a reference safety threshold with respect to the logical scenarios. Thereby, the logical scenario with combined curve to the left is shown as an example in order to prospectively estimate the risk regarding false alarms on subsequent traffic.

$$\mathcal{R}\left(\mathcal{C}\_{\sf s}^{1}\right) \approx \left[\mathbf{P}\_{\sf f}(\mathcal{S}\_{\sf s}^{\sf S} \mid \mathcal{C}\_{\sf s}^{1}) \ast P(\mathcal{C}\_{\sf s}^{1})\right] + \left[\mathbf{P}\_{\sf f}(\mathcal{S}\_{\sf s}^{2} \mid \mathcal{C}\_{\sf s}^{1}) \ast P(\mathcal{C}\_{\sf s}^{1})\right] + $$

$$\left[\mathbf{P}\_{\sf f}(\mathcal{S}\_{\sf s}^{1} \mid \mathcal{C}\_{\sf s}^{1}) \ast P(\mathcal{C}\_{\sf s}^{1})\right] + \left[\mathbf{P}\_{\sf f}(\mathcal{S}\_{\sf s}^{0} \mid \mathcal{C}\_{\sf s}^{1}) \ast P(\mathcal{C}\_{\sf s}^{1})\right] \tag{7.3}$$

Figure 7.1 represents the PDF and CDF estimations of the minimum time to collision TTCE1 min[s] using the triangular distribution function with the corresponding severity levels. The TTCE1 min[s] is used as a prototypical safety measure for the reliability analysis. The original measurements of TTCE1 min range between 0.03[s] and 2.27[s]. The triangular distribution is selected to approximate the TTCE1 min distribution, where the lower bound of TTCE1 min = −0.3[s] and the upper bound and mode of TTCE1 min = 1.91[s].

Figure 7.1: Estimation of the PDF and CDF of the response variable TTCE1 min[s] of the cluster C 1 s at the escalation level E 1 s .

### **7.2 Monte Carlo Simulation**

The MCS method estimates Pf by generating independent samples from the PDF of *fZ* (*z*) and taking the average of the sample indicator values as an unbiased estimator of the expected probability of failure [Zio13]. While the joint density function *fZ* (*z*) depends on random input variables, the estimator of its probability failure P¯ f is also a random variable [Buc09]. Equation 7.4 represents the probability failure estimator P¯ f as follows:

$$\bar{\mathbf{P}}\_{\rm f} = \bar{\mathbf{E}}[I(\mathbf{g}(z))] = \frac{1}{\mathcal{N}\_{\rm mc}} \sum\_{\mathbf{k}=1}^{\mathcal{N}\_{\rm mc}} I(\mathbf{g}(z\_{\mathbf{k}})) \tag{7.4}$$

The estimator variance Var(P¯ f) can be computed approximately from the generated samples. In accordance with the central limit theorem, the standard deviation converges towards 0 for a large number of samples Nmc→ ∞. If the estimator's variance converges towards 0, the confidence interval of the estimator becomes narrower and thus the estimator gets more confident. The number of samples is used to normalize the standard deviation of the unbiased estimator P¯ f of the target failure probability Pf [P +14]. The normalized standard deviation is used as measure for the completion of the required number of generated samples and is known as standard error eP¯ f .

$$\mathbf{e}\_{\bar{\mathbf{P}}\_{\rm f}} = \sqrt{\frac{\text{Var}(\bar{\mathbf{P}}\_{\rm f})}{\mathbf{N}\_{\rm mc}}} \tag{7.5}$$

The MCS method evaluates the probability of failure by determining whether the LSFs are exceeded. The trial is repeated many times to ensure the convergence of the statistical results. In each trail, a sample value is generated and evaluated as a Hit or Miss (relevant or irrelevant) according to the LSF definition. The MCS requires a large number of samples to accurately predict the probability of failure, especially if the expected value is small. Figure 7.2 illustrates the result of MCS method used to obtain the failure probability of the severity level S 3 s . As shown in figure 7.2, the failure probability P¯ f (S3 s |C1 s ) and the standard error eP¯ f (S3 s |C1 s ) are plotted against the number of samples Nmc.

The MCS sampling is a Hit-or-Miss sampling method. Here, the blue dots indicate the target Hit samples at which the corresponding LSF condition for each severity level is met. Meanwhilst, the yellow spots show the Miss condition, that means they don't fulfill the conditions required for the severity level. The Hitor-Miss MCS method generates Nmc samples with 1000 (assumed budget of samples), where the number of Hits is 99 and the number of Misses is 901. The failure probability P¯ f (S3 s |C1 s ) is calculated to be 99 1000 = 0.1. For this severity level, the Hit samples are very little compared to Miss samples and are located for the lower values of E1 x [km/h] towards the left end of the plot. The target is missed for the values of E1 x higher than 50[km/h] (In which E1 x indicates the velocity at escalation level 1). The footage representing such a scenario cluster 1 — is depicted in chapter 5. The stopping criteria of simulation depend on the comparison between the standard error of the exceedance probability1 eP¯ f (S3 s |C1 s ) = 0.0094 and the target standard error P¯ f = 0.05.

Figure 7.2: Estimation of the failure probability for the severity level S 3 s using the MCS method.

1 The exceedance probability represents a probability of exceeding a certain limit that can be quantified and demonstrated to be less than an acceptable value.

Figure 7.3 shows the failure probability of the severity level S 2 s using the MCS method with a number of samples Nmc = 1800. The number of Hits is 335 and the number of Misses is 1465. Considering the graph for severity level S 2 s as shown in figure 7.3, here, the condition seems to be fulfilled for more numbers of samples. This means that the number of Hit samples is increased and also the value of E1 x [km/h] for which the condition seems to be fulfilled. After the first few samplings, this failure probability value falls and then becomes stable at a level P¯ f (S2 s |C1 s ) = 335 1800 = 0.19. In addition, the standard error becomes stable gradually at the value eP¯ f (S2 s |C1 s ) = 0.0092 to be less than the required standard error P¯ f with 0.05. Ultimately, the standard error does not seem to be entirely nullified, but is really low. It makes a cloud of the Hit samples slightly shifted towards the right side. It is important to note that the figure 7.3 shows the ratio of Hits and their corresponding velocity E1 x [km/h]. It is obvious that as the velocity E1 x [km/h] increases, the sensor error d E1 TTC[s] also increases for the specified criticality range S 2 s .

Figure 7.3: Estimation of the failure probability for the severity level S 2 s using the MCS method.

Figure 7.4 illustrates the failure probability of the severity level S 1 s using the MCS method, with a number of samples Nmc= 500. The number of Hits is 257 and the number of Misses is 243. Here, the Hit condition is improved as compared to the previous severity level. For the severity level S 1 s , the values of E1 x [km/h] have also increased and the cloud of Hit samples moves rightwards again. According to the results of this sampling, the failure probability of this level of severity is high at the beginning of sampling. It falls immediately after some samples and then stabilizes at a certain level of P¯ f (S1 s |C1 s ) = 257 500 = 0.51 with infinitesimal difference in each sampled value. The standard error eP¯ f (S1 s |C1 s ) gives the value 0.022 as abort criterion of the simulation, which is smaller than the target standard error P¯ f .

Figure 7.4: Estimation of the failure probability for the severity level S 1 s using the MCS method.

Figure 7.5 describes the failure probability for the severity level S 0 s using the MCS with a number of samples Nmc = 1600. The number of Hits is 321 and the number of Misses is 1279. This sampling does show the stability in the failure probability with P¯ f (S0 s |C1 s )= 321 1600 = 0.2. The value of the standard error is less than the previous case with eP¯ f (S0 s |C1 s ) = 0.01.

Finally switching to the figure 7.5, the cloud of Hit samples lies to the right side of the graph for the severity level S 0 s . Here, the target Hits are a little less in number compared to severity levels S 1 s and S 2 s . It is significant to note, that with correspondence to figure 7.1 the studied range contains only a small number of events. Thus, rendering this statistic improbable. However, the indication from the graph follow accordingly to the expected flow.

Figure 7.5: Estimation of the failure probability for the severity level S 0 s using the MCS method.

The figures 7.2 to 7.5 illustrate that there is a movement in the position of the target Hits starting from the bottom left moving towards the top right through the severity levels, looking like a cloud of Hits.

# **7.3 Adaptive Importance Sampling**

Since the estimator variance corresponds to the confidence, variance reduction techniques aim at influencing the sampling in such a way that the estimator variance becomes smaller.

The Importance Sampling is a variance reduction method to guide the sampling using biased normal distribution for variance reduction. The estimator includes the Importance Sampling weight to warrant unbiasedness of the estimator. The samples are not generated following the prescribed joint PDF of *fZ* (*z*), but with an importance sampling density denoted as *hY* (*y*). Setting *hY* (*y*)/*hY* (*y*) into the probability integral does not change its value, but the integral provides the expected value.

$$\mathbf{P\_{f}} = \mathbb{E}\left[I(\mathbf{g(y)})\frac{f\_{\mathbf{Z}}(\mathbf{y})}{h\_{Y}(\mathbf{y})}\right] = \int\_{-\infty}^{\infty} \cdots \int\_{-\infty}^{\infty} I(\mathbf{g(y)}) \frac{f\_{\mathbf{Z}}(\mathbf{y})}{h\_{Y}(\mathbf{y})} h\_{Y}(\mathbf{y}) d\mathbf{y} \tag{7.6}$$

$$\bar{\mathbf{P}}\_{\mathbf{f}} = \bar{\mathbb{E}}\left[I(\mathbf{g}(\mathbf{y})) \frac{f\_{\mathbf{Z}}(\mathbf{y})}{h\_{\mathbf{Y}}(\mathbf{y})}\right] = \frac{1}{\mathcal{N}\_{\text{as}}} \sum\_{\mathbf{k}=1}^{\mathcal{N}\_{\text{as}}} I(\mathbf{g}(\mathbf{y}\_{\mathbf{k}})) \frac{f\_{\mathbf{Z}}(\mathbf{y}\_{\mathbf{k}})}{h\_{\mathbf{Y}}(\mathbf{y}\_{\mathbf{k}})} \tag{7.7}$$

The AIS reduces the estimator variance and involves several simulation runs [O +18]. The importance sampling density *hY* (*y*) may have a larger scatter. In the first iteration, the samples falling into the failure domain {*z* ∈ R n |g(*z*) < } : *Z*|g(*z*) ≤ are statistically evaluated. The result is used to define the parameter of normal Gaussian distribution type for the subsequent Importance Sampling iterations. To increase the number of Hits, the AIS method is managed through the use of information about the LSF domain. Each sample is weighted according to the ratio of the original density function *fZ* (*z*) to the importance density function *hY* (*y*) to ensure correct statistics [KS13]. Furthermore, the AIS techniques can be applied to efficiently search for critical scenarios by speeding up the parameter space exploration [O +18]. Equations 7.8 and 7.9 show the expected values of the importance sampling density *hY* (*y*) according to the estimated mean and covariance of the samples in the failure domain in iterative steps [Li07].

$$\mathbb{E}[Y] = \mathbb{E}[Z|\mathbf{g}(z) \le \mathcal{Y}] \tag{7.8}$$

$$\mathbb{E}\{YY^T\} = \mathbb{E}\{ZZ^T|\mathbf{g}(z) \le \boldsymbol{\gamma}\}\tag{7.9}$$

The third iteration should be performed to prove the stability of the results in an iterative adaption scheme. Accordingly, the goal of the importance sampling density *hY* (*y*) is to converge the estimator standard error to zero by adjusting the importance sampling density *hY* (*y*) iteratively [BMC15]. Typically, the target standard error for the termination criteria is set to 0.1, but in order to conduct more samples, the target standard error was set to 0.05. (The value = 0.05 was previously utilized in MCS sampling).

Figure 7.6 shows the estimation of failure probability of the severity level S 3 s using the AIS method with number of samples Nas = 800. The AIS algorithm applies four iterations, where each iteration consists of 200 samples. The number of Hits is 512 and the number of Misses is 288. The standard error is calculated to be eP¯ f (S3 s |C1 s ) = 0.0088. The standard error of the estimator doesn't appear to be entirely eliminated, but is extremely low. The failure probability is computed to be P¯ f (S3 s |C1 s ) = 0.1 as confirmation of the MCS result for the same severity level.

Figure 7.6: Estimation of failure probability for the severity level S 3 s using the AIS method. For comparative purposes, these calculations in MCS are shown in figure 7.2.

The figure 7.7 describes the behavior of the failure probability and standard error after sampling using number of samples Nas = 900. The sampling is performed in five iterations, with the first three iterations having a budget of 100 samples, the fourth with 200 samples, and the fifth with 400 samples. The number of Hits is 434 and the number of Misses is 446. The standard error is computed to be eP¯ f (S2 s |C1 s ) = 0.0088. This shows that with AIS, the variance is successfully eliminated. This proves that the AIS method is effective in eliminating the variance with less numbers of samples compared to the MCS method. The failure probability is calculated to be P¯ f (S2 s |C1 s ) = 0.18 as confirmation of the MCS result for the severity level S 2 s .

Figure 7.7: Estimation of failure probability for the severity level S 2 s using the AIS method. For comparative purposes, these calculations in MCS are shown in figure 7.3.

As shown in figure 7.8, the probability of failure P¯ f (S1 s |C1 s ) with 0.52 using the AIS method confirms the result extracted from the MCS method for the same severity level. The convergence of the AIS method occurs with the required standard error of 0.05 using a number of samples Nas = 800. Accordingly, the standard error eP¯ f (S1 |C1 s ) of 0.036 is used as abort criteria for the simulation runs.

The sampling is performed in four iterations, with the first three iterations having a budget of 100 samples and the fourth with 500 samples. The number of Hits is 616 and the number of Misses is 184.

Figure 7.8: Estimation of failure probability for the severity level S 1 s using the AIS method. For comparative purposes, these calculations in MCS are shown in figure 7.4.

Figure 7.9 shows that the failure probability at severity level S 0 s using the AIS method and number of samples Nas = 800. The sampling is executed in four iterations, with the first three iterations having a budget of 100 samples and the fourth with 500 samples. The number of Hits is 581 and the number of Misses is 219. Because of the very few available TTCE1 min events of the cluster C 1 s at the severity level S 0 s as shown in figure 7.1, the AIS method generates a statistical uncertainty with the result of the failure probability. As aforementioned, due to poor sampling in the region of TTCE1 min > 2[], there were not enough sample to explorate the values within the specific used budget of samples, resulting in an non-coherent probability failure not achieving eP¯ f (S0 |C1 s ) less than 0.05. Figure 7.9 shows that AIS method is the least suitable for these error ranges of TTCE1 min and MCS method ought to be used instead.

Figure 7.9: Estimation of failure probability for the severity level S 0 s using the AIS method. For comparative purposes, these calculations in MCS are shown in figure 7.5.

Figures 7.6 to 7.9 show a similar type of movement of the Hit cloud from the bottom left to the top right in comparison with figures 7.2 to 7.5. In many cases, the AIS method requires fewer samples to achieve the required standard error. For several iterations, the parameters of the importance sampling density function *hY* (*y*) can be determined adaptively. The probability of failure decreases from S 1 s to S 3 s , while S 0 s occurs with a bigger probability than S 1 s . The safety statement can be obtained in that the failure probability for each traffic scenario is approximated by estimating the exceedance region in the parameter space. At the beginning of the AIS method, the required standard error P¯ f of 0.1 is chosen for the investigations, which generally represents a good confidence in the result. While the AIS fulfills its required standard error of 0.1 after a few simulation runs, but the iteration results differ strongly, the required standard error is reduced to P¯ f = 0.05 to force more iterations for the AIS method. For comparability reasons, P¯ f = 0.05 is also given for the MCS algorithm in all severity levels. The MCS method is used as a reference, where the results of MCS and AIS methods confirm each other.

# **8 Conclusions**

The incomplete and implicit requirements are a part of the verification challenges of ADFs. There is no complete public computer-aided traffic simulator that contains the rules for dealing with exceptions. Moreover, testing can detect the presence of errors, but not their absence. In addition, the Pesticide paradox phenomenon occurs after fixing the software errors found by a scenario catalog. As a consequence, the same catalog may no longer be suitable for the residual errors. The incompleteness of the functional requirements can thus be covered by requirements extraction from the field observation. The on-road testing plays a decisive role to identify the missing functional requirements and transform these requirements adaptively into the deployed scenario catalogs. The scenario-based testing assumes that the majority of the mileage on the motorway is accomplished without particular events, while critical scenarios in real traffic are rather rare and randomly distributed. The identification and reproducibility of critical scenarios from the KDD of on-road testing in laboratory simulations or proving grounds leads to a significant reduction of the FOT distances required for statistical approval. The scenario-based functional decomposition, therefore, splits complex functions into sub-functions in order to identify relevant driving scenarios that reduce the approval effort for ADFs.

### **8.1 Executive Summary**

The status quo evaluation refers to large-scale verification as one of the decisive challenges for the economical, reliable and safe use of ADFs in CMV product engineering. Therefore, the systems engineering has established data-based and knowledge-driven test methods to assure the required dependability of their products.

The required test distance is approximately 220 million kilometers with errorfree driving to show that the automated CMV operates with a confidence level of 95% as safely as a human driver, as indicated in section 2.4.

#### RQ1: **How can the knowledge-based and data-driven test platforms be combined in a complementary and collaborative manner?**

Due the rare nature of critical events, the FOTs will require a prohibitive number of driving hours to prove the statistical superiority. At the same time, the required total mileage is increased if an error occurs during the validation process. Therefore, the sole reliance on FOTs is inadequate and, in particular, time and cost-intensive when applied to the next generation of ADFs, e.g. truck platooning and hub-to-hub transportation. Owing to the complexity of the ADFs, a test oracle for safety certification is unlikely to be provided by a single test platform. Therefore, innovative approaches are required to enable systematic testing under reproducible conditions with regard to robustness, reliability and safety, as discussed in chapters 3 and 5. In addition, functional decomposition is also necessary to support the argumentation for a reasonably low residual risk resulting from imperfections of the environmental perception sensors, as elucidated in chapter 4.

#### RQ2: **How can the** ADFs **be effectively and efficiently tested?**

The context-driven approach — as indicated in section 3.2 — employs a grey-box test strategy that combines the insights from the observed road data with functional requirements within the ODD coverage. In figure 3.4, the argument structure of the context-driven test concept is shown with the help of the GSN. The required test termination criteria can be met by quantifying the conflicts between the test concept requirements, and thus, achieving a trade-off between efficiency and effectiveness criteria of the test procedures. The correlation between the various test coverage criteria intends to support controlling the decision of choosing whether more test kilometers need to be collected or more simulations require to be carried out. The optimized test strategy requires a selection of the necessary test methods for various scenarios and their interaction with other test methods. Thereby, new and innovative approaches need to be designed, especially in simulation and in the laboratory. The knowledge-based test platforms are executed at component and system levels to generate the corresponding test evidence, as explained in chapter 4.

Moreover, the test results of the data-driven test platforms are evaluated using statistical extrapolation techniques and field-based observations within the ODD coverage, as discussed in chapter 5.

#### RQ3: **How can the prospective risk be measured in order to achieve reasonable termination criteria for the testing of the** ADFs?

As indicated in chapter 5, the measurable safety approach envisages the identification of failure types to break down the functional complexity. Furthermore, the presented framework structure utilizes a back-end database filled with catalogs of corner driving scenarios from different sources of field-based observations. The extensive data-sets are collected world-wide under realistic driving conditions with test CMV fleets. The evaluation of algorithms requires an efficient search and interpretation of relevant traffic situations with help of RCA to modify the algorithm accordingly. In this scheme, the processing chain includes clustering of multivariate time-series data-sets and finding critical driving situations to identify and allocate the necessary test cases for various suitable test environments. The platform-independent mechanism is intended to offer a consistent scenario description format for the various test environments. Further, these new test cases complement the existing test cases developed from expert knowledge in an adaptive ODD coverage manner.

A case study is provided to illustrate the measurable safety framework, in which the behavior of CMVs equipped with an AEB system with respect to stationary objects is discussed. The AEB can also unexpectedly issue warnings and brake the vehicle if it detects stationary objects next to the vehicle's own lane, e.g. broken-down vehicles, signs, bridges and traffic islands. The extracted clusters and their parameter space define the probability distributions of the associated parameters of each logical scenario. The minimum TTC is used as a prototypical safety measure. Thereby, the sensitivity analysis reduces the number of input variables to the most important ones — as discussed in chapter 6 — , which influence the safety measure. Subsequently, the reliability analysis identifies the failure region in the parameter space to predict the probability of failure for each logical scenario using MCS and AIS methods, as indicated in chapter 7. The generated tolerable risk curve is then used as a reference safety threshold — as portrayed in figure 6.1 — for the test termination criteria.

Meanwhilst, the real PoS can only be provided after the market introduction of automated CMVs in which the probability of collisions can be estimated based on the relevant parameter space within the ODD. However, the probability of failure can be approximated by estimating the exceedance probability of parameters with higher criticality while validating the performance of existing levels of automation. Accordingly, there is a hypothesis based on the assumption that erroneous triggering events of the present ADFs during the field observation can be considered as disengagement events for automated CMVs. Therefore, the prospective risk assessment of an existing ADF provides reference values for the risk acceptance threshold. Such reference values act as a benchmark for the further development and optimization of a similar ADF at the next level of automation.

### **8.2 Technical and Scientific Contributions**

The proposed framework enables functional verification of ADFs on the embedded ECUs in complex automotive sensor networks. The framework utilizes a real-time capable system architecture in a distributed heterogeneous co-simulation environment. In this architecture, object-list-based sensor models are designed to simulate realistic sensor behavior. The real-time interaction between the HiL test bench and the DuT imposes timing constraints to ensure traceability and reproducibility of the test results. The described architecture contains both the structural design and time-efficient integration into the test bench and is applied to RADAR and camera sensors. An important feature of the architecture is the high portability between various driving simulation frameworks due to well-defined in-/output interfaces [BMK+16]. Furthermore, a run-time fault injection has been introduced to simulate sensor failures, such as latency, detection failure and false one-to-many object labeling. These failures are used for testing the robustness of the ADFs. Accordingly, a phenomenological sensor model can be developed iteratively. Sensor failures can be added in a continuous manner in order to achieve increasing degrees of realism. For efficient verification, an astute selection of relevant test scenarios is conducted to avoid repetitive test scenarios. The research results of this thesis have been published in internationally well-recognized journals and peer-reviewed conferences.

Six invention disclosures focusing on test procedures have been filed and used as references in this work. In the *DE<PHONE_NUMBER>71A1*, the invention relates to a method for testing a lane assistance system for a vehicle by extracting an ontology-based category of adequate and relevant scenarios for existing field tests [ESF19]. In the *DE<PHONE_NUMBER>29A1*, the invention deals with a cluster analytical characterization of driving situations based on detected sensor signals for the detection of surroundings and their system reactions in the driving operation of the vehicle [ESS+19c]. In the *DE<PHONE_NUMBER>64A1*, the invention relates to a method for testing a blind spot assistance system for a vehicle, particularly for a truck. On the basis of the Tractrix zone of the Egovehicle, objects are detected by sensors and their distance and relative speed to the Ego-vehicle are measured. The cluster analysis is applied to determine the criticality of the detected ambient objects [ESS+19b].

In the *DE<PHONE_NUMBER>65A1*, the invention relates to a method for testing a map-based system for speed limit in a vehicle. The system detects traffic signs based on maps and fused image, wherein acquired information to be tested are classified by means of a cluster analysis for scenarios into equivalence classes [ESS+19a]. In the *DE<PHONE_NUMBER>07A1*, the invention refers to a method for testing an emergency braking function of a vehicle based on a confidence for existing field tests. Thereby, the sensitivity and reliability analysis identifies a failure range in the parameter space to predict a failure probability for each extracted logical scenario using sampling methods [ESD20].

In the *DE<PHONE_NUMBER>44A1*, the invention relates to a method for evaluating systematic and statistical errors of multi-radar-based recognition systems of a vehicle. According to the invention, exceptions from field-based observations and knowledge experience are identified by means of processes of knowledge development by clustering acquired data records and determining the exceptions in order to identify and assign necessary test data records for various suitable test environments. By means of the determined test data-sets, test data-sets developed on the basis of expert knowledge are completed in an adaptive test coverage [EDA20].

# **8.3 Future Research Directions**

The presented work raises several issues that require substantial future research activities. The quality of ADFs depends primarily on the environmental sensors providing the vehicle's environmental perception as the basis for situation analysis and decision making. An acceptable level of maturity of these sensors must be accomplished as a prerequisite for an adequate field validation. The logical and statistical errors or functional deficiencies are assessed by objective and subjective evaluation criteria.

Since it is not possible to guarantee complete safety for automated CMVs, one of the major challenges in automated truck driving is to argue for a reasonably low residual risk resulting from limitations of the environmental perception sensor. Currently, relevant safety norms do not support such arguments. While the Threat Assessment and Remediation Analysis (TARA) is one of the key activities defined in the ISO/SAE21434 for automotive cyber-security, a SOTIF similar standard is also required for security issues.

Moreover, the standards for safety (ISO 26262 and ISO/PAS 21448) and security (ISO/SAE 21434) need to be more harmonized. The cyber-security is a condition in which assets are adequately protected against threat scenarios to electrical or electronic components of road vehicles and their functions [fS+20]. Therefore, further research will include the application of the test method for security issues. These activities have to be integrated into an ASE approach that supports the structure of the context-driven test concept. This research work needs to be complemented by activities with standard organizations to form a consensus on risk evaluation and acceptable argumentation structures that would feed into future standards and CoP guidelines for the safeguarding of ADFs.

# **9 Glossary**

# **9.1 List of Acronyms**

#### **Preliminaries**

#### **Framework Conception**

#### **Framework Implementation**

# **9.2 List of Variables and Constants**

#### **Preliminaries**

lateral acceleration of the Ego-vehicle.

longitudinal acceleration between the Ego-vehicle and the object ahead.

lateral acceleration between the Ego-vehicle and the object ahead.

b pitch angle while braking in the straight-ahead direction.

#### **Framework Implementation**

	- min lower bound of the truncated normal distribution of the lateral distance between the Ego-vehicle and the object ahead at the time of triggering the first escalation event.

(S3 s |C1 s ) standard error of S 3 s of the cluster C 1 s at the escalation level E 1 s . (S2 s |C1 s ) standard error of S 2 s of the cluster C 1 s at the escalation level E 1 s . (S1 s |C1 s ) standard error of S 1 s of the cluster C 1 s at the escalation level E 1 s . (S0 s |C1 s ) standard error of S 0 s of the cluster C 1 s at the escalation level E 1 s . road actual road curvature.

f

d

eP¯ f

eP¯ f

eP¯ f

eP¯

y

 E1

i road curvature at the position (i) of the clothoid.

ego curvature of the Ego-vehicle at the time of triggering the first escalation event.

ego lower bound of the truncated normal distribution for the road curvature at the time of triggering the first escalation event.

 max

 min

> ego upper bound of the truncated normal distribution for the road curvature at the time of triggering the first escalation event. length of the clothoid arc to the point (i).

Li P¯ f (S3 s |C1 s

) failure probability of S 3 s of the cluster C 1 s at the escalation level E 1 s .

# **9.3 List of Quantities and States**

#### **Preliminaries**

vSiL generated code result of a back-to-back test within a SiL test setup.

#### **Framework Conception**

#### **Framework Implementation**

# **9.4 List of Matrices and Vectors**

#### **Preliminaries**

#### **Framework Conception**

# **9.5 List of Notations and Co-ordinate Systems**

#### **Preliminaries**

#### **Framework Conception**

#### **Framework Implementation**

# **List of Figures**

# **List of Tables**

# **Bibliography**

	- [AB15] M. Aeberhard and T. Bertram. Object Classification in a High-Level Sensor Data Fusion Architecture for Advanced Driver Assistance Systems. In *18th International Conference on Intelligent Transportation Systems*. IEEE, Gran Canaria, Spain, 2015. DOI: 10.1109/ITSC.2015.76.
	- [Abe08] S. Abendroth. Automatisiertes Testen im Nutzfahrzeugbereich. In *Automatisiertes Testen Eingebetteter Systeme in der Automobilindustrie*. Carl Hanser Verlag, 2008. DOI: 10.3139/9783446419018.008.
	- [AD11] M. Althoff and J.M. Dolan. Set-based computation of vehicle behaviors for the online verification of autonomous vehicles. In *14th International IEEE Conference on Intelligent Transportation Systems (ITSC)*, pages 1162–1167. IEEE, Washington, DC, USA, 2011. DOI: 10.1109/ITSC.2011.6083052.
	- [AD12] M. Althoff and J.M. Dolan. Reachability computation of low-order models for the safety verification of high-order road vehicle models. In *2012 American Control Conference (ACC)*, pages 3559–3566. IEEE, Montreal, QC, Canada, 2012. DOI: 10.1109/ACC.2012.6314777.
	- [AD14] M. Althoff and J.M. Dolan. Online Verification of Automated Road Vehicles Using Reachability Analysis. In *IEEE Transactions on Robotics*, volume 30, pages 903–918. IEEE, 2014. DOI: 10.1109/TRO.2014.2312453.
	- [AFS93] R. Agrawal, C. Faloutsos, and A. Swami. Efficient similarity search in sequence databases. In *International Conference on Foundations of Data Organization and Algorithms*, volume 730, pages 69–84. Springer Verlag, Berlin, Heidelberg, 1993. DOI: 10.1007/3-540-57301-1\_5.
	- [AG19] Daimler Truck AG. The new Actros all new features in detail. In *Actros*. Article - (Retrieval date: 01.06.2022), Barcelona, Spain, 2019.
	- [AGZ18] M. Arief, P. Glynn, and D. Zhao. An Accelerated Approach to Safely and Efficiently Test Pre-Production Autonomous Vehicles on Public Streets. In *21st International Conference on Intelligent Transportation Systems*, pages 2006–2011. IEEE, Maui, HI, USA, 2018. DOI: 10.1109/ITSC.2018.8569371.
	- [AL18] M. Althoff and S. Lutz. Automatic Generation of Safety-Critical Test Scenarios for Collision Avoidance of Road Vehicles. In *Intelligent Vehicles Symposium (IV)*, pages 1326–1333. IEEE, Changshu, China, 2018. DOI: 10.1109/IVS.2018.8500374.
	- [Alt10] M. Althoff. Reachability Analysis and its Application to the Safety Assessment of Autonomous Cars. In *Technische Universität München*. Dissertation - (Retrieval date: 01.06.2022), 2010. Lehrstuhl für Steuerungs- und Regelungstechnik.
	- [Ber19] M. Berk. Safety Assessment of Environment Perception in Automated Driving Vehicles. In *Technische Universität München*. Dissertation - (Retrieval date: 01.06.2022), 2019. Fachgebiet für Risikoanalyse und Zuverlässigkeit.
	- [BHS17] O. Bartels, A. Hellmann, and P. Seiniger. Driving tests for the approval of automatically commanded steering functions. In *International Technical Conference on the Enhanced Safety of Vehicles*. Article - (Retrieval date: 01.06.2022), 2017.
	- [BMC15] M.F. Bugallo, L. Martino, and J. Corander. Adaptive importance sampling in signal processing. In *Digital Signal Processing*, volume 47, pages 36–49. Elsevier Inc., 2015. DOI: 10.1016/j.dsp.2015.05.014.
	- [BSH17] H. Beglerovic, M. Stolz, and M. Horn. Testing of autonomous vehicles using surrogate models and stochastic optimization. In *International Conference on Intelligent Transportation Systems*. IEEE, Yokohama, Japan, 2017. DOI: 10.1109/IT-SC.2017.8317768.
	- [Buc09] C. Bucher. Computation of Failure Probabilities. In *Computational Analysis of Randomness in Structural Mechanics*, volume 32, pages 56–67. CRC Press, London, UK, 2009. DOI: 10.1201/9780203876534.
	- [Bun22] Bundesrat. Verordnung zur Regelung des Betriebs von Kraftfahrzeugen mit automatisierter und autonomer Fahrfunktion und zur änderung strassenverkehrsrechtlicher Vorschriften vom 24. Juni 2022. In *Bundesgesetzblatt Teil I*. Standard - (Retrieval date: 01.07.2022), Bonn, Germany, 2022.
	- [CAR15] V. Cantillo, J. Arellana, and M. Rolong. Modelling pedestrian crossing behaviour in urban roads: A latent variable approach. In *Transportation Research Part F: Traffic Psychology and Behaviour*, volume 32, pages 56–67. Elsevier Ltd., 2015. DOI: 10.1016/j.trf.2015.04.008.
	- [CFHL07] X. Chen, J. Feng, M. Hiller, and V. Lauer. Application of Software Watchdog as a Dependability Software Service for Automotive Safety Relevant Systems. In *International Conference on Dependable Systems and Networks*. IEEE, 2007. DOI: 10.1109/DSN.2007.14.
		- [Cho16] J.B. Choi. Environmental Perception for Automated Vehicles: Localization, Mapping and Tracking. In *Technische Universität Braunschweig*. Dissertation - (Retrieval date: 01.06.2022), 2016. Institut für Regelungstechnik.
	- [CS15] B. Costello and R. Suarez. Truck driver shortage analysis 2015. In *American Trucking Associations*. Article - (Retrieval date: 01.06.2022), 2015.
	- [Cza18] K. Czarnecki. Operational Design Domain for Automated Driving Systems: Taxonomy of Basic Terms. In *Waterloo Intelligent Systems Engineering Lab*. WISE Lab, 2018. DOI: 10.13140/RG.2.2.18037.88803.
	- [DES16] DESTATIS. Unfallentwicklung auf deutschen straßen 2015. In *Statistisches Bundesamt 2016*. DESTATIS - (Retrieval date: 01.06.2022), Wiesbaden, Germany, 2016.
	- [DG18] W. Damm and R. Galbas. Exploiting learning and scenariobased specification languages for the verification and validation of highly automated driving. In *1st International Workshop on Software Engineering for AI in Autonomous Systems*. ACM, 2018. DOI: 10.1145/3194085.3194086.
	- [DS11] S. Durekovic and N. Smith. Architectures of Map-Supported ADAS. In *Intelligent Vehicles Symposium (IV)*, pages 207–211. IEEE, Baden-Baden, Germany, 2011. DOI: 10.1109/IVS.2011.5940402.
	- [DS15] B. Duraisamy and T. Schwarz. On Track-to-Track Data Association for Automotive Sensor Fusion. In *International Conference on Information Fusion*. IEEE, Washington, DC, USA, 2015. Electronic ISBN: 978-0-9824-4386-6.
	- [Düs10] T. Düser. X-in-the-Loop-ein durchgängiges Validierungsframework für die Fahrzeugentwicklung am Beispiel von Antriebsstrangfunktionen und Fahrerassistenzsystemen. In *Karlsruher Institut für Technologie*. Dissertation - (Retrieval date: 01.06.2022), 2010. Institut für Produktentwicklung.
	- [EH16] M. El-Haji. Ontologie-basierte Definition von Anforderungen an Validierungswerkzeuge in der Fahrzeugtechnik. In *Karlsruher Institut für Technologie*. Dissertation - (Retrieval date: 01.06.2022), 2016. Institut für Fahrzeugsystemtechnik - Institutsteil Fahrzeugtechnik.
	- [Elg12] M. Elgharbawy. Further development and optimization of a realtime Hardware-in-the-Loop test bench for camera-based driver assistance systems. In *Master's Thesis*. Paderborn University, 2012. Heinz Nixdorf Institut.
	- [Ete17] A. Etemad. AdaptIVe: Automated Driving Applications and Technologies for Intelligent Vehicles. In *Automated Driving: Safer and More Efficient Future Driving*, pages 535–540. Springer International Publishing, Cham, Switzerland, 2017. DOI: 10.1007/978-3-319-31895-0\_23.
	- [FBK17] L. Fraade-Blanar and N. Kalra. Autonomous Vehicles and Federal Safety Standards: An Exemption to the Rule? In *RAND Cooperation*. Article - (Retrieval date: 01.06.2022), 2017.
	- [FBS09] S. Fuchs, B. Butting, and E. Sax. Automated synthesis and configuration of Hardware-in-the-Loop test equipment. In *Applied Electronics*. IEEE, Pilsen, Czech Republic, 2009. Print ISBN:<PHONE_NUMBER>81-0.
		- [FC11] FESTA-Consortium. FESTA Handbook Version 4. In *Field Operational Tests Networking and Methodology Promotion*. FOT-NET - (Retrieval date: 01.06.2022), 2011.
	- [Fei18] M. Feilhauer. Simulationsgestützte Absicherung von Fahrerassistenzsystemen. In *Universität Stuttgart*. Dissertation - (Retrieval date: 01.06.2022), 2018. Institut für Höchstleistungsrechnen.
	- [Fis17] T. Fischer. Eine Technologie für das durchgängige und automatisierte Testen eingebetteter Software. In *Karlsruher Institut für Technologie*. Dissertation - (Retrieval date: 01.06.2022), 2017. KIT-Fakultät für Elektrotechnik und Informationstechnik.
	- [fS+15] International Organization for Standardization et al. Systems and software engineering - System life cycle processes. In *ISO/IEC/IEEE 15288:2015*. Standard - (Retrieval date: 01.06.2022), 2015.
	- [FSLL19] P. Feig, J. Schatz, V. Labenski, and T. Leonhardt. Assessment of Technical Requirements for Level 3 and Beyond Automated Driving Systems Based on Naturalistic Driving and Accident Data Analysis. In *Proceedings of ESV Conference*. Article - (Retrieval date: 01.06.2022), 2019.
	- [FWP+19] F. Fahrenkrog, L. Wang, T. Platzer, A. Fries, F. Raisch, and K. Kompaß. Prospective Safety Effectiveness Assessment of

Automated Driving Functions – from the Methods to the Results. In *Proceedings of ESV Conference*. Article - (Retrieval date: 01.06.2022), 2019.

	- [GB12] B. Geller and T. Bradley. Quantifying Uncertainty in Vehicle Simulation Studies. In *SAE International Journal of Passenger Cars-Mechanical Systems*. SAE Mobilus, 2012. DOI: 10.4271/2012-01-0506.
	- [GKS18] J. Guo, U. Kurup, and M. Shah. Is It Safe to Drive? An Overview of Factors, Challenges, and Datasets for Driveability Assessment in Autonomous Driving. In *Artificial Intelligence*. arXiv - (Retrieval date: 01.06.2022), 2018.
	- [Gol18] K. Golowko. Absicherung der ADAS-Funktionen schwerer Nutzfahrzeuge. In *ATZ-Automobiltechnische Zeitschrift*. Springer, 2018. DOI: 10.1007/s35148-018-0117-1.
	- [H+12] L. Hammarstrand et al. Adaptive Radar Sensor Model for Tracking Structured Extended Objects. In *IEEE Transactions on Aerospace and Electronic Systems*, volume 48, pages 1975– 1995. IEEE, 2012. DOI: 10.1109/TAES.2012.6237574.
	- [H+15] T. Hanke et al. Generic architecture for simulation of ADAS sensors. In *16th International Radar Symposium (IRS)*, pages 125–130. IEEE, Dresden, Germany, 2015. DOI: 10.1109/IRS.2015.7226306.
	- [Hal20] S. Hallerbach. Simulation-based testing of cooperative and automated vehicles. In *Carl von Ossietzky Universität Oldenburg*. Dissertation - (Retrieval date: 01.06.2022), 2020. Fakultät II - Informatik, Wirtschafts- und Rechtswissenschaften.
	- [Har03] D.W. Harwood. Review of Truck Characteristics as Factors in Roadway Design. In *National Cooperative Highway Research Program*. Transportation Research Board, 2003. DOI: 10.17226/23379.
	- [HS10] M. Hörwick and KH. Siedersberger. Strategy and architecture of a safety concept for fully automatic and autonomous driving assistance systems. In *Intelligent Vehicles Symposium*, pages 955–960. IEEE, La Jolla, CA, USA, 2010. DOI: 10.1109/IVS.2010.5548115.
	- [HS15] Y. Horita and R.S. Schwartz. Extended electronic horizon for automated driving. In *14th International Conference on ITS Telecommunications (ITST)*, pages 32–36. IEEE, Copenhagen, Denmark, 2015. DOI: 10.1109/ITST.2015.7377396.
	- [Hul16] D. Hull. The Tesla Advantage: 1.3 Billion Miles of Data. In *Bloomberg*. Article - (Retrieval date: 01.06.2022), 2016.
	- [Int17] SAE International. Event Data Recorder. In *SAE J1698*. SAE Mobilus, 2017. Standard - (Retrieval date: 01.06.2022).
	- [Int18] SAE International. Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles. In *SAE J3016*. SAE Mobilus, 2018. Standard - (Retrieval date: 01.06.2022).
	- [Int20] SAE International. Automated Driving System Data Logger. In *SAE J3197*. SAE Mobilus, 2020. Standard - (Retrieval date: 01.06.2022).
	- [JN04] H. Janssen and W. Niehsen. Vehicle surround sensing based on information fusion of monocular video and digital map. In *Intelligent Vehicles Symposium*, pages 244–249. IEEE, Parma, Italy, 2004. DOI: 10.1109/IVS.2004.1336389.
	- [JS14] K. Jo and M. Sunwoo. Generation of a Precise Roadway Map for Autonomous Cars. In *IEEE Transactions on Intelligent Transportation Systems*, volume 15, pages 925–937. IEEE, 2014. DOI: 10.1109/TITS.2013.2291395.
	- [JS+19a] S. Jesenski, J.E. Stellet, et al. Generation of Scenes in Intersections for the Validation of Highly Automated Driving Functions. In *Intelligent Vehicles Symposium*, pages 502–509. IEEE, Paris, France, 2019. DOI: 10.1109/IVS.2019.8813776.
	- [JS+19b] S. Jesenski, J.E. Stellet, et al. Simulation-Based Methods for Validation of Automated Driving: A Model-Based Analysis and an Overview about Methods for Implementation. In *Intelligent Transportation Systems*, pages 1914–1921. IEEE, Auckland, New Zealand, 2019. DOI: 10.1109/ITSC.2019.8917072.
	- [JSW19] P. Junietz, U. Steininger, and H. Winner. Macroscopic Safety Requirements for Highly Automated Driving. In *Transportation Research Record*. SAGE journals, 2019. DOI: 10.1177/0361198119827910.
	- [K+18a] P. Koopman et al. Cerification of Highly Automated Vehicles for Use on UK Roads. In*Creating An Industry-Wide Framework for Safety*. Report - (Retrieval date: 01.06.2022), 2018.
	- [K+18b] M. Koren et al. Adaptive Stress Testing for Autonomous Vehicles. In *Intelligent Vehicles Symposium (IV)*. IEEE, Changshu, China, 2018. DOI: 10.1109/IVS.2018.8500400.
	- [K+18c] R. Krajewski et al. Data-Driven Maneuver Modeling using Generative Adversarial Networks and Variational Autoencoders for Safety Validation of Highly Automated Vehicles. In *21st International Conference on Intelligent Transportation Systems (ITSC)*. IEEE, Maui, HI, USA, 2018. DOI: 10.1109/IT-SC.2018.8569971.
	- [K+19a] S. Kitajima et al. Multi-agent traffic simulations to estimate the impact of automated technologies on safety. In *Traffic injury prevention*. Taylor & Francis, 2019. DOI: 10.1080/15389588.2019.1625335.
	- [K+19b] F. Kruber et al. Unsupervised and Supervised Learning with the Random Forest Algorithm for Traffic Scenario Clustering and Classification. In *Intelligent Vehicles Symposium*. IEEE, 2019. DOI: 10.1109/IVS.2019.8813994.
	- [Kar20] A. Karpathy. System and method for obtaining training data. In *United States Patent and Trademark Office*. TESLA INC, 2020. AN:<PHONE_NUMBER>.
	- [Kir15] M. Kirschbaum. Highly automated driving for commercial vehicles. In *6th International Munich Chassis Symposium*. Springer Vieweg, Wiesbaden, Germany, 2015. DOI: 10.1007/978-3-658-09711-0\_2.
	- [KM05] A.J. Ko and B.A. Myers. A framework and methodology for studying the causes of software errors in programming systems. In *Journal of Visual Languages & Computing*, volume 16, pages 41–84. Elsevier Ltd., 2005. DOI: 10.1016/j.jvlc.2004.08.003.
	- [KS13] N. Kurtz and J. Song. Cross-entropy-based adaptive importance sampling using Gaussian mixture. In *Structural Safety*, volume 42, pages 35–44. Elsevier Ltd., 2013. DOI: 10.1016/j.strusafe.2013.01.006.
	- [Küh13] W. Kühn. Road Categorization. In *Fundamentals of Road Design*. WIT PRESS, 2013. ISBN: 978-1-84564-097-2.
	- [L+15] R. Lee et al. Adaptive stress testing of airborne collision avoidance systems. In *34th Digital Avionics Systems Conference*. IEEE, Prague, Czech Republic, 2015. DOI: 10.1109/DASC.2015.7311450.
	- [L+18a] J. Langner et al. Estimating the Uniqueness of Test Scenarios derived from Recorded Real-World-Driving-Data using Autoencoders. In *Intelligent Vehicles Symposium*, pages 1860–1866. IEEE, Changshu, China, 2018. DOI: 10.1109/IVS.2018.8500464.
	- [L+19] J. Langner et al. Logical Scenario Derivation by Clustering Dynamic-Length-Segments Extracted from Real-World-Driving-Data. In *Proceedings of the 5th International Conference on Vehicle Technology and Intelligent Transport Systems*. SciTePress, 2019. DOI: 10.5220/0007723304580467.
	- [Lat08] F. Lattemann. Test-Operations. In *Automatisiertes Testen Eingebetteter Systeme in der Automobilindustrie*. Carl Hanser Verlag, 2008. DOI: 10.3139/9783446419018.007.
	- [LBS19] A. Lauber, N. Brenner, and E. Sax. Automated vehicle depots as an initial step for an automated public transportation. In *UITP Global Public Transport Summit*. Article - (Retrieval date: 01.06.2022), Stockholm, Schweden, 2019. Institut für Technik der Informationsverarbeitung.
	- [Lei20] A. Leitner. ENABLE-S3: Project Introduction. In *Validation and Verification of Automated Systems*. Springer Nature, Cham, Switzerland, 2020. DOI: 10.1007/978-3-030-14628-3\_4.
	- [Li07] M. Li. Robust optimization and sensitivity analysis with multiobjective genetic algorithms: Single-and multi-disciplinary applications. In *University of Maryland*. Dissertation - (Retrieval date: 01.06.2022), 2007. Department of Mechanical Engineering.
	- [LPN11] S.M. Loos, A. Platzer, and L. Nistor. Adaptive Cruise Control: Hybrid, Distributed, and Now Formally Verified. In *International Symposium on Formal Methods*. Springer Verlag, Berlin, Heidelberg, 2011. DOI: 10.1007/978-3-642-21437-0\_6.
	- [LS19] S.H. Leilabadi and S. Schmidt. In-depth Analysis of Autonomous Vehicle Collisions in California. In *Intelligent Transportation Systems Conference*. IEEE, Auckland, New Zealand, 2019. DOI: 10.1109/ITSC.2019.8916775.
	- [M+15] M.A. Mohammad et al. Ontology-based Framework for Risk Assessment in Road Scenes Using Videos. In *Procedia Computer Science*. Elsevier B.V., 2015. DOI: 10.1016/j.procs.2015.08.300.
	- [M+16] J. Mazzega et al. Testing of highly automated driving functions. In *ATZ worldwide*. Springer, 2016. DOI: 10.1007/s38311-016- 0101-x.
	- [M+17] SM.S. Mahmud et al. Application of proximal surrogate indicators for safety evaluation: A review of recent developments and research needs. In *International Association of Traffic and Safety Sciences Research*. Elsevier Ltd., 2017. DOI: 10.1016/j.iatssr.2017.02.001.
	- [Mul85] M. Mulazzani. Reliability versus Safety. In *IFAC Proceedings Volumes*. Elsevier Ltd., 1985. DOI: 10.1016/S1474- 6670(17)60097-1.
	- [Mul18] G.E. Mullins. Adaptive Sampling Methods for Testing Autonomous Systems. In *University of Maryland*. Dissertation - (Retrieval date: 01.06.2022), 2018. Computer Science.
	- [MV19] S. Müller and F. Voigtländer. Automated Trucks in Road Freight Logistics: The User Perspective. In *Interdisciplinary Conference on Production, Logistics and Traffic*. Springer Nature, Cham, Switzerland, 2019. DOI: 10.1007/978-3-030-13535-5\_- 8.
	- [MW08] T. Most and J. Will. Metamodel of Optimal Prognosis an automatic approach for variable reduction and optimal metamodel selection. In *Proc. Weimarer Optimierungs- und Stochastiktage*. Article - (Retrieval date: 01.06.2022), 2008.
		- [N+09] D. Nienhuser et al. A Situation context aware Dempster-Shafer fusion of digital maps and a road sign recognition system. In *Intelligent Vehicles Symposium*. IEEE, Xi'an, China, 2009. DOI: 10.1109/IVS.2009.5164490.
	- [NS11] M. Nentwig and M. Stamminger. Hardware-in-the-Loop testing of computer vision based driver assistance systems. In *Intelligent Vehicles Symposium*. IEEE, 2011. DOI: 10.1109/IVS.2011.5940567.
	- [O+18] M. O'Kelly et al. Scalable End-to-End Autonomous Vehicle Testing via Rare-event Simulation. In *Advances in Neural Information Processing Systems*. arXiv - (Retrieval date: 01.06.2022), 2018.
	- [Ott13] C. Otto. Fusion of data from heterogeneous sensors with distributed fields of view and situation evaluation for advanced driver assistance systems. In *Karlsruher Institut für Technologie*. Dissertation - (Retrieval date: 01.06.2022), 2013. Institut für Industrielle Informationstechnik.
	- [P+01] A. Pasquini et al. Analysis of Incidents Involving Interactive Systems. In *Computer Safety, Reliability and Security*. Springer Verlag, Berlin, Heidelberg, 2001. DOI: 10.1007/3-540-45416- 0\_11.
	- [R+15] A. Richter et al. Reducing the gap between simulated and real life environments by introducing high-precision data. In *Proceedings of the Driving Simulation Conference 2015 Europe VR*. Driving Simulation Association, 2015. ISBN: 978-3-9813099- 3-5.
	- [R+16] A. Robicquet et al. Learning Social Etiquette: Human Trajectory Understanding in Crowded Scenes. In *European Conference on Computer Vision*. Springer International Publishing, Cham, Switzerland, 2016. DOI: 10.1007/978-3-319-46484-8\_33.
	- [S+08] A. Saltelli et al. Sensitivity Analysis: From Theory to Practice. In *Global Sensitivity Analysis: The Primer*, pages 237–275. John Wiley & Sons, Ltd, 2008. DOI: 10.1002/9780470725184.ch6.
	- [S+09] M. Serfling et al. Camera and imaging radar feature level sensorfusion for night vision pedestrian recognition. In *Intelligent Vehicles Symposium*. IEEE, Xi'an, China, 2009. DOI: 10.1109/IVS.2009.5164345.
	- [Sch05] Peter Scholz. Echtzeit, Echtzeitsysteme, Echtzeitbetriebssysteme. In *Softwareentwicklung eingebetteter Systeme: Grundlagen, Modellierung, Qualitätssicherung*. Springer Verlag, Berlin, Heidelberg, 2005. DOI: 10.1007/3-540-27522-3\_3.
	- [Sch15] HP. Schöner. Challenges and approaches for testing of highly automated vehicles. In *Energy Consumption and Autonomous Driving*. Springer International Publishing, Cham, Switzerland, 2015. DOI: 10.1007/978-3-319-19818-7\_11.
	- [Sie14] K. Siegemund. Contributions to ontology-driven requirements engineering. In *Technischen Universität Dresden*. Dissertation - (Retrieval date: 01.06.2022), 2014. Fakultät Informatik - Lehrstuhl Softwaretechnologie.
	- [Sjo22] K. Sjoberg. Automated trucks overtake self-driving cars [connected and automated vehicles]. In *IEEE Vehicular Technology Magazine*. IEEE, 2022. DOI: 10.1109/MVT.2021.3133558.
	- [Smi17] B.W. Smith. Automated Driving and Product Liability. In *Michigan State Law Review*. Michigan State University College of Law, 2017. DOI: 10.17613/3bj8-z207.
	- [SMJ19] M. Saraoglu, A. Morozov, and K. Janschek. MOBATSim: MOdel-based Autonomous Traffic Simulation Framework for Fault-Error-Failure Chain Analysis. In *International Federation of Automatic Control*. Elsevier Ltd., 2019. DOI: 10.1016/j.ifacol.2019.08.077.
	- [SN19] S.E. Shladover and C. Nowakowski. Regulatory challenges for road vehicle automation: Lessons from the california experience. In *Transportation research part A: policy and practice*. Elsevier Ltd., 2019. DOI: 10.1016/j.tra.2017.10.006.
	- [SS06] LP. Shiu and CY. Sin. Top-Down, Middle-Out, and Bottom-Up Processes: A Cognitive Perspective of Teaching and Learning Economics. In *International Review of Economics Education*, volume 5, pages 60–72. Elsevier Ltd., 2006. DOI: 10.1016/S1477-3880(15)30124-9.
	- [SS13] B. Schick and S. Schmidt. Evaluation of Video-Based Driver Assistance Systems with Sensor Data Fusion by Using Virtual Test Driving. In *Proceedings of the FISITA 2012 World Automotive Congress*, pages 1363–1375. Springer Verlag, Berlin, Heidelberg, 2013. DOI: 10.1007/978-3-642-33738-3\_36.
	- [Ste14] S. Steinmeyer. Probabilistische Fahrzeugumfeldschätzung für Fahrerassistenzsysteme. In *Technische Universität Braunschweig*. Dissertation - (Retrieval date: 01.06.2022), 2014. Fakultät für Elektrotechnik, Informationstechnik, Physik.
	- [Ste16] J.E. Stellet. Statistical modelling of algorithms for signal processing in systems based on environment perception. In *Karlsruher Institute for Technology*. Dissertation - (Retrieval date: 01.06.2022), 2016. Institute for Anthropomatics and Robotics.
	- [T+17] A.S. Trigell et al. Advanced vehicle dynamics of heavy trucks with the perspective of road safety. In *Vehicle System Dynamics*, volume 55, pages 1572–1617. Taylor & Francis, 2017. DOI: 10.1080/00423114.2017.1319964.
	- [Tal10] N.N. Taleb. A Brief History of the Black Swan Problem. In *The Black Swan: The Impact of the Highly Improbable*, volume 25. Random House, 2010. DOI: 10.5465/amp.25.2.87.
	- [Tar12] A.P. Tarko. Use of crash surrogates and exceedance statistics to estimate road safety. In *Accident Analysis & Prevention*, volume 45, pages 230–240. Elsevier Ltd., 2012. DOI: 10.1016/j.aap.2011.07.008.
	- [Tel14] D. Tellmann. Hardware-in-the-Loop-gestützte Entwicklungsplattform für Fahrerassistenzsysteme: Modelle der Umfeldsensorik und angepasste Fahrermodelle. In *Universität Kassel*. Dissertation - (Retrieval date: 01.06.2022), 2014. Fachbereich 16 Elektrotechnik/Informatik.
	- [UL22] Underwriters Laboratories. Standard for Safety for the Evaluation of Autonomous Vehicles and Other Products. In *ANSI/UL 4600*. Standard - (Retrieval date: 01.06.2022), 2022.
	- [VDA17] VDA. Automative SPICE Process Assessment / Reference Model - Version 3.1. In *Quality Management in the Automotive Industry*. Standard - (Retrieval date: 01.06.2022), 2017.
	- [VGL+12] B. Vanholme, D. Gruyer, B. Lusetti, et al. Highly Automated Driving on Highways Based on Legal Safety. In *IEEE Transactions on Intelligent Transportation Systems*, volume 14, pages 333–347. IEEE, 2012. DOI: 10.1109/TITS.2012.2225104.
	- [vNC14] K. von Neumann-Cosel. Virtual Test Drive: Simulation umfeldbasierter Fahrzeugfunktionen. In *Technische Universität München*. Dissertation - (Retrieval date: 01.06.2022), 2014. Lehrstuhl für Echtzeitsysteme und Robotik.
	- [VR15] I.T. Voyles and C.J. Roy. Evaluation of Model Validation Techniques in the Presence of Aleatory and Epistemic Input Uncertainties. In *17th AIAA Non-Deterministic Approaches Conference*. Aerospace Research Central, Kissimmee, Florida, 2015. DOI: 10.2514/6.2015-1374.
	- [W+14] A. Welzel et al. Accurate camera-based traffic sign localization. In *17th International IEEE Conference on Intelligent Transportation Systems (ITSC)*. IEEE, Qingdao, China, 2014. DOI: 10.1109/ITSC.2014.6957730.
	- [Wie17] A. Wieczorek. The influence of self-driving transport vehicles on the field of logistics. In *Transport Economics and Logistics*, volume 66, pages 107–114. Research Journal of the University of Gdansk, 2017. DOI: 10.5604/01.3001.0010.5602.
	- [Wil15] B. Wilmes. Hybrides Testverfahren für Simulink/Targetlink Modelle. In *Technische Universität Berlin*. Dissertation - (Retrieval date: 01.06.2022), 2015. Fakultät IV - Elektrotechnik und Informatik.
	- [Wil16] B. Wilmes. TASMO: Automated Test Data Generation for Simulink Model Coverage. In *Simulation and testing for vehicle technology*, pages 123–133. Springer International Publishing, Cham, Switzerland, 2016. DOI: 10.1007/978-3-319-32345-9\_- 10.
	- [Win13] H. Winner. Challenges of Automotive Systems Engineering for Industry and Academia. In *Automotive Systems Engineering*, pages 3–15. Springer Verlag, Berlin, Heidelberg, 2013. DOI: 10.1007/978-3-642-36455-6\_1.
	- [WPZ19] X. Wang, H. Peng, and D. Zhao. Combining Reachability Analysis and Importance Sampling for Accelerated Evaluation of Highly Automated Vehicles at Pedestrian Crossing. In *Dynamic Systems and Control Conference*, volume 3. ASME, 2019. DOI: 10.1115/DSCC2019-9179.
	- [WW16] W. Wachenfeld and H. Winner. The Release of Autonomous Vehicles. In *Autonomous Driving: Technical, Legal and Social Aspects*, pages 425–449. Springer Verlag, Berlin, Heidelberg, 2016. DOI: 10.1007/978-3-662-48847-8\_21.
	- [WW19] C. Wang and H. Winner. Overcoming Challenges of Validation Automated Driving and Identification of Critical Scenarios. In *2019 IEEE Intelligent Transportation Systems Conference (ITSC)*, pages 2639–2644. IEEE, Auckland, New Zealand, 2019. DOI: 10.1109/ITSC.2019.8917045.
		- [Xi08] C. Xi. Requirements and concepts for future automotive electronic architectures from the view of integrated safety. In *Universität Karlsruhe*. Dissertation - (Retrieval date: 01.06.2022), 2008. Fakultät für Elektrotechnik und Informationstechnik.
	- [Z+18a] H. Zhang et al. An ontology-guided semantic data integration framework to support integrative data analysis of cancer survival. In *Medical Informatics and Decision Making*. BMC, 2018. ISSN: 1472-6947.
	- [Zha16] D. Zhao. Accelerated Evaluation of Automated Vehicles. In *University of Michigan*. Dissertation - (Retrieval date: 01.06.2022), 2016. Mechanical Engineering.
	- [Zio13] E. Zio. System Reliability and Risk Analysis. In *The Monte Carlo Simulation Method for System Reliability and Risk Analysis*, pages 7–17. Springer, London, UK, 2013. DOI: 10.1007/978-1-4471-4588-2\_2.
	- [ZRBE20] A. Zlocki, D. Raudszus, J. Bock, and L. Eckstein. Methoden zur Absicherung von Komponenten und Funktionen. In *ATZextra 25*, pages 28–33. Springer, 2020. DOI: 10.1007/s35778-020- 0111-6.
		- [ZWZ18] J. Zhu, W. Wang, and D. Zhao. A Tempt to Unify Heterogeneous Driving Databases using Traffic Primitives. In *Computer Vision and Pattern Recognition*. arXiv: 1805.04925 - (Retrieval date: 01.06.2022), 2018.

# **Project-Related Publications**

# **Conferences and Journals**

	- [E+16] M. Elgharbawy et al. An Agile Verification Framework for Traffic Sign Classification Algorithms in Heavy Vehicles. In *2016 IEEE/ACS 13th International Conference of Computer Systems and Applications (AICCSA)*, pages 1–8. IEEE, Agadir, Morocco, 2016. DOI: 10.1109/AICCSA.2016.7945719.
	- [E+18] M. Elgharbawy et al. A Data-driven Verification Framework for Active Safety Functions. In *Proceedings of the Driving Simulation Conference 2018 Europe VR*, pages 131–135. Driving Simulation Association, Antibes, France, 2018. ISBN: 978-2-85782-734-4.
	- [ESFG20] M. Elgharbawy, A. Schwarzhaupt, M. Frey, and F. Gauterin. Measurable Safety of Automated Driving Functions using Stochastic Analysis Methods. In *Proceedings of the Driving Simulation Conference 2020 Europe VR*, pages 27–34. Driving Simulation Association, Antibes, France, 2020. ISSN: 0769-0266.
	- [ESm+19] M. Elgharbawy, A. Schwarzhaupt, R. Arenskrieger, H. Elsayed, M. Frey, and F. Gauterin. A Testing Framework for Predictive Driving Features with an electronic Horizon. In *Special TRF issue: Driving simulation*, volume 61, pages 291–304. Transportation Research Part F: Traffic Psychology and Behaviour, 2019. DOI: 10.1016/j.trf.2017.08.002.
	- [ESO+19] M. Elgharbawy, I. Scherhaufer, K. Oberhollenzer, M. Frey, and F. Gauterin. Adaptive Functional Testing for Autonomous Trucks. In *International Journal of Transportation Science and Technology*, volume 8, pages 202–218. Tongji University Press, 2019. DOI: 10.1016/j.ijtst.2018.11.003.
	- [ESS+16] M. Elgharbawy, A. Schwarzhaupt, G. Scheike, M. Frey, and F. Gauterin. A Generic Architecture of ADAS Sensor Fault Injection for Virtual Tests. In *2016 IEEE/ACS 13th International Conference of Computer Systems and Applications (AICCSA)*, pages 1–7. IEEE, Agadir, Morocco, 2016. DOI: 10.1109/AICC-SA.2016.7945680.

# **Supervised Theses**

	- [Are16] R. Arenskrieger. HiL-Absicherungsumgebung für Fahrerassistenzsysteme mit elektronischem Horizont. In *Karlsruher Institut für Technologie*. Masterarbeit, 2016. Institut für Fahrzeugsystemtechnik.
	- [Ber16] B. Bernier. Conception and implementation of an algorithm for detection and classification of traffic signs in heavy vehicles. In *Karlsruhe Institute of Technology*. Master's Thesis, 2016. Institute for Vehicle System Technology.
	- [Els18] H. Elsayed. Data-driven analysis of naturalistic driving for automated driving functions. In *University of Stuttgart*. Master's Thesis, 2018. Institute for Visualization and Interactive Systems.
	- [Kan16] M. Kanberger. Evaluation of testability of camera-based driver assistance systems based on Hardware-in-the-Loop simulations by verification of CAN messages. In *University of Applied Sciences Esslingen*. Master's Thesis, 2016. Graduate School.

### **Invention Disclosures**

[ESS+19c] H. Elsayed, A. Schwarzhaupt, I. Scherhaufer, M. Gut, M. Elgharbawy, and M. Frey. Verfahren zum Testen eines Bremsassistenzsystems für ein Fahrzeug. In *German Patent and Trade Mark*. Daimler AG, Karlsruher Institut für Technologie (KIT), 2019. AN:<PHONE_NUMBER>29.

# **Karlsruher Schriftenreihe Fahrzeugsystemtechnik FAST Institut für Fahrzeugsystemtechnik (ISSN 1869-6058)**

Eine vollständige Übersicht der Bände finden Sie im Verlagsshop

Die Bände sind unter www.ksp.kit.edu als PDF frei verfügbar oder als Druckausgabe bestellbar.

Die Bände sind unter www.ksp.kit.edu als PDF frei verfügbar oder als Druckausgabe bestellbar.

# Karlsruher Schriftenreihe Fahrzeugsystemtechnik

With the further development of automated driving, the functional performance increases resulting in the need for new and comprehensive testing concepts. This work aims to enable the transition from quantitative mileage to qualitative test coverage by aggregating the results of both knowledge-based and data-driven test platforms. The validity of the test domain can be extended cost-effectively throughout the software development process to achieve meaningful test termination criteria.

ISSN 1869-6058 ISBN 978-3-7315-1254-7


**3. Methodology**

Two genetic algorithms designed to detect communities in graphs were applied in large networks modelling supergrids. These evolutionary approaches are guided by *modularity* [23], which is an applied objective function extensively used in community detection due to its simplicity and ease of calculation. Modularity provides a numerical value that represents the quality of the solution, with greater values corresponding to a more accurate community structure. Therefore, the aim was to find communities that maximise the value of *modularity* (*Q*), defined as:

$$Q = \frac{1}{2M} \sum (a\_{ij} - \frac{K\_i K\_j}{2M}) \delta(i, j), \tag{1}$$

where *M* represents the total number of edges in the network; the sub-indices *i* and *j* indicate two nodes (vertices) of the network, *Ki* and *Kj* being the degrees of the *i*-th and *j*-th nodes, respectively; the parameter *aij* is the element of the *i*-th row and the *j*-th column of the adjacency matrix; and *δ*(*i*, *j*) represents the relationship between the *i*-th node and the *j*-th node (i.e., *δ*(*i*, *j*) = 1 if node *i* and node *j* are in the same community; otherwise, *δ*(*i*, *j*) = 0).

Finding these communities by maximising the modularity [23] or another objective function is an NP-hard problem [46]. Brandes et al. [47] proved that modularity maximisation is an NP-hard problem, even for the restricted version with a bond of two clusters, and suggested further investigation of approximation algorithms and heuristics for solving this problem. More recently, other authors have demonstrated the high complexity of calculating modularity on sparse graphs and dense graphs separately [48]. Due to the high complexity of the community detection problem, researchers have applied heuristic and meta-heuristic methods to obtain high quality solutions, in a reasonable computational time.

The field of evolutionary computation [49] is closely related to computational intelligence, with a focus on designing algorithms to solve complex global optimisation problems. Evolutionary algorithms are problem-solving procedures that include evolutionary processes as the key design elements, such that a population of individuals is continually and selectively evolved until a termination criteria is fulfilled. Genetic algorithms (GAs) [50] are probably the most widely used evolutionary techniques. A genetic algorithm mimics natural selection by evolving, over time, a population of individual solutions to the problem at hand until a termination condition is fulfilled and the best individual is taken as an acceptable solution. Two important characteristics of GAs are the representation used (e.g., binary or real) and the genetic operators employed (e.g., mutation and crossover). GAs have been successfully applied to solve electrical problems [51–53].

In this study, two genetic algorithms were adapted to solve community detection problems in power grids. These algorithms (MIGA and GGA+) were recently proposed and shown to be more effective than other approaches to community detection, as assessed by benchmarks typically used to compare algorithms. Figure 1 shows the flowchart of MIGA and GGA+. The main characteristics of both methods are briefly described below.

• The modularity and improved genetic algorithm (MIGA) [54] takes the modularity (*Q*) as the objective function, and uses the number of community structures as prior information to improve stability and accuracy of community detection. MIGA also uses simulated annealing [55] as local search strategy. Note that many authors have previously considered the use of local search strategies [56] for solving hard optimisation problems.

•The generational genetic algorithm (GGA+): GGA+ [57] includes efficient and safe initialisation methods in which a maximum node size is assigned to each community. Several operators are applied to migrate or exchange nodes between communities while using the modularity function as the objective function. An important feature of GGA+ is that it is able to rapidly obtain community partitions with different degrees of abstraction.

**Figure 1.** Flowchart of (**a**) MIGA and (**b**) GGA+.


**3. Terminology**

It is important to know the terminology frequently used when dealing with this kind of procedure in relation to (a) different types of graft; (b) distinction between grafts and transplants; (c) the process of new bone formation, and; (d) the use of consolidation and osteointegration, since these terms vary greatly in the literature and an attempt to standardized them has not been effective. The nomenclature most commonly used is as follows:

### **3.1 Types of grafts**

Grafts received from oneself and reimplanted in oneself should be called autogenic, autogenous, autologous or autograft. The graft obtained from other individuals, but intended to a host of the same species, should be called allogeneic, allogenous, homologous, allograft or homograft, while grafts from donors to hosts of different species should be called xenogenic, xenogenous, xenologous, xenografts or interspecific.

novel issue and it is crucial that already in the XXI century, we have to know the reasons for

This chapter is intended to present a brief review of the history of bone grafts, indications for use, storage details and differences of processing techniques as well as to present the experience of our department with the use of freeze-dried bone from bovine origin in a

The idea of transplanting tissues between individuals of the human species and even between individuals of distinct species is very ancient, dating to the early Christian era. Empirically, Cosmas and Damian, in the second century, were regarded as the pioneers of bone transplantation by having removed a leg of an individual due to an advanced tumour and soon deployed a new leg taken from a freshly killed Moro. However, it was Sir William MacEwan in 1880 who successfully reconstructed the humerus of a boy with the tibia obtained from another child. On the other hand, the first transplant between different species was conducted by the Dutch surgeon Job van Meekren in 1668. According to Godwin, he successfully transplanted part of the skull of a dog in a traumatic defect of the skull of a soldier. Deeply concerned of being excommunicated by the church for having undergone a non-Christian method of treatment, the soldier asked the surgeon to remove the graft, but due to time elapsed, this could not be completely removed since it was already

Is not well known when the first studies on bone grafts started. However, Ollier and Barth in the late 19th century, were the first to intensively investigate and report the biological properties of the tissue and concluded, though partially correct, that bone and periosteum

Assuming that the cells survive in the graft, even when removed from the donor, Albee in 1915, concluded in a classical work, the most suitable tissues for transplant are those originated from the connective tissue such as bone, fat and fascia. Ten years later, in 1925, Lexer published his experience with transplants and total joint replacement called "arthroplasty" to all surgical procedures performed in an attempt to restore joint movement.

It is important to know the terminology frequently used when dealing with this kind of procedure in relation to (a) different types of graft; (b) distinction between grafts and transplants; (c) the process of new bone formation, and; (d) the use of consolidation and osteointegration, since these terms vary greatly in the literature and an attempt to standardized them has not been effective. The nomenclature most commonly used is as

Grafts received from oneself and reimplanted in oneself should be called autogenic, autogenous, autologous or autograft. The graft obtained from other individuals, but intended to a host of the same species, should be called allogeneic, allogenous, homologous, allograft or homograft, while grafts from donors to hosts of different species should be

remain viable when transplanted, contributing to new bone formation.

called xenogenic, xenogenous, xenologous, xenografts or interspecific.

its use and, over all matters, to clearly determine our results.

thousand orthopaedic procedures.

**2. History**

fully integrated.

**3. Terminology**

**3.1 Types of grafts**

follows: