#  Boundary conditions for similarity Index

Madhu Kashyap Jagadeesh 11affiliation: Department of Physics, Jyoti Nivas
College, Bengaluru-560095, Karnataka, India and Pursharth Saxena22affiliation:
Department of Mathematics, Christ (Deemed to be University), Bengaluru 560
029, India<EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

The recent development, shows that the Bray-Curtis’s formula for similarity
Index (1957), has been applied in various fields like Ecology, Astrophysics,
etc. In this paper, we found the possible boundary conditions for this evolved
formula (i.e the numerical range in which the formula becomes in-effective to
give the expected result). Here we have simulated the real world data in the
form of normally distributed random numbers, that directly shows the range (or
conditions) at which this formula gives unambiguous similarity result.

Similarity Index, Metric Tool, Boundary Conditions

##  1 Introduction

Similarity Indexes are used to compare two or more quantitative/numeric data
set, such as: the variation from a reference state. These indices are applied
for classification of objects, clustering and retrieval problems, or to
compute the overlaps between quantitative data (Kashyap et al., 2017). The
range of the index is defined on the closed interval of [0,1], where 0 and 1
corresponds to absolute dissimilarity and absolute similarity respectively.
According to (Boesch, D. F., 1977), from ecological studies, there are many
multivariate techniques like similarity index, but not many of them are
preferred. Since this one is widely accepted and used, we have chosen a random
data set which disapproves with the above said agreement, with certain
boundary conditions. There are many techniques applied in the ecological
community (Such as: (Austin M. P. And Orloci L., 1966), (Orloci L., 1966), and
(Anderson A. J. B., 1971)), and according to (Bray and Curtis, 1957) there
will be distortion in the data set. Later in 2011 the similarity index reached
extra-terrestrial research, like finding Earth-like planets using Earth
Similarity Index (ESI) (Schulze-Makuch et al., 2011) and for Mars-like planets
applying Mars Similarity Index (MSI) (Kashyap et al., 2017).

The structure of the paper is as follows: section 2 contains Mathematical
formulation, and section 3 follows up with assumptions and graphical analysis,
the last section is discussion and conclusion.

##  2 Mathematical formulation of Bray-Curtis Similarity Indices

Similarity between samples having multivariate data was well established by
Deza and Deza in 2006. The practical application of the same (with ecology
community) was done by Looman and Campbell, 1960. This substantiates the usage
of Bray–Curtis’ work of 1957, which can be mathematically expressed as:

$d_{\rm
BC}={\sum_{i=1}^{n}|p_{i}-q_{i}|\over\sum_{i=1}^{n}\left(p_{i}+q_{i}\right)}\,.$
(1)

Where $p_{i}$ and $q_{i}$ are measurable quantities between which the distance
is to be measured, and $n$ is the total number of variables (Kashyap et al.
2017). Later in 1981 Bloom’s transformation technique for the similarity
measures for intersection was described as:

$s_{\rm BC}=1-d_{\rm
BC}=1-{\sum_{i=1}^{n}|p_{i}-q_{i}|\over\sum_{i=1}^{n}\left(p_{i}+q_{i}\right)}\,.$
(2)

Here, $0$ conveys the complete absence of relationships, whereas, $1$ shows a
complete match of the two data records in the $n$-dimensional space (Schulz,
2007).

For heterogeneous data, a process of standardization, i.e. balancing and
equating the contribution of different types of variable, is needed to find
the Similarity (Greeenacre and Primicerio 2013).

Traditionally, the similarity indices are subdivided into equal 0.2 intervals
(Bloom, 1981), defining very low, low, moderate, high and very high similarity
regions. Therefore, the threshold can be defined as:

$V=\left[1-\Big{|}\frac{x_{ref}-x}{x_{ref}+x}\Big{|}\right]^{w_{x}}\,.$ (3)

where $x$ can be any physical entity (such as: Mass, Concentration, Radius,
etc…), $w_{x}$ is the weight for that particular entity, $x_{r}ef$ is the
reference value, and the dimension $n=1$. Usually for formulating similarity
index, threshold $V=0.8$.

We define the physical limit as $x_{a}$ and $x_{b}$, which defines the limit
for variation of the variable with respect to $x_{ref}$ (i.e.
$x_{a}<x_{ref}<x_{b}$). Weight exponent are then calculated for the lower
limit, $w_{a}$ and the upper limit, $w_{b}$

$w_{a}=\frac{\ln{V}}{\ln\left[1-\left|\frac{x_{ref}-x_{a}}{x_{ref}+x_{a}}\right|\right]}\,,\quad
w_{b}=\frac{\ln{V}}{\ln\left[1-\left|\frac{x_{b}-x_{ref}}{x_{b}+x_{ref}}\right|\right]}\,,\quad$
(4)

The average weight is found by the geometric mean,

$w_{x}=\sqrt{{w_{a}}\times{w_{b}}}\,.$ (5)

In this paper, we use Eq. 2 to define the similarity index

$SI_{x}={\left[1-\Big{|}\frac{x-x_{ref}}{x+x_{ref}}\Big{|}\right]^{w_{x}}}\,,$
(6)

The assumption in the Bray-Curtis scale is that samples are taken from same
physical measure, e.g. mass, or volume. And the final form of similarity index
obtained satisfies the Bray-Curtis assumption.

##  3 Assumptions for boundary conditions

Let $R={\left\\{{a_{0},a_{1},a_{2}\dots a_{m}}\right\\}}\quad m\in\mathbb{N}$
be a non-empty set containing raw data.
Then, Consider a non-empty set, $S={\left\\{{x_{0},x_{1},x_{2}\dots
x_{n}}\right\\}}\quad n\in\mathbb{N}$ to be a subset of R, such that $n\leq
m$.

The weight exponents for calculating Similarity Index, and accounting for the
Similarity Index itself, is dependent on $x_{min}$, $x_{max}$, and $x_{ref}$,
i.e the minimum, maximum, and reference value respectively. Where
$x_{min},\quad x_{max}\in R$ and $x_{ref}$, is the pre-defined reference
point, basically set up by the application requirement, this can also belong
to the set $S$ or set $R$.

Basic range definitions of the above mentioned parameters are:

$\displaystyle x_{min}\in[0,9]$ $\displaystyle x_{max}\in[100,999]$
$\displaystyle x_{ref}\in[10,99]$

Then we define the boundary condition as

$x_{i}\leq x_{ref}\forall x_{i}\in S$ (7)

Similarly,

$x_{i}>x_{ref}\forall x_{i}\in S$ (8)

In the above expressions, Eq (7) gives expected value for the similarity index
analysis. However, Eq (8), on the other hand, does not yield the expected
value. This pattern was observed in the raw data in the form of concentration
of a trace gas (volume). Which satisfies the assumption established by Bray &
Curtis, 1957.

##  4 Results

The patterns involving the inequality in Eq (8) were observed while dealing
with concentration of atmospheric gases. Since the application part of the
paper has not been published, that data-set is deemed to confidential, and
hence random data samples were generated based on the same pattern as it was
seen in the concentration of that specific trace gas. The reference for
calculating the weight exponents using python source code is made available in
appendix section.

Here $2000$ random samples (set $R$) were drawn form a Normal Distribution
having mean = $2$ and variance = $1$. The samples were then divided into five
parts and average value of these five parts was taken as the set $S$. All the
samples were multiplied by a weight of 100 for the different case studies, the
corresponding graphical analysis is shown below.

Since the reference value $x_{ref}$, is an inherent property of the parameter
under study, and not the equation, $220$ and $50$ were randomly chosen to meet
the conditions of Eq (7) and Eq (8) respectively. In Table 1, $x_{ref}$ is the
reference value for which the conditions satisfying and dis-satisfying
similarity index are observed. $X=S$ is the set as described above.

Table 1: Similarity index projections Array Index $\rightarrow$ 0 1 2 3 4 $X$
200.35 200.51 194.24 206.88 207.21 Similarity Index ($x_{ref}=50$) 0.91687
0.91681 0.91901 0.91464 0.91453 Similarity Index ($x_{ref}=220$) 0.99327
0.99332 0.99098 0.99560 0.99572

Here the 2nd row of the table represents the condition where similarity index
metric tool does not yield the expected value. Hence we define the boundary
conditions as suggested in the assumption.

Graphical Analysis:

In Fig 1 the values of $X$ are plotted. The abscissa in this particular case
study is taken as the position of $x_{i}\in X$ in the array. i.e for abscissa
0 the corresponding value of ordinate is: $x_{0}$, and so on. In practice
abscissa for the similarity index can be anything, eg. mass, concentration,
etc, which again is an inherent property of the parameter under observation
and therefore, taking abscissa as array index does not interfere with the
results obtained.

Figure 1: Values $x_{i}\in X$ with respect to $x_{i}$’s position in the array

Fig. 2 consists of the raw data, $X$ which is plotted on graph (c), the graph
(b) consists of the similarity index satisfying the conditions of inequality
in Eq (7) and is in very good agreement with the actual data set (most of the
previously published applied data sets (like: Kashyap et al. 2017) did come
under this boundary condition). Whereas graph (a) consists of the similarity
index dissatisfying the inequality in Eq (7) and satisfying the inequality in
Eq(8), and hence, is not in proportionality with the raw data, instead, it
shows an inverse proportionality with the raw data ($X$). The array index, $2$
and $4$ are highlighted in all the subgraphs for visual verification.

Figure 2: Comparison Plot of Similarity Index. Array Index 2, in the raw data
(graph (a)) as well as Similarity Index Satisfying the Inequality 7 (graph
(b)), has the lowest value, while Array Index 4 has the highest value. The
opposite is true for graph (a)

.

The graphical representation in Figure 2 clearly shows that graph (a)
(satisfying the inequality in Eq (8) is not directly proportional with the
right hand side actual data. In order to substantiate the claim further, in
Fig 3 the Similarity Index is observed as the value of $x_{ref}$ is ranged
from $50$ (Eq 8) to $220$ (Eq 7).

Figure 3: Plot variation in Similarity Index as the value of $x_{ref}$ is
ranged from 50 to 220 graph (d) and graph (e) are of special interest here as
they encounter an $x_{ref}$ values which have some points in $X$ which are
greater than $x_{ref}$ while some points are less than $x_{ref}$.

The case studies for different ranges of $x_{ref}$ is done in the Figure 3.
Graph (a) is same as graph (a) in Fig 2. Graph (b), (c) are also similar to
graph (a) having $x_{ref}$ value as $180,190$ respectively, which is still
less than all elements of the set $X$. Note that as the value of $x_{ref}$ is
increased, the value Similarity Index of respective $x_{i}\in X$, in
comparison is also increased.

In graph (d), Array Index 2 corresponds to $194.24$, as shown in Table 4,
which is clearly less than $x_{ref}$ = 200, and hence the dip is observed.
Even though $194.24$ is the lowest value in $X$, its corresponding Similarity
Index value doesn’t yield the least possible value, because except for Array
Index 2, all the other Array Indices’s value is greater than 200. In a
nutshell, only one element of $X$ satisfies the condition 7, which shows
direct proportionality with the raw data. The other elements satisfy the
condition 8 and hence show inverse proportionality with the set $X$ ($X=S$).

Graph (e) and (f) are similar to graph (c) in Figure 2, i.e all the elements
of $X$ satisfy the condition 7, all the elements are smaller than $x_{ref}$.

Note that The Graph (e) gives the maximum value for respective similarity
index on comparison with Graph (f), despite the fact that the $x_{ref}$ value
of Graph (f) is greater than that of Graph (e). Since the dataset $X$ is
obtained from the normal distribution, we can say that the optimal Similarity
Index value is obtained, when the $x_{ref}$ value is closest to a certain
central value (in this case, mean). Graph (b) and (c) also verify the same
thing.

Verification of the effect of threshold w.r.t similarity index, which has a
constant $x_{ref}$ value:

For same dataset, Similarity Index is calculated based on different threshold
value using [Eq 3]. Example: Now choose $x_{ref}=180$, and the corresponding
calculations are tabulated in Table 2

Table 2: Similarity Index for different threshold values Threshold Array Index
0 1 2 3 4 $0.8$ 0.99298 0.99293 0.99504 0.99082 0.99071 $0.7$ 0.98881 0.98872
0.99209 0.98537 0.98520 $0.6$ 0.98401 0.98389 0.98869 0.97911 0.97887 $0.5$
0.97837 0.97821 0.98469 0.97176 0.97144 $0.4$ 0.97150 0.97129 0.97981 0.96285
0.96242

Figure 4 is the result of Table 2, and the observations are then made
regarding the role threshold value plays with respect to the proportionality
of Similarity Index and the set $X=S$.

Figure 4: Similarity Index of $X$ with different threshold values. The
clustering of points in graph (a), (b), and (c) is because of the limited
space in the y-axis (ordinate). The distance among the points in the all the
other graphs is the same.

As evident, from the graphs in Figure 4, the threshold value has no effect on
the proportionality of the Similarity Index with respect to $x_{ref}$. The
similarity index value for all the graphs in the figure, irrespective to the
threshold value, maintains an inverse proportionality with respect to the $X$.
The threshold value only controls the range that will be obtained by the
Similarity Index.

##  5 Discussion and Conclusion

We know that a counter example is sufficient to substantiate a fallacy in a
previously accepted axiom/postulate. The data set mentioned in this paper are
the direct and actual means of examples to see the limitations (or boundary
conditions) of similarity index, which is widely used in the application field
(Such as: Astrophysics and ecology (Schulze-Makuch et al., 2011) and (Looman
and Campbell, 1960)). From the above findings, we claim our assumption
considered is true, i.e the similarity index does not yield the expected
result if the conditions in Eq. 7 is not satisfied or in other words Eq. 8 is
satisfied. Figure 2 and Figure 3 verifies the claim made in section 3 through
graphical technique. The effect of threshold value on similarity index was
analyzed and verified by Fig 4, which upholds the claim that threshold value
has no impact on the boundary condition for this research findings. The
present and future scope for the application of similarity index in the fields
like exo-planets, ecology,… etc, are the motivation to upgrade the similarity
index metric tool, with our new boundary conditions. Further research is
suggested on why this anomaly is observed? and the mathematical analysis is
mandated for the same.

##  6 Appendix

The python source code for this paper is made available on:
https://github.com/Jar-win/similarity-index-analysis

## Acknowledgments

We would like to thank Center Pollution Control Board (CPCB), New Delhi,
India, for the atmospheric pollution data. Which intern helped us to find this
anomaly.

## References

  * Anderson A. J. B. (1971) Anderson A. J. B. 1971. Ecol, 59, 713.
  * Austin M. P. And Orloci L. (1966) Austin M. P. And Orloci L. 1966. Ecol, 54, 217.
  * Bloom (1981) Bloom, S. A. 1981. Marine Ecology Progress Series 5, 125.
  * Boesch, D. F. (1977) Boesch, D. F. 1977. Report 77. VIMS (EPA-600/3-7703)
  * Bray and Curtis (1957) Bray J. R. & Curtis J. T. 1957. Ecological Monographes, 27, 325.
  * Bloom (1981) Bloom, S. A. 1981. Marine Ecology Progress Series 5, 125.
  * Deza and Deza (2007) Deza, E. and Deza, M. 2016, Encyclopedia of Distances, 4th revised edition, Springer-Verlag, ISBN 978-3-662-52844-0.
  * Greenacre and Primicerio (2013) Greenacre, M. & Primicerio, R. 2013, in Multivariate Analysis of Ecological Data, (Fundaciön BBVA, 2013). ISBN: 978-84-92937-50-9.
  * Kashyap et al. (2017) _Kashyap J. M., Gudennavar S. B., Doshi U.,Safonova M._ (2017). Astrophysics and Space Science, 362, 146.

  * Looman and Campbell (1960) Looman, J. and Campbell, J. B. 1960. Ecology, 41(3), 409.
  * Orloci L. (1966) Orloci L. 1966. Ecol, 54, 1041.
  * Schulz (2007) Schulz J., 2007. Bray-Curtis dissimilarity. Algorithms – Similarity. Alfred-Wegener-Institute for Polar and Marine Research, Bremerhaven, Germany. http://www.code10.info/. Retrieved 01/06/2016.
  * Schulze-Makuch et al. (2011) Schulze-Makuch, D., Méndez, A., Fairén, A. D., et al. 2011. Astrobiology, 11, 1041.


# Spin independence of the strongly enhanced effective mass in ultra-clean
SiGe/Si/SiGe two-dimensional electron system

M. Yu. Melnikov, A. A. Shakirov, and A. A. Shashkin  Institute of Solid State
Physics, Chernogolovka, Moscow District 142432, Russia  S.-H. Huang and C. W.
Liu  Department of Electrical Engineering and Graduate Institute of
Electronics Engineering, National Taiwan University, Taipei 106, Taiwan  S. V.
Kravchenko  Physics Department, Northeastern University, Boston, MA 02115, USA

###### Abstract

The effective mass at the Fermi level is measured in the strongly interacting
two-dimensional (2D) electron system in ultra-clean SiGe/Si/SiGe quantum wells
in the low-temperature limit in tilted magnetic fields. At low electron
densities, the effective mass is found to be strongly enhanced and independent
of the degree of spin polarization, which indicates that the mass enhancement
is not related to the electrons’ spins. The observed effect turns out to be
universal for silicon-based 2D electron systems, regardless of random
potential, and cannot be explained by existing theories.

###### pacs:

71.30.+h,73.40.Qv,71.18.+y

Much interest has been recently directed toward the behavior of low-disorder,
strongly interacting two-dimensional (2D) electron systems. The interaction
strength is characterized by the ratio of the Coulomb and Fermi energies,
determined by the dimensionless parameter, $r_{\text{s}}=1/(\pi
n_{\text{s}})^{1/2}a_{\text{B}}$ (here $n_{\text{s}}$ is the electron density
and $a_{\text{B}}$ is the Bohr radius in semiconductor). In the low-density
limit ($r_{\text{s}}\gtrsim 35$), the electrons are expected to form a Wigner
crystal Tanatar and Ceperley (1989); Attaccalite et al. (2002). The Fermi
liquid theory is applicable at relatively high electron densities
($r_{\text{s}}\lesssim 1$). Within this theory, the effective electron mass,
$m$, and Landé $g$-factor are both enhanced due to spin exchange effects, with
renormalization of the $g$-factor being dominant Iwamoto (1991); Kwon et al.
(1994); Chen and Raikh (1999). Between the Fermi liquid and Wigner crystal,
intermediate phases may exist (see, e.g., Refs. Attaccalite et al. (2002);
Spivak and Kivelson (2006); Spivak et al. (2010); Zverev et al. (2012);
Shaginyan et al. (2016)). A number of theories predict a diverging effective
mass near the Wigner crystal or a precursor. Flattening of the single-particle
spectrum at the Fermi level preceded by the diverging effective mass at low
electron densities is predicted in Refs. Zverev et al. (2012); Shaginyan et
al. (2016). The effective mass divergence also follows from the
renormalization-group theory of the 2D metal-insulator transition Punnoose and
Finkel’stein (2005). Other theoretical approaches envisioning the divergence
of the effective mass at low electron densities utilize the analogy between a
strongly interacting 2D electron system and He3 Spivak and Kivelson (2006);
Spivak et al. (2010), Gutzwiller’s variational method Brinkman and Rice
(1970); Dolgopolov (2002), or the dynamical mean-field theory Camjayi et al.
(2008). Experimentally, a sharp increase of the effective mass and the spin
susceptibility, which is proportional to the product $gm$, near the critical
density for the metal-insulator transition has been observed in a variety of
strongly interacting 2D electron systems: silicon metal-oxide-semiconductor
field-effect transistors (MOSFETs) Shashkin et al. (2002); Anissimova et al.
(2006); Mokashi et al. (2012); Kuntsevich et al. (2015), GaAs/AlGaAs
heterostructures Zhu et al. (2003); Tan et al. (2005), AlAs quantum wells
Gokmen et al. (2010), MgZnO/ZnO heterostructures Kasahara et al. (2012);
Falson et al. (2015, 2022), and SiGe/Si/SiGe quantum wells Melnikov et al.
(2017a), the effect being most pronounced in silicon-based structures. The
dramatic increase of the effective mass at the Fermi level with the tendency
to diverge is likely to occur near the quantum Wigner crystal in Si MOSFETs
Brussarski et al. (2018) or the flat-band state in ultra-clean SiGe/Si/SiGe
quantum wells Melnikov et al. (2017a).

The renormalization of the effective mass is expected to be strongly sensitive
to the polarization of spins, according to the following theoretical
approaches. The effective mass increase with the degree of spin polarization
induced by a parallel magnetic field is predicted in Refs. Spivak and Kivelson
(2006); Spivak et al. (2010) due to the increase of the fraction of
crystallites in the mixed state between the Fermi liquid and the Wigner
crystal. In contrast, a decrease of the effective mass with spin polarization
is expected from Gutzwiller’s variational method, according to Refs.
Dolgopolov (2002); Spaek and Gopalan (1990). Calculations of the magnetization
within the ring-diagram approximation Zhang and Das Sarma (2006) predict an
increase of the spin susceptibility with the magnetic field, whereas a
decrease of the effective mass with spin polarization follows from numerical
calculations going beyond the random-phase approximation Asgari et al. (2009).
In measurements of the effective electron mass in a dilute two-valley 2D
electron system in silicon MOSFETs in tilted magnetic fields, the enhanced
value $m$ was found, within the experimental uncertainty, to be independent of
the degree of spin polarization Shashkin et al. (2003), which is in
disagreement with the above-mentioned theories. Measurements of the spin
susceptibility in Si MOSFETs were reported in which non-monotonic 10%
variations of the spin susceptibility with parallel magnetic field were
observed Pudalov et al. (2004); Pudalov and Gershenson (2021); note that the
observed weak change of the spin susceptibility can be due to the change of
the $g$-factor with spin-independent mass. On the other hand, the application
of the parallel magnetic field at low electron densities in AlAs quantum wells
resulted in a decrease of the effective mass even below the band value in the
single-valley case and an about 15% increase of the effective mass in the two-
valley case Gokmen et al. (2010); Padmanabhan et al. (2008). The contradictory
experimental results call for experiments to be performed in alternative 2D
electron systems.

In this paper, we report measurements of the effective mass at the Fermi level
in the two-valley 2D electron system in ultra-clean SiGe/Si/SiGe quantum wells
in tilted magnetic fields. This electron system is distinguished in that (i)
the electron mobility is almost two orders of magnitude higher than the
mobility in the least-disordered Si MOSFETs; and (ii) the behaviors of the
effective mass at the Fermi level and the energy-averaged effective mass are
qualitatively different at low electron densities, indicating the band
flattening at the Fermi level Melnikov et al. (2017a); Zverev et al. (2012);
Shaginyan et al. (2016). The effective mass is extracted from the temperature
dependence of the weak-field Shubnikov-de Haas (SdH) oscillations in the low-
temperature limit. It is enhanced by a factor of $\gtrsim 2$ at low electron
densities, i.e., in the strongly-interacting regime where the ratio between
Coulomb and Fermi energies reaches a factor of about 15. We find that within
the experimental uncertainty, the value of the effective mass does not depend
on the degree of spin polarization, which points to a spin-independent origin
of the effective mass enhancement. The observed effect turns out to be
universal for silicon-based 2D electron systems, regardless of random
potential, and is in contradiction with existing theories.

The samples were double-gate ultra-high mobility SiGe/Si/SiGe quantum wells
similar to those described in detail in Ref. Dolgopolov et al. (2021). The
approximately 15 nm wide silicon (001) quantum well is sandwiched between
Si0.8Ge0.2 potential barriers. The peak electron mobility in these samples
reached 110 m2/Vs. The samples were patterned in Hall-bar shapes with the
distance between the potential probes of 150 $\mu$m and width of 50 $\mu$m
using standard photo-lithography. Measurements were carried out in an Oxford
TLM-400 dilution refrigerator. Data were taken by a standard four-terminal
lock-in technique in a frequency range 1–11 Hz at currents 0.03–4 nA in the
linear regime of response.

Figure 1: (a) Shubnikov-de Haas oscillations in perpendicular magnetic fields
at $T=30$ mK and $n_{\text{s}}=3.97\times 10^{10}$ cm-2. A fragment of the
weak-field SdH oscillations averaged over 32 traces is shown in the inset. (b)
Change of the amplitude of the weak-field SdH oscillations in perpendicular
magnetic fields with the temperature at $n_{\text{s}}=6.8\times 10^{10}$ cm-2
and $B_{1}=$ 0.079 T (squares), $B_{2}=$ 0.084 T (diamonds), and $B_{3}=$
0.089 T (circles). The values of $T$ for the data obtained in $B_{1}$ and
$B_{2}$ are multiplied by $B_{3}/B_{1}$ and $B_{3}/B_{2}$, correspondingly.
The solid line is a fit using Eq. (1).

Figure 2: (a) The effective mass as a function of electron density determined
by SdH oscillations in perpendicular magnetic fields. The solid line
corresponds to the one in the inset. The inset shows the dependence
$n_{\text{s}}m_{0}/m$ on electron density. The solid line is a linear fit that
extrapolates to zero at $n_{\text{s}}\approx 1.0\times 10^{10}$ cm-2. (b) The
effective mass vs. the degree of spin polarization for the following electron
densities in units of $10^{10}$ cm-2: 1.75 (half-filled squares), 2.07
(diamonds), 2.70 (squares), and 3.97 (dots). The dashed horizontal lines are
guides to the eye.

Typical SdH oscillations of the resistivity $\rho_{\text{xx}}(B)$ are shown in
Fig. 1 (a), with the inset displaying a fragment of the weak-field
(sinusoidal) oscillations. The experiments were demanding because at electron
densities $\gtrsim 10^{10}$ cm-2, measuring $\rho_{\text{xx}}(B)$ oscillations
required weak magnetic fields $\sim 0.1$ T and low temperatures $\lesssim 0.1$
K. To improve the signal-to-noise ratio, the traces $\rho_{\text{xx}}(B)$ were
recorded up to 70 times and then averaged. Temperature dependence of the
amplitude, $A$, of the weak-field SdH oscillations for the normalized
resistance, $R_{xx}/R_{0}$ (where $R_{0}$ is the average resistance), is
displayed in Fig. 1 (b). To determine the effective mass, we fit the data for
$A(T)$ using the Lifshitz-Kosevich formula

$\displaystyle A(T)$ $\displaystyle=$ $\displaystyle
A_{0}\frac{2\pi^{2}k_{B}T/\hbar\omega_{c}}{\sinh(2\pi^{2}k_{B}T/\hbar\omega_{c})},$
$\displaystyle A_{0}$ $\displaystyle=$ $\displaystyle
4\exp(-2\pi^{2}k_{B}T_{D}/\hbar\omega_{c}),$ (1)

where $\omega_{c}=eB_{\perp}/mc$ is the cyclotron frequency, $B_{\perp}$ is
the perpendicular component of the magnetic field, and
$T_{\text{D}}=\hbar/2\pi k_{B}\tau_{\text{q}}$ is the Dingle temperature (here
$\tau_{\text{q}}$ is the quantum scattering time). Damping of the SdH
oscillations with temperature may be influenced by the temperature-dependent
$\tau_{\text{q}}$. However, in our experiments, possible corrections to the
effective mass due to the temperature dependence of $\tau_{\text{q}}$ are
within the experimental uncertainty and do not exceed 10%; note that the
temperature corrections to $m$ and $\tau_{\text{q}}$, calculated in Ref.
Adamov et al. (2006), are small and can be ignored in our case. The amplitude
of the SdH oscillations follows the calculated curve down to the lowest
achieved temperatures, which confirms that the electrons were in good thermal
contact with the bath and were not overheated.

In Fig. 2 (a), we show the effective mass, normalized by $m_{0}$ (where
$m_{0}=0.19\;m_{\text{e}}$ is the effective mass for noninteracting electrons
and $m_{\text{e}}$ is the free electron mass), as a function of electron
density determined by SdH oscillations in perpendicular magnetic fields. In
measurements, appreciably lower values of $n_{\text{s}}$ have been reached as
compared to previous experiments. The effective mass sharply increases with
decreasing $n_{\text{s}}$, in agreement with previously obtained results. As
seen from the inset to the figure, the inverse effective mass extrapolates
linearly to zero at a density $n_{\text{s}}\approx 1.0\times 10^{10}$ cm-2.
This is consistent with the results of measurements of the product $gm$ at the
Fermi level, in which case $g\approx g_{0}$ (where $g_{0}=2$ is the $g$-factor
for noninteracting electrons) Melnikov et al. (2019).

Figure 3: Dingle temperature as a function of the electron density extracted
from SdH oscillations in perpendicular magnetic fields. The dashed line is a
guide to the eye. The inset shows the density dependence of the corresponding
quantum scattering time $\tau_{\text{q}}$ (circles) and the transport
scattering time $\tau$ (squares) recalculated from the electron mobility.

A strong enhancement of the effective mass at the Fermi level in the limit of
low electron densities is theoretically expected to be related to spin
effects. To probe a possible influence of spins on the effective mass
enhancement, we introduced a parallel magnetic field component,
$B_{\parallel}$, that couples primarily to the electrons’ spins; note that the
orbital effects due to the finite thickness of the 2D electron system can be
neglected in our samples in the range of magnetic fields studied Melnikov et
al. (2017b).

The main result of the paper is shown in Fig. 2 (b): within the experimental
uncertainty, the enhanced effective mass at the Fermi level does not depend on
the degree of spin polarization,
$p=(B_{\parallel}^{2}+B_{\perp}^{2})^{1/2}/B_{\text{c}}$ (where $B_{\text{c}}$
is the parallel magnetic field of the complete spin polarization corresponding
to the magnetoresistance saturation). This indicates that the origin of the
mass enhancement in our samples has no relation to the spin exchange effects.
This finding is in agreement with the results obtained in Si MOSFETs Shashkin
et al. (2003).

In Fig. 3, we plot the Dingle temperature, $T_{\text{D}}$, as a function of
the electron density extracted from SdH oscillations in perpendicular magnetic
fields. Within the experimental uncertainty, the Dingle temperature changes
weakly with electron density at $n_{\text{s}}\gtrsim 4\times 10^{10}$ cm-2,
while at lower electron densities, the value $T_{\text{D}}$ drops with
decreasing $n_{\text{s}}$. In the inset to Fig. 3, we compare the density
dependences of the corresponding quantum scattering time $\tau_{\text{q}}$ and
the transport scattering time, $\tau$, determined from the electron mobility.
The transport scattering time exceeds by far $\tau_{\text{q}}$, indicating the
dominant long-range random potential in the samples; at the same time, the
presence of a short-range random potential and electron backscattering is
inferred from the pronounced metallic temperature dependence of the resistance
in zero magnetic field Melnikov et al. (2015). In addition, both scattering
times converge somewhat at low electron densities. One can indeed expect a
decrease of the ratio $\tau/\tau_{\text{q}}$ at low $n_{\text{s}}$ because for
the same transferred wavevector, the decrease of the Fermi wavevector with
decreasing electron density should lead to a larger-angle electron scattering
and smaller ratios $\tau/\tau_{\text{q}}$ Dolgopolov et al. (2003). It is
worth noting that a similar decrease of the Dingle temperature at low electron
densities was observed in Si MOSFETs Shashkin et al. (2003), where
$\tau_{\text{q}}$ and $\tau$ are approximately equal to each other, and the
short-range random potential is realized.

The independence of the strongly enhanced effective mass at the Fermi level of
the degree of spin polarization, observed in ultra-clean SiGe/Si/SiGe quantum
wells, is concurrent with the results obtained in Si MOSFETs, although the
random potentials in these electron systems are very different. We arrive at a
conclusion that the observed effect is universal for silicon-based 2D electron
systems, regardless of random potential.

Although the theoretically predicted divergence of the effective mass at low
densities due to the electron interaction effects Brinkman and Rice (1970);
Dolgopolov (2002); Punnoose and Finkel’stein (2005); Spivak and Kivelson
(2006); Camjayi et al. (2008); Spivak et al. (2010); Zverev et al. (2012);
Shaginyan et al. (2016) agrees with the experiments, the observed spin
polarization independence of the effective mass cannot be explained by
existing theories. Particularly, within the models of Refs. Spivak and
Kivelson (2006); Zhang and Das Sarma (2006); Spivak et al. (2010), the
effective mass is expected to increase with spin polarization, while the
prediction of Refs. Dolgopolov (2002); Spaek and Gopalan (1990); Asgari et al.
(2009) is the opposite. Note that in contrast to the irrelevance of spin
effects in the observed behavior of the effective mass, the exchange and
correlation effects manifest themselves in the observed increase in the
critical density for the metal-insulator transition in spin-polarizing
magnetic fields (see, e.g., Refs. Dolgopolov et al. (2017); Melnikov et al.
(2020)). Since two or more competing mechanisms or states are in play, the
problem of the interaction-enhanced effective mass in spin-polarizing magnetic
fields is very complicated and requires considerable theoretical efforts.

In summary, we have found that at low electron densities in the strongly
interacting 2D electron system in ultra-clean SiGe/Si/SiGe quantum wells, the
effective mass at the Fermi level is strongly enhanced and independent of the
degree of spin polarization, which indicates that the mechanism underlying the
effective mass enhancement is unrelated to the electrons’ spins. The observed
effect turns out to be universal for silicon-based 2D electron systems,
regardless of random potential, and is not explained by existing theories.

We gratefully acknowledge discussions with V. Dobrosavljević, V. A. Khodel, M.
Shayegan, and B. Tanatar. The ISSP group was supported by RSF Grant No.
22-22-00333. The NTU group acknowledges support by the Ministry of Science and
Technology, Taiwan (Projects No. 111-2218-E-002-040-MBK, 111-2622-8-002-001,
and 111-2634-F-A49-008). S.V.K. was supported by NSF Grant No. 1904024.

## References

  * Tanatar and Ceperley (1989) B. Tanatar and D. M. Ceperley, Phys. Rev. B 39, 5005 (1989).
  * Attaccalite et al. (2002) C. Attaccalite, S. Moroni, P. Gori-Giorgi, and G. B. Bachelet, Phys. Rev. Lett. 88, 256601 (2002).
  * Iwamoto (1991) N. Iwamoto, Phys. Rev. B 43, 2174 (1991).
  * Kwon et al. (1994) Y. Kwon, D. M. Ceperley, and R. M. Martin, Phys. Rev. B 50, 1684 (1994).
  * Chen and Raikh (1999) G.-H. Chen and M. E. Raikh, Phys. Rev. B 60, 4826 (1999).
  * Spivak and Kivelson (2006) B. Spivak and S. A. Kivelson, Ann. Phys. 321, 2071 (2006).
  * Spivak et al. (2010) B. Spivak, S. V. Kravchenko, S. A. Kivelson, and X. P. A. Gao, Rev. Mod. Phys. 82, 1743 (2010).
  * Zverev et al. (2012) M. V. Zverev, V. A. Khodel, and S. S. Pankratov, JETP Lett. 96, 192 (2012).
  * Shaginyan et al. (2016) V. R. Shaginyan, A. Z. Msezane, G. S. Japaridze, K. G. Popov, and V. A. Khodel, Front. Phys. 11, 117103 (2016).
  * Punnoose and Finkel’stein (2005) A. Punnoose and A. M. Finkel’stein, Science 310, 289 (2005).
  * Brinkman and Rice (1970) W. F. Brinkman and T. M. Rice, Phys. Rev. B 2, 4302 (1970).
  * Dolgopolov (2002) V. T. Dolgopolov, JETP Lett. 76, 377 (2002).
  * Camjayi et al. (2008) A. Camjayi, K. Haule, V. Dobrosavljević, and G. Kotliar, Nat. Phys. 4, 932 (2008).
  * Shashkin et al. (2002) A. A. Shashkin, S. V. Kravchenko, V. T. Dolgopolov, and T. M. Klapwijk, Phys. Rev. B 66, 073303 (2002).
  * Anissimova et al. (2006) S. Anissimova, A. Venkatesan, A. A. Shashkin, M. R. Sakr, S. V. Kravchenko, and T. M. Klapwijk, Phys. Rev. Lett. 96, 046409 (2006).
  * Mokashi et al. (2012) A. Mokashi, S. Li, B. Wen, S. V. Kravchenko, A. A. Shashkin, V. T. Dolgopolov, and M. P. Sarachik, Phys. Rev. Lett. 109, 096405 (2012).
  * Kuntsevich et al. (2015) A. Y. Kuntsevich, Y. V. Tupikov, V. M. Pudalov, and I. S. Burmistrov, Nat. Commun. 6, 7298 (2015).
  * Zhu et al. (2003) J. Zhu, H. L. Stormer, L. N. Pfeiffer, K. W. Baldwin, and K. W. West, Phys. Rev. Lett. 90, 056805 (2003).
  * Tan et al. (2005) Y.-W. Tan, J. Zhu, H. L. Stormer, L. N. Pfeiffer, K. W. Baldwin, and K. W. West, Phys. Rev. Lett. 94, 016405 (2005).
  * Gokmen et al. (2010) T. Gokmen, M. Padmanabhan, and M. Shayegan, Phys. Rev. B 81, 235305 (2010).
  * Kasahara et al. (2012) Y. Kasahara, Y. Oshima, J. Falson, Y. Kozuka, A. Tsukazaki, M. Kawasaki, and Y. Iwasa, Phys. Rev. Lett. 109, 246401 (2012).
  * Falson et al. (2015) J. Falson, Y. Kozuka, J. H. Smet, T. Arima, A. Tsukazaki, and M. Kawasaki, Appl. Phys. Lett. 107, 082102 (2015).
  * Falson et al. (2022) J. Falson, I. Sodemann, B. Skinner, D. Tabrea, Y. Kozuka, A. Tsukazaki, M. Kawasaki, K. von Klitzing, and J. H. Smet, Nat. Mater. 21, 311 (2022).
  * Melnikov et al. (2017a) M. Y. Melnikov, A. A. Shashkin, V. T. Dolgopolov, S.-H. Huang, C. W. Liu, and S. V. Kravchenko, Sci. Rep. 7, 14539 (2017a).
  * Brussarski et al. (2018) P. Brussarski, S. Li, S. V. Kravchenko, A. A. Shashkin, and M. P. Sarachik, Nat. Commun. 9, 3803 (2018).
  * Spaek and Gopalan (1990) J. Spaek and P. Gopalan, Phys. Rev. Lett. 64, 2823 (1990).
  * Zhang and Das Sarma (2006) Y. Zhang and S. Das Sarma, Phys. Rev. Lett. 96, 196602 (2006).
  * Asgari et al. (2009) R. Asgari, T. Gokmen, B. Tanatar, M. Padmanabhan, and M. Shayegan, Phys. Rev. B 79, 235324 (2009).
  * Shashkin et al. (2003) A. A. Shashkin, M. Rahimi, S. Anissimova, S. V. Kravchenko, V. T. Dolgopolov, and T. M. Klapwijk, Phys. Rev. Lett. 91, 046403 (2003).
  * Pudalov et al. (2004) V. M. Pudalov, M. Gershenson, and H. Kojima, _On the Electron-Electron Interactions in Two Dimensions_ (Springer Netherlands, Dordrecht, 2004), pp. 309–327.
  * Pudalov and Gershenson (2021) V. M. Pudalov and M. E. Gershenson, Phys. Rev. B 104, 035407 (2021).
  * Padmanabhan et al. (2008) M. Padmanabhan, T. Gokmen, N. C. Bishop, and M. Shayegan, Phys. Rev. Lett. 101, 026402 (2008).
  * Dolgopolov et al. (2021) V. T. Dolgopolov, M. Y. Melnikov, A. A. Shashkin, S.-H. Huang, C. W. Liu, and S. V. Kravchenko, Phys. Rev. B 103, L161302 (2021).
  * Adamov et al. (2006) Y. Adamov, I. V. Gornyi, and A. D. Mirlin, Phys. Rev. B 73, 045426 (2006).
  * Melnikov et al. (2019) M. Y. Melnikov, A. A. Shashkin, V. T. Dolgopolov, A. Y. X. Zhu, S. V. Kravchenko, S.-H. Huang, and C. W. Liu, Phys. Rev. B 99, 081106(R) (2019).
  * Melnikov et al. (2017b) M. Y. Melnikov, V. T. Dolgopolov, A. A. Shashkin, S.-H. Huang, C. W. Liu, and S. V. Kravchenko, J. Appl. Phys. 122, 224301 (2017b).
  * Melnikov et al. (2015) M. Y. Melnikov, A. A. Shashkin, V. T. Dolgopolov, S.-H. Huang, C. W. Liu, and S. V. Kravchenko, Appl. Phys. Lett. 106, 092102 (2015).
  * Dolgopolov et al. (2003) V. T. Dolgopolov, E. V. Deviatov, A. A. Shashkin, U. Wieser, U. Kunze, G. Abstreiter, and K. Brunner, Superlattices Microstruct. 33, 271 (2003).
  * Dolgopolov et al. (2017) V. T. Dolgopolov, A. A. Shashkin, and S. V. Kravchenko, Phys. Rev. B 96, 075307 (2017).
  * Melnikov et al. (2020) M. Y. Melnikov, A. A. Shashkin, V. T. Dolgopolov, S.-H. Huang, C. W. Liu, A. Y. X. Zhu, and S. V. Kravchenko, Phys. Rev. B 101, 045302 (2020).


11institutetext: Chalmers University of Technology, SE-412 96 Gothenburg,
Sweden 22institutetext: University of Gothenburg, SE-405 30 Gothenburg, Sweden
33institutetext: Veoneer Sweden AB, SE-103 02 Stockholm, Sweden

# Setting AI in context††thanks: This project has received funding from the
European Union’s Horizon 2020 research and innovation program under grant
agreement No 957197.:
A case study on defining the context and operational design domain for
automated driving

Hans-Martin Heyn 1122 0000-0002-2427-6875 Padmini Subbiah 11 Jennifer Linder
11 Eric Knauss 1122 0000-0002-6631-872X Olof Eriksson 33

###### Abstract

[Context and motivation] For automated driving systems, the operational
context needs to be known in order to state guarantees on performance and
safety. The operational design domain (ODD) is an abstraction of the
operational context, and its definition is an integral part of the system
development process. [Question / problem] There are still major uncertainties
in how to clearly define and document the operational context in a diverse and
distributed development environment such as the automotive industry. This case
study investigates the challenges with context definitions for the development
of perception functions that use machine learning for automated driving.
[Principal ideas/results] Based on qualitative analysis of data from semi-
structured interviews, the case study shows that there is a lack of
standardisation for context definitions across the industry, ambiguities in
the processes that lead to deriving the ODD, missing documentation of
assumptions about the operational context, and a lack of involvement of
function developers in the context definition. [Contribution] The results
outline challenges experienced by an automotive supplier company when defining
the operational context for systems using machine learning. Furthermore, the
study collected ideas for potential solutions from the perspective of
practitioners.

###### Keywords:

Artificial Intelligence context machine learning operational design domain
requirements engineering systems engineering

##  1 Introduction

Automated driving systems (ADS) rely on machine learning (ML) especially for
cognition tasks and sensor fusion. Machine learning, as part of artificial
intelligence (AI), experiences an advent of methods, tools, and applications,
especially due to breakthroughs in applying deep neural networks to machine
learning problems.

The growing interest in the development of systems that can take control of
driving is accompanied by concerns regarding safety, i.e., assuring that the
ADS is able to operate safely and as expected in the desired operational
context [23, 27].

At present, an answer to the safety concern is to keep the context for
automated driving very limited, for example to factory sites, harbours, or
mining operations. By developing ADS for many different, limited, contexts,
the hope is to take the experiences and lessons learned from these limited
contexts and apply them to a wider context, such as automated driving on
highways. The challenge is, that a wider context will cause a formidable grow
in possible scenarios and situations. With the current processes and methods
that were developed (or naturally grown) for limited contexts, it will be
difficult to capture and describe all the possible scenarios that the vehicle
can encounter in a wider context. Another challenge is owed to the way of
working in the automotive industry. Much of the product development is done
either solely by a supplier company, or in cooperation with the original
equipment manufacturer (OEM). This requires efficient and correct processes
for communication of information regarding the system context between the
customers, the OEM, and the supplier companies.

This case study investigates qualitatively the current challenges and solution
ideas of a Tier 1 supplier111a Tier 1 supplier develops and sells products and
solutions directly to an OEM regarding the definition of context and
operational design domain from use cases for systems of automated driving that
incorporate machine learning. Interviews with employees in a variety of
different positions at the supplier company are the main source of data for
this study in addition to data collected at different OEMs and partner
companies. The findings were triangulated with background literature and a
focus group validated the findings from the interview study. The study finds
deficits in the standardisation of context definitions and ODDs, uncertainty
and lack of transparency in processes for context definitions, insufficient
documentation of context assumptions, and too little participation of function
developers in use case interpretation and context definitions.

Section 2 of this paper describes the background and briefly the history of
context definitions for computer systems and provides a problem definition and
research questions. Section 3 explains the applied methodology. Section 4
presents the validated findings of the study. Section 5 includes the
triangulation of findings to the background literature, a summary, and a
discussion of the main findings.

##  2 Background

A system’s desired behaviour and responsibilities are often described through
textual use cases. They state how a system reacts to different situations with
as little text as possible, but also clearly convey the reactions to these
situations [3]. The task of a requirement engineer is to translate the use
cases into requirements for the system. Different requirements concern
different parts of the system: Examples of these are functional requirements,
quality requirements, safety requirements, etc. However, some of the
requirements are linked to a specific context in which they are valid in [16].
At design time, contextual attributes that can change at run time need to be
identified in order to avoid uncertain or even undesired behaviour of the
system at run time. A contextual requirement then forms a tuple of desired
behaviour (requirement) and the required state of the contextual attributes
[15]. However, considering every contextual attributes and their changes at
design time requires a complete understanding of the operational environment
which is not feasible for complex or even chaotic behaving environments [22].
What does it mean to talk about the ”context of a system”? The Oxford
Learner’s Dictionary defines ”context” as:

”[Context is] the situation in which something happens and that helps you to
understand it” [30].

For computer systems, Dey provided a more specific definition of context:

”Context is any information that can be used to characterise the situation of
an entity. An entity is a person, place, or object that is considered relevant
to the interaction between a user and an application, including the user and
applications themselves” [14, p. 4].

Researchers in systems engineering extended the definition of context of a
system by including the environment in which the system shall operate [1, 10].
Chen et al. further extended the definition of context by adding system
capabilities and the situational roles, beliefs, and intentions of people
engaged with the system to the definition of context:

”[Context is] information about a location, its environmental attributes
(e.g., noise level, light intensity, temperature, and motion) and the people,
devices, objects and software agents it contains. Context may also include
system capabilities, services offered and sought, the activities and tasks in
which people and computing entities are engaged, and their situational roles,
beliefs, and intentions” [2, p. 1-2].

Nemoto et al. introduces ”spatial-temporal elements”, and thus adds a temporal
dimension to the context [18]. The development in vehicle automation increased
the discussion around context definition for computer systems. Because of the
temporal dimension, the context is highly dynamic, and it is important to find
a systematic way to describe and confine the context of a vehicle automation
system [28]. Traditionally, scenarios are created with the aim to represent
typical driving situations in a given context. Damak argues that it is
difficult to capture all relevant contextual elements in a scenario-based
approach. He therefore proposed to build the scenarios in discrete stages
based on different context elements, such as use case, environment, road
infrastructure, and traffic objects [8].

Besides context definition, the term ”operational design domain” (ODD) has
become popular when discussing capabilities and limitations of vehicle
automation systems. The SAE standard J3016 [24] introduced a widely adopted
classification of driving automation into six levels: Level 0 indicates no
automation at all, and thus an ODD is not applicable. Level 1 to 4 indicate
different levels of automation, from merely driver assistance (Level 1) to
full automation in a predefined environment (Level 4). On the highest level of
automation (Level 5), the ODD is ”unlimited”. The standard defines the ODD as:

”[The ODD describes the] operating conditions under which a given system for
driving automation or feature thereof is specifically designed to function,
including, but not limited to, environmental, geographical, and time-of-day
restrictions, and/or the requisite presence or absence of certain traffic or
roadway characteristics” [24, p. 14].

This definition leaves much room for interpretation of what constitutes an ODD
and how to argue for completeness. Many proposals for what constitutes an ODD
have been put forward in recent years (for example a discrete list of ODD
items [19], a detailed ontology of road structures [7], runtime monitoring
requirements [4], another categorised list of ODD items [17], or internal
system capabilities [9]). Still, there is a lack of a common definition for
the ODD, which creates challenges in communication and collaboration between
the stakeholders of the system [29].

Problem Definition: A use case assumes a context, and the resulting
requirements will only be valid in that assumed context. For safety relevant
systems, the dependencies of the requirements on the context is specifically
obvious: A specific level of safety is only guaranteed in a clearly specified
and tested ODD. Outside of the ODD, the behaviour of the system cannot be
guaranteed to any safety level. The problem is, that there is neither a clear
understanding of how to define the context in which a system shall operate,
nor is there a common definition for an ODD. Ideally, the use cases for a
system should include information about the context in which it shall operate
within. However, use cases are often quite broad and non-specific, which
requires the practitioners to interpret how the context for the requirements
needs to be defined from the use case. Especially for adaptive systems, it is
important to relate requirements to a specific context, because the context
might change while the system is active [15].

###  2.1 Research Design

The problem of unclear context definitions is especially problematic for the
development of functions for automated driving that use some form of AI:
Without a clearly outlined context of the desired use case, it will be
impossible to refine a testable operational design domain in which performance
and safety aspects can be guaranteed or to find the right data sets for
training and validation of the AI. This study investigates and explores the
current status, challenges, and possible improvements for deriving context
definitions from use cases for automated driving and advanced driver
assistance systems. The study is carried out as a case study at a Tier 1
automotive supplier that develops and provides sensor systems for automated
driving systems.

The aim of this study is to provide views and information on challenges with
deriving context definitions and ODDs from use cases, in the setting of a Tier
1 supplier providing machine learning supported sensor systems for automated
driving. This empirical study does not provide a set of solutions for the
challenges.

Research Questions: Following the research approach for empirical case studies
outlined in [6], the research questions that guide this study are formulated
as open-ended questions. They focus on the previously described central
phenomenon of deriving context definitions and ODDs from use cases.

Research Question 1 (RQ1): What is the current understanding of context
definitions?

Research Question 2 (RQ2): What are the challenges with deriving context
definitions from use cases?

Research Question 3 (RQ3): Which support would be appropriate for deriving
context definitions from use cases?

##  3 Methodology

Figure 1 gives an overview of the applied methodology, which consists of four
steps: Preparation of interviews, data collection through interviews, data
analysis, and result validation.

Figure 1: Overview of the applied methodology

###  3.1 Preparation of interviews

The aim of the data collection was to illuminate the situation and challenges
with context definitions as they are experienced primarily from the
perspective of a Tier 1 supplier. The reason behind choosing a Tier 1 supplier
as site of the investigation is that automated driving functions are part of
highly complex systems that are primarily developed in cooperation between an
OEM and its Tier 1 suppliers [21]. Because of the cooperation with OEMs,
public authorities and other research organisations in the development of
automated driving functions, we chose to collect some of the data from OEMs, a
public traffic regulation authority and a research company. All of these
parties have worked in collaboration with the Tier 1 supplier in the past on
automatic driving projects.

Sampling strategy: This empirical study follows a maximum variation strategy
for sampling [6]. The participants were chosen purposefully to represent a
wide variety of experiences and positions involved in the development of
automated driving functions that use machine learning [20]. To simplify the
filtering of suitable candidates, we defined four position groups:

Positions with a high level perspective: system managers, system engineers,
system architects, and system designers;

Positions dealing primarily with requirements: requirements engineers and
(public) policy makers;

Positions with a customers / end-user focus: product owners and function
owners;

Positions with a clear focus on development: function developers and system
developers which develop the function/system based on given specifications.

The aim of the sampling strategy was to have representation of each of these
groups to ensure a view on the entire system development chain. Seven
participants from two Tier 1 supplier companies located in the United States
and Sweden were interviewed. To add the OEM’s perspective to the sampling
data, we also interviewed four participants from three different OEMs (one OEM
each from Sweden, China, and Japan). In order to increase diversity among the
participants and to reduce company induced bias in the results, one additional
person could be interviewed from a Japanese automotive technology research
company, and one person participated from the Swedish Transport Administration
(Trafikverket). Altogether 13 interviewees participated in this study. A full
list of participants and their respective roles is given in Table 1222Note
that due to privacy concern, we intentionally chose not to reveal the
respective company.

Table 1: Participants of the case study | Inter-
---
viewee
Role | Experience
A | Group Manager | ADAS features and collision avoidance features.
B | Functional Developer | ADAS features.
C | System Engineer | Planning and control for safety critical issues.
D | Researcher | Innovation for sensors and systems.
E | Developer | Algorithms for obstacle detection.
F |  | Functional Developer and
---
Functional Safety Engineer
ADAS feature development.
G | Project Manager | ADAS vehicles.
H | Researcher | Data management and computer vision.
I | Product Owner | Ground truth systems.
J | Technical Lead | AI and machine learning projects.
K |  | Technical Specialist
---
Adaptive Cruise Control
| Blind spot detection, lane changing,
---
adaptive cruise control, collision avoidance.
L | Functional Safety Manager | Functional safety work and documentation.
M | Researcher | Standardisation of safety methodologies.

###  3.2 Data collection through interviews

The interview questions, collected in an interview guide only available to the
interviewers, were formulated based on the a-priori formulated research
questions. It was divided into three sections: The first section aimed at
identifying the participant’s current role and experience. The second section
established some ground concepts with the interviewee. This was done to avoid
misunderstandings, for example due to different definition of terms. For
example, the interviewees were given the SAE’s description of the operational
design domain and providing examples for different context definitions. For
each of these examples, the interviewees were asked to provide their opinion
on applicability and problems with the provided examples. The third section
explored the process of deriving context definitions from use cases. The aim
was to investigate the multiple facets of the process, including identifying
the main concerns, describing of what works well in the process, and
registering possible improvements. In some interviews some additional follow-
up questions were included. The interviews were conducted individually with
each participant for about one hour remotely via Microsoft Teams or Zoom. One
interviewer asked the questions, while the second interviewer observed and
took notes. Each interview started by presenting information about the study’s
objective.

###  3.3 Data analysis

Except for one interview, all the interviews were recorded and transcribed.
For one interview, both the interviewer and an observer took notes. The data
analysis consisted of the three steps illustrated in Figure 2. The coding
strategy was determined through pilot coding, conducted by two researchers
independently and the results evaluated and discussed with all authors.

Figure 2: Steps of the data analysis

First cycle coding: The first cycle coding consisted of three steps: With
attribute coding, meta information such as the role of the interviewees and
work experiences were recorded. Descriptive coding allowed for developing
codes that represent different topics of the statements given by the
interviewees. Finally, initial coding, as suggested by Saldaña, was used to
highlight and understand the interviewees’ thoughts [25].

Second cycle coding: With focused coding, the initial codes from the first
cycle were divided into broad categories. These broad categories were further
split into subcategories. Afterwards, pattern coding was used to find emerging
themes among the subcategories. Finally, each statement from the interviews
were assigned to one of the created subcategories. A second independent group
of three researchers validated the found themes and the assignment of both
challenges and potential solutions mentioned by the interviewees to the
themes. With this step, it was tried to reduce bias in the selection of the
themes.

Validation of findings: This study uses the ”member checking” validation
strategy as described in [5, ch. 9] to assess the validity and accuracy of the
results. For this purpose, a focus group with four interviewees from the data
collection phase was conducted. The session was conducted remotely using
Mentimeter, a web-tool that allows for interactive questioning of the
participants. The questions for the focus group were prepared a-priori. The
themes related to the challenges were presented to the focus group, and the
participants were asked to either agree or disagree with the challenges.
Furthermore, the participants were asked to discuss the themes related to the
proposed solutions of the challenges, and they were asked to rank the themes
according to the participant’s opinion on how important the solution of a
particular challenge is.

##  4 Results

This section presents the results based on the data obtained through the
interview sessions. The section is divided into subsections presenting each
research questions individually because the research questions built upon each
other.

###  4.1 RQ1: What is the current understanding of context definitions?

The first 15 minutes of the interviews were used to establish an overview of
the interviewee’s understanding of the terms context and ODD.

What is context? The interviewees were asked to elaborate on their
understanding of what is meant by context definition in relation to automated
driving. All interviewees were not entirely sure what context definition
describes in relation to automated driving systems. Two interviewees
considered the context definition and ODD to be identical in that they both
describe environmental conditions in which the system is designed to operate:

“[…] it’s kind of the same things as the ODD is describing but context
contains all the possible combinations, if you like, of where this is going to
operate.”

However, the majority of interviewees did not see context and ODD as identical
definitions. Instead, they stated that the ODD is a form of representation of
the context.

What entails the context? All interviewees describe the context as a dynamic,
and rather wide entity, that should include situations, scenarios, and the
environments in which a system operates.

”But it is a way to define a situation, or define a system, or defining […] a
scenario, or an environment […]. So that is what I would call a context.”

One interviewee includes also the functional state of the vehicle in the
context. All interviewees stated that knowledge and a clear description about
the context is important for validation, safety, and security of the system.
If assumptions about the context of a system are made, they need to be clearly
communicated as assumptions, which is according to the interviewees not always
the case.

What is the ODD? All interviewees agreed with the SAE J3016 definition of the
ODD [24, Page 14]. Although SAE J3016 provides a definition of the ODD, all
interviewees mention that there is lack of standardisation of the format of an
ODD. A majority of interviewees added that a description of a design domain
beyond the actual ODD, in which the system’s performance is degraded but still
safe, is necessary in addition to the ODD. As an example, interviewee F
described the process in place for deriving the ODD: After the use case is
defined, an exploratory search starts to identify the context in which ”the
use case is actually happening”. Once they established an understanding of the
context, the internal conditions of the vehicle and the external environment
are analysed for the given context. The information about internal conditions
of the vehicle and external environment states are what defines the ODD. In
this process, the ODD is derived from the use cases via an exploratory search
of the context. Interviewee K however describes the process for defining the
ODD different: After an initial ODD is defined, the ODD is analysed and use
cases and requirements are derived based on the initial ODD. An iterative
process is started to adjust ODD and use cases ”back and forth” until ODD and
use cases comply with each other. This is done by first reducing the number of
use cases to the most relevant ones that fit into the initial ODD. Then, they
try to get a better understanding of the capabilities of the system, which
allows them to widen the initial ODD and to take up more use cases gradually.
In contrast to the process described by interviewee F, here they start with
the ODD and derive the use cases based on the targeted ODD.

”[…] in the ODD you have to describe it as sort of a graceful degradation of
the system when you go outside it’s never included”.

What entails the ODD? A majority of interviewees claimed that the SAE
definition is incomplete. A major missing aspect that was mentioned is the
internal state of the vehicle, i.e., capability of sensors and actors.
Furthermore, the drivers’ behaviour in and around the vehicle should be
described in the ODD (e.g. can the system operate with an intoxicated or
distracted driver?). Ten interviewees reminded that the road and lane
conditions should explicitly be highlighted in the ODD, as they play a major
role in the correct function of automated driving systems.

What is the difference between context and ODD? All interviewees associate the
ODD with safety and performance guarantees. The context is associated with
validity of requirements derived from a given use case. An interpretation of
the interviewees’ answers can be, that the ODD is an abstraction, or mode, of
the context. Similar to the World-Machine Model described by Jackson in [13],
the ODD can be interpreted as an abstraction of the context in which a given
system can testable operate with desired characteristics, such as safety,
reliability, and performance. And in some cases, as described by two of the
interviewees, the ODD as abstraction and the context itself, seem identical
for a given use case. This can occur, for example, if the the context derived
from the use case is limited enough such that the ODD can completely embrace
the operational context.

###  4.2 RQ2: What are the challenges with deriving context definitions from
use cases?

This section describes the major challenges through themes identified from the
interviews. Three areas of challenges illustrated were identified from the
interview sessions and validated by the focus group. They are ”deriving
context definitions”, ”process and communication of context definition”, and
”deriving the ODD from context definition”. For each area of challenge, only
the themes validated with a simple majority by the focus group members are
presented in Table LABEL:tab:Challenges. A full list of themes can be made
available upon request.

Table 2: Themes relates to challenges with deriving context definitions. Theme | Description
---|---
Deriving context definitions
Difficult to
describe
context |  There is a lack of terminology for describing non-numeric parameters within the context, like the weather: It is for example not clear what ”in good weather conditions” actually means. Therefore, non-numeric parameters are difficult to compare between different context descriptions. Furthermore, the environment is often dynamic and containing unknown unknowns, which can change the context of a system unpredictably.
Lack of
standard |  A common language for context definitions is lacking, which makes it difficult to work on a system or product in different countries, companies, or even teams. Unlike requirement specification, there is no correspondent context specification, which results in ambiguities in the context in which requirements are valid.
Lack of
transparency |  A lack of transparency in use case creation and requirement negotiation leads to challenges when defining the desired and feasible context of the system. It is seen as important that the function developers obtain more knowledge about the use case in order to understand the necessary context in which a function / system shall operate. Furthermore, it was stated that there is no good practise in determining if a system still operates within its designated context.
Defining ODDs
Lack of
arguments for
completeness |  It is difficult to know when an ODD captures all relevant scenarios and elements, mainly because there is no standardised method or template for determining if the ODD is complete.
Difficult to
capture all
scenarios in
ODD |  To enumerate all possible scenarios in the ODD is impossible. It is difficult to determine, which scenarios the ODD should entail, and which scenarios are not required to be captured by the ODD. Especially ”edge case” scenarios are difficult to describe in the ODD, because many assumptions are necessary in these scenarios, which are often not well documented in the context description.
Hard to
understand
context
definitions |  Function developers are not always involved in defining the context definitions, which makes it difficult for them to develop an ODD that fits the desired context.
Lack of
standard
for ODD |  ODD tends to mean different things, which makes it difficult to understand what an ODD shall entail. Different OEMs, and even different teams within an OEM, have different approaches to define the ODD, which creates confusion for Tier 1 suppliers. The lack of a standardisation has been mentioned by all interviewees as a major obstacle.
Overly
cautious |  A consequence of not knowing the right context of the system is that the ODD will be overly cautious. Developers will start with a too strict and too limited ODD, and only expand it once safety has been proven within small extensions. This can lead to unnecessary long developing and testing times, or overly cautious systems.
Process and communication
Assumption
not
documented |  A concern mentioned was that assumptions about the context and in the requirements are not being properly documented as such. In many cases it is necessary to make assumptions, but they must be clearly documented as such. A typical context assumption for a function is to assume a linear behaviour of some measured dynamic.
Insufficient
involvement
of function
developers |  Function developers are detached from the overall picture, because they are not involved enough in defining requirements and context definitions. This makes it hard for them to understand the context in which the system is supposed to function.
Lack of
feedback |  Sometimes changes in the context, and even requirements, of a system are only discovered during the development. A feedback loop is often missing to verify if these changes in context are acceptable.
Misinter-
pretation of
requirements |  Textual requirements and context definitions can be misinterpreted by different peoples with different views on the system. Often, the person writing the requirements and context definitions has no direct contact to the person implementing them.
Too difficult
process |  The process of deriving contextual information and requirements from use cases was described as being ”blurry and unsharp by nature”. It was also mentioned, that a common structured process is either missing or too complex for deriving both requirements and context definitions.

###  4.3 RQ3: Which support would be appropriate for deriving context
definitions from use cases?

For each of the three areas of challenges described in Section 4.2 the
interviewees were asked to suggest improvements. Out of the interviews, themes
were identified and presented to the focus group for validation. All themes
that achieved a simple majority vote in the focus group are presented in Table
LABEL:tab:Improvements.

Table 3: Themes related to improvement ideas for deriving context definitions. Theme | Description
---|---
Improvement ideas for deriving context definitions
More
diverse data |  According to one interviewee, a more diverse set of sensor data of the environment allows for easier interpretation and limitation of the context in which the system operates.
Standardised
approach |  All interviewees suggested to standardised context definition to ease cooperation between different teams and companies.
Improvement ideas for defining the ODD
Automatic
tool for
deriving ODD |  The described improvement would encompass a tool that can take as input the context, requirements based on the context and test cases. It would then propose an appropriate ODD, that is valid in the desired context, entails all requirements, and is verified through test cases.
Complete-
ness criteria
for ODD |  One interviewee suggested that an explicit method and criteria are needed to proof that the ODD is complete and correct.
Information
should be
described
clearly |  The ODD should contain more information about the context of the system. According to one interviewee, this would improve identifying wrong assumptions about the context early.
Measure of
exposure to
a safety event |  One interviewee explained that the hazard and risk assessment to evaluate the dimensions of the ODD from a safety viewpoint. The HARA includes assumptions about the context, and especially the exposure to a hazardous context can then be used to decide on the ODD.
Standardised
process |  All interviewees suggested to improve the standardisation of processes for defining the ODD.
Improvement ideas for the process and communication
Better
continuous
improvement |  Automated driving is a new technology, which needs to evolve continuously. The processes for deriving relevant artefacts, such as the context definition and ODD need to evolve together with the technology.
Faster
feedback |  One suggested improvement was to derive requirements faster, and creating faster feedback loops between the stakeholders. This expedites also the definition of the context, because assumptions can be made and verified faster.
Improved
leveraging from
other contexts |  Typically, automated driving systems are developed for and tested in confined areas, such as factory areas or harbours. Lessons are learnt in these confined contexts, and it is important to be able to leverage the knowledge from these confined contexts into new contexts.
Improved
integration
into
SAFe setup |  It is perceived that requirement engineering is not well integrated in scaled agile frameworks (SAFe), which hinders an efficient development and handling of requirements, and consequently context definitions.
Involvement
of function
developers |  Function developers should be more involved in the requirement engineering process, including context definition and deriving the ODD. On one hand, the function developers would get a better understanding of the requirements and context. On the other hand, they can contribute with deep knowledge about the used technologies, which eases the understanding of the the technology’s capability for desired contexts.

##  5 Discussion

###  5.1 Triangulation with background literature

Already in 2001, Dey described that for computing environments there is only
”an impoverished understanding of what context is and how it can be used”
[14]. The results from the interviews show that although the understanding
about context definitions have increased over 20 years, it is still difficult
to use context definitions in practise. A main reason for this seems to be
lack of standardisation and processes when dealing with context definitions,
and ODDs as abstraction thereof. Some attempts for standardisation of an ODD
taxonomy have been attempted recently, such as described in [12, 26] or
through pending standards such as ISO/TR 4804:2020 [11]. This lack of
standardisation in regards to ODDs (as abstraction of context definitions) has
also been described by Gyllenhammar et al. [9]. The difficulty to capture all
scenarios in an ODD has also been described in [17] and arguing for
completeness has been discussed in [10]. A theme mentioned by all interviewees
was that the process for deriving context definitions is difficult in the
sense that it contains too many uncertainties. Damak et al. identified this
difficulty as well, and developed a method to adopt architectural decisions
for automated driving systems to the operational context [8]. Thron et al.
observed challenges in the communication of ODDs between stakeholders of a
system [29]. A majority of interviewees, including all function developers,
described a lack of transparency in requirement negotiations (and context
definition) for desired use cases, which indicates that the communication
problem is not solved.

###  5.2 Discussion and main findings

Keeping the background literature in mind, we argue for four main findings
that can trigger future investigations: Firstly, we identified confusion in
the definitions of context, and operational design domain. The connection
between the context definition and operational design domain is ambivalent and
requires more clarification, for example through standardisation. Furthermore,
we identified a lack of clear processes leading to context definitions and
ODDs. Although many attempts of creating some form of standard or template
process exist, especially for the case of ODD, there is no clear picture in
our case. Major problems are arguing for completeness and lack of stakeholder
involvement. Also, we noticed problems when defining and documenting
assumption about the context. Interviewees reported that assumptions about the
context are not documented as such, and therefore it is difficult to
differentiate assumptions about the context from facts during function
development. Lastly, we observed a disconnection of the function developers
from the requirement engineering, which also includes the context definition
from use cases. Especially with the introduction of more agile frameworks, it
is beneficial to move parts of the requirement engineering towards the
function developers, including defining context from use cases and deriving of
ODDs. Applying machine learning in systems for automated driving systems
requires that the context of the systems can be clearly defined and described
in well working processes. Machine learning is a key technology for perception
systems in automated driving. Often implicitly, by selecting data sets for
training and validation, machine learning models are limited to the context
represented in these data sets. Specifically, a machine learning system
requires not only an understanding of the desired behaviour (given through use
cases, and functional requirements), but also of the context in which the
system operates. The context is important, because it defines both the
necessary training and necessary testing scenarios of the system. The training
scenarios, and in most learning scenarios the training dataset, define the
final behaviour of the machine learning system. Desired behaviour and context
are therefore closely intertwined, and that might be a reason for the
difficulties observed in defining the context and consequently the ODD for a
machine learning system. Should the desired behaviour in form of use cases be
defined first, or should the desired context be first? We saw in the answers
that there is no clear picture on the order and the processes of defining use
cases, context and ODD. This interlacement of use case and context, and the
lack of established processes, could be the reason for the overly cautious
definitions of ODDs that was reported in the interviews. We argue that a
better understanding is needed how the context influences the desired
behaviour of a system with machine learning components. This relation between
desired behaviour and context needs to be made explicit, in order to
understand the consequences on the desired behaviour of context changes. Based
on the explicit definition of the context and its relation to the desired
behaviour of the system, data requirements for training and validation data
can be derived (see also [31]).

###  5.3 Threats to validity

The study focused on context definitions for the development of automated
driving systems that use machine learning for the perception system. Automated
driving system are often considered context-aware systems and therefore the
findings of this study could potentially be transferred to other context-aware
systems. Most of the interviews were conducted at one Tier 1 supplier company
with offices in Sweden and the United States. To support generalisability of
the results, a sampling strategy was chosen that included different roles on
different levels and at both locations of the case company. In addition,
individual interviewees outside of the case company were included in the
study. A threat to validity is the sole focus on the automotive industry. In
order to support transferability of the results to other fields, the interview
questions were formulated with the intent of being non-specific to the
automotive field.

###  5.4 Conclusion

This case study was conducted in the setting of an automotive supplier company
by collecting qualitative data through interviews with automotive experts,
thematic analysis of the data, and validation of findings through a focus
group and background literature triangulation. The results show a lack of
standardisation of concepts and processes for defining the operational context
and deriving the ODD for automated driving systems. Because of the typically
distributed development of systems in the automotive industry, this creates
challenges which lead to misinterpretation and slow iteration loops between
the stakeholders. A major problem the study identifies are missing
documentations of context assumptions. Whether a context is assumed, or
explicitly given through a use case, can make a difference during the function
development and testing. Furthermore, the study reveals a lack of involvement
of function developers in the requirement engineering activities that lead to
the context definition. As a result, function developers often misinterpret or
question the defined operational context.

The study also elicited possible solutions to the challenges. Besides obvious
solutions, such as more standardisation and deeper involvement of function
developers in the definition of the operational context, the participants also
suggested ideas such as diverse data about the context, completeness criteria
for the ODD, more efficient leveraging from other contexts, and improved
integration of context definitions into scaled agile frameworks. These ideas
can serve future efforts and research towards a standardisation of context
definitions for automated driving systems or other context-aware systems that
use AI.

## References

  * [1] Brown, P.J.: The stick-e document: a framework for creating context-aware applications. Tech. Rep. June, Electronic Publishing-Chichester (1996)
  * [2] Chen, H., Finin, T., Joshi, A.: An ontology for context-aware pervasive computing environments. The Knowledge Engineering Review 18(3), 197–207 (2003)
  * [3] Cockburn, A.: Writing effective use cases. Tech. rep., Addison-Wesley Longman (2000)
  * [4] Colwell, I., Phan, B., Saleem, S., Salay, R., Czarnecki, K.: An Automated Vehicle Safety Concept Based on Runtime Restriction of the Operational Design Domain. Intelligent Vehicles Symposium, Proceedings pp. 1910–1917 (2018)
  * [5] Creswell, J.W.: Research Design: Qualitative, Quantitative, and Mixed Methods Approaches. Sage Publications, 4th edn. (2014)
  * [6] Creswell, John W.; Poth, C.N.: Qualitative Inquiry and Research Design: Choosing Among Five Approaches. Sage Publishing, fourth edn. (2017)
  * [7] Czarnecki, K.: Operational Design Domain for Automated Driving Systems - Taxonomy of Basic Terms (2018)
  * [8] Damak, Y., Leroy, Y., Trehard, G., Jankovic, M.: Operational context-based design method of autonomous vehicles logical architectures. In: 15th International Conference of System of Systems Engineering (SoSE). pp. 439–444. IEEE (2020)
  * [9] Gyllenhammar, M., Johansson, R., Warg, F., Chen, D., Heyn, H.M., Sanfridson, M., Söderberg, J., Thorsén, A., Ursing, S.: Towards an Operational Design Domain That Supports the Safety Argumentation of an Automated Driving System. 10th European Congress on Embedded Real Time Systems pp. 1–10 (2020)
  * [10] Henricksen, K., Indulska, J.: A software engineering framework for context-aware pervasive computing. In: Proceedings of the Second Annual Conference on Pervasive Computing and Communications. pp. 77–86. IEEE (2004)
  * [11] International Organization for Standardization: ISO/TR 4804:2020 Road vehicles — Safety and cybersecurity for automated driving systems — Design, verification and validation. International Organization for Standardization, Geneva (2020), www.iso.org
  * [12] Irvine, P., Zhang, X., Khastgir, S., Schwalb, E., Jennings, P.: A two-level abstraction odd definition language: Part i. In: 2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC). pp. 2614–2621. IEEE (2021)
  * [13] Jackson, M.: The world and the machine. In: 17th International Conference on Software Engineering (ICSE). pp. 283–283. IEEE (1995)
  * [14] K. Dey, A.: Understanding and using context. Personal and ubiquitous computing pp. 4–7 (2001)
  * [15] Knauss, A., Damian, D., Franch, X., Rook, A., Múller, H.A., Thomo, A.: Acon: A learning-based approach to deal with uncertainty in contextual requirements at runtime. Information and Software Technology 70, 85–99 (2016)
  * [16] Knauss, A., Damian, D., Schneider, K.: Eliciting contextual requirements at design time: A case study. In: 4th International Workshop on Empirical Requirements Engineering (EmpiRE). pp. 56–63. IEEE (2014)
  * [17] Koopman, P., Fratrik, F.: How many operational design domains, objects, and events? In: Proceedings of AAAI Workshop on Artificial Intelligence Safety. Honolulu, USA (2019)
  * [18] Nemoto, Y., Uei, K., Sato, K., Shimomura, Y.: A Context-based Requirements Analysis Method for PSS Design. Procedia CIRP 30, 42–47 (2015)
  * [19] NHTSA: Automated Driving Systems: a vision for safety (2017)
  * [20] Palinkas, L.A., Horwitz, S.M., Green, C.A., Wisdom, J.P., Duan, N., Hoagwood, K.: Purposeful sampling for qualitative data collection and analysis in mixed method implementation research. Administration and Policy in Mental Health and Mental Health Services Research 42(5), 533–544 (2015). https://doi.org/10.1007/s10488-013-0528-y
  * [21] Pfeffer, R., Basedow, G.N., Thiesen, N.R., Spadinger, M., Albers, A., Sax, E.: Automated driving - challenges for the automotive industry in product development with focus on process models and organizational structure. In: 2019 International Systems Conference (SysCon). pp. 1–6. IEEE (2019)
  * [22] Ramirez, A.J., Jensen, A.C., Cheng, B.H.C.: A taxonomy of uncertainty for dynamically adaptive systems. In: 7th International Symposium on Software Engineering for Adaptive and Self-Managing Systems (SEAMS). pp. 99–108. IEEE (2012)
  * [23] Reschka, A., Böhmer, J.R., Nothdurft, T., Hecker, P., Lichte, B., Maurer, M.: A surveillance and safety system based on performance criteria and functional degradation for an autonomous vehicle. Conference on Intelligent Transportation Systems, Proceedings (ITSC) pp. 237–242 (2012)
  * [24] SAE: J3016B Taxonomy and Definitions for Terms Related to Driving Automation Systems for On-Road Motor Vehicles. Tech. rep., SAE International (2018), https://www.sae.org/standards/content/j3016_201806/
  * [25] Saldaña, J.: The coding manual for qualitative researchers. Sage Publishing, second edn. (2013)
  * [26] Schwalb, E., Irvine, P., Zhang, X., Khastgir, S., Jennings, P.: A two-level abstraction odd definition language: Part ii. In: 2021 IEEE International Conference on Systems, Man, and Cybernetics (SMC). pp. 1669–1676. IEEE (2021)
  * [27] Shalev-Shwartz, S., Shammah, S., Shashua, A.: On a Formal Model of Safe and Scalable Self-driving Cars. arXiv pp. 1–37 (2017)
  * [28] Soultana, A., Benabbou, F., Sael, N.: Context-awareness in the smart car. In: Proceedings of the 4th International Conference on Smart City Applications (SCA). pp. 1–8. ACM Press, New York, New York, USA (2019)
  * [29] Thorn, E., Kimmel, S., Chaka, M.: A Framework for Automated Driving System Testable Cases and Scenarios (2018), https://www.nhtsa.gov/sites/nhtsa.dot.gov/files/documents/13882-automateddrivingsystems_092618_v1a_tag.pdf
  * [30] University of Oxford: Oxford Learner’s Dictionary, Entry: Context (2021), https://www.oxfordlearnersdictionaries.com/definition/english/context
  * [31] Vogelsang, A., Borg, M.: Requirements engineering for machine learning: Perspectives from data scientists. In: IEEE 27th International Requirements Engineering Conference (RE). pp. 245–251. IEEE (2019)