# A Survey on Uncertainty Reasoning and Quantification for Decision Making:
Belief Theory Meets Deep Learning

Zhen Guo*, Zelin Wan*, Qisheng Zhang  zguo, zelin<EMAIL_ADDRESS>https://orcid.org/0000-0002-6563-5934, https://orcid.org/0000-0001-5293-0363,
https://orcid.org/0000-0001-8785-8437 Department of Computer ScienceVirginia
Tech7054 Haycock RoadFalls ChurchVAUSA22043 ,  Xujiang Zhao*, Feng Chen
xujiang.zhao<EMAIL_ADDRESS>https://orcid.org/0000-0003-4950-4018,
https://orcid.org/0000-0002-4508-5963 Department of Computer ScienceUniversity
of Texas at Dallas800 W Campbell RdRichardsonTXUSA75080 ,  Jin-Hee Cho, Qi
Zhang  jicho<EMAIL_ADDRESS>https://orcid.org/ 0000-0002-5908-4662,
https://orcid.org/0000-0002-3607-3258 Department of Computer ScienceVirginia
Tech7054 Haycock RoadFalls ChurchVAUSA22043 ,  Lance M. Kaplan
<EMAIL_ADDRESS>https://orcid.org/0000-0002-3627-4471,
https://orcid.org/ US Army Research Laboratory2800 Powder Mill
Rd.AdelphiMDUSA20783 ,  Dong H. Jeong<EMAIL_ADDRESS>https://orcid.org/0000-0001-5271-293X Department of Computer Science of
Information TechnologyUniversity of the District of Columbia4200 Connecticut
Ave NWWashington, DCUSA20008 and  Audun Jøsang<EMAIL_ADDRESS>https://orcid.org/0000-0001-6337-2264 Department of InformaticsUniversity of
OsloOle-Johan Dahls hus GaustadalléenOsloNorway23b 0373

(2022)

###### Abstract.

An in-depth understanding of uncertainty is the first step to making effective
decisions under uncertainty. Deep/machine learning (ML/DL) has been hugely
leveraged to solve complex problems involved with processing high-dimensional
data. However, reasoning and quantifying different types of uncertainties to
achieve effective decision-making have been much less explored in ML/DL than
in other Artificial Intelligence (AI) domains. In particular, belief/evidence
theories have been studied in KRR since the 1960s to reason and measure
uncertainties to enhance decision-making effectiveness. We found that only a
few studies have leveraged the mature uncertainty research in belief/evidence
theories in ML/DL to tackle complex problems under different types of
uncertainty. In this survey paper, we discuss several popular belief theories
and their core ideas dealing with uncertainty causes and types and quantifying
them, along with the discussions of their applicability in ML/DL. In addition,
we discuss three main approaches that leverage belief theories in Deep Neural
Networks (DNNs), including Evidential DNNs, Fuzzy DNNs, and Rough DNNs, in
terms of their uncertainty causes, types, and quantification methods along
with their applicability in diverse problem domains. Based on our in-depth
survey, we discuss insights, lessons learned, limitations of the current
state-of-the-art bridging belief theories and ML/DL, and finally, future
research directions.

Belief theory, uncertainty reasoning, uncertainty quantification, decision
making, machine/deep learning

††copyright: acmcopyright††journalyear: 2022††ccs: Computing methodologies
Knowledge representation and reasoning††ccs: Computing methodologies Machine
learning algorithms

##  1\. Introduction

###  1.1. Motivation

In all sorts of business processes and our private life, we are confronted
with various kinds of decisions involving multiple choices and relative
uncertainty. A clear understanding of the uncertainty is a prerequisite of
sound and effective decision making. Although the topic of reasoning and
decision making under uncertainty has been studied for decades in various
artificial intelligence (AI) domains, including belief/evidence theory, game
theory, and machine/deep learning (ML/DL), the different manifestations of
uncertainty based on its root causes have not been investigated in-depth. The
era of the Internet and Big Data has brought a flood of information which can
be leveraged for decision making. Under such a situation, the challenge for
timely, accurate decision making is no longer the lack of information, but the
risk from a lack of understanding and managing inherent uncertainty resulting
from unreliable, incomplete, deceptive, and conflicting information.

In AI, a series of belief or evidence theories have a long history studying
reasoning and/or decision making under uncertainty. However, there has been
still limited understanding since uncertainty is not caused only by a lack of
evidence or unpredictability. In addition, ML/DL algorithms have considered
uncertainty (e.g., aleatoric or epistemic uncertainty) to offer solutions for
effective decision making. However, there has been no common, solid
understanding of multidimensional uncertainty where each domain has a
different and/or limited understanding in uncertainty even if they are in the
pursuit of a common goal of effective decision making.

Our survey paper aims to conduct an in-depth survey on a series of belief
models and introduce a new solution domain that can leverage uncertainty
research in belief/evidence theory to develop ML/DL solutions for attaining
effective decision making. In particular, we are interested in quantifying
different types of uncertainty caused by different root causes. This will help
provide solutions for ML/DL that can meet explainable AI, the so-called XAI,
by providing how uncertainty derives from, what is the reason behind, and
ultimately how it affects the effectiveness of decision making.

The state-of-the-art decision making research has been fully recognized the
significant importance of considering uncertainty for effective decision
making. However, there has been little study that has extensively surveyed
existing belief models to study uncertainty and its applicability for decision
making in the ML/DL domain.

###  1.2. Comparison of Our Survey Paper with Other Existing Survey Papers

In this section, we discuss existing survey papers that have discussed
uncertainty research. And then, we identify the key differences between the
existing survey papers and our survey paper.

Li et al. (2012) discussed the causes of different uncertainties and how to
process them in belief models for making effective decisions in various
domains. According to the nature of aleatory and epistemic uncertainty, they
classified uncertainty types processing in probability theory, fuzzy theory,
information-gap theory and derived uncertainty theory for their comparison.
They focused on how different uncertainties can be processed in data
management techniques. Kabir et al. (2018) surveyed prediction interval
techniques using deep neural networks (DNNs). The prediction interval
techniques quantify the level of uncertainty or randomness and have been
widely applied in the medical and electricity fields. They discussed aleatoric
and epistemic uncertainty (see Section 2 for their definitions) to explain
uncertainty in prediction using DNNs. They also discussed how a Bayesian
method is used to optimize the weight of an NN during training and applied for
NN-based prediction intervals in various fields.

Hariri et al. (2019) surveyed various AI techniques, including ML, Natural
Language Processing (NLP), and computational intelligence, that can recognize
and reduce uncertainty in Big Data. Abdar et al. (2021) reviewed over 700
papers that studied uncertainty quantification in ML/DL. They mainly focused
on discussing Bayesian and ensemble techniques and their applications in
various related areas, such as image processing, computer vision, medical
applications, NLP, and text mining.

We also discuss the contributions of the survey papers focusing on the
uncertainty mainly in ML/DL in the following papers. Hüllermeier and Waegeman
(2021) distinguished aleatoric uncertainty from epistemic uncertainty. They
explained how these two uncertainties are represented in various ML problems
or models and can contribute to decision making under the assessed
uncertainty. Ulmer (2021) surveyed the methods of quantifying uncertainty in
evidential deep learning model based on the conjugate prior and posterior
distributions and unknown outlier samples. This model estimates uncertainty
from the Dirichlet distribution by data (aleatoric) uncertainty, model
(epistemic) uncertainty, and distributional uncertainty. Gawlikowski et al.
(2021) provided a comprehensive survey on the uncertainty in DNNs. They
discussed two types of uncertainties: reducible uncertainty and unreducible
uncertainty.’ The concept of reducible uncertainty is aligned with that of
epistemic uncertainty where the reducible uncertainty can be introduced by
variability in a real-world, errors in model structure, or in training
parameter (i.e., batch size, optimizer). Unreducible uncertainty means
uncertainty by noises in measurement (i.e., sensor noise) and is in line with
aleatoric uncertainty. The authors classified uncertainty estimation methods
based on the cross-combination of the nature (i.e., deterministic or
stochastic) and number (i.e., single or multiple) of DNNs. Since the
discussions of uncertainty in (Hüllermeier and Waegeman, 2021; Ulmer, 2021;
Gawlikowski et al., 2021) are very limited in the scope, we did not include
them in Table 1.

Unlike the existing survey papers above (Li et al., 2012; Kabir et al., 2018;
Hariri et al., 2019; Abdar et al., 2021), our paper provides an in-depth
survey of eight different belief models with emphasizing how to reason and
quantify uncertainty based on the root causes and types of the uncertainty. In
addition, we discussed how a belief model is applicable in the DL domain. This
will allow researchers to leverage both the solid methodologies of uncertainty
reasoning/quantification in belief models and DL techniques for attaining
effective decision making. Finally, in Table 1, we summarize the key
differences between our survey paper and the existing four survey paper on
uncertainty research. We selected the key criteria based on the common
discussion points covered by the existing survey papers considered in this
paper as well as the key discussion points made in our survey paper.

Table 1. Comparison of Our Survey Paper with the Existing Surveys on
Uncertainty Research

Key Criteria |  Our Survey (2022) |  Li et al. (2012) (2012) |  Kabir et al. (2018) (2018) |  Hariri et al. (2019) (2019) |  Abdar et al. (2021) (2021)
---|---|---|---|---|---
Ontology of uncertainty |  ✔ |  ▲ |  ▲ |  ▲ |  ▲
Causes of uncertainty |  ✔ |  ✔ |  ✔ |  ▲ |  ✔
Uncertainty reasoning & quantification in DST |  ✔ |  ✔ |  ✘ |  ✘ |  ✘
Uncertainty reasoning & quantification in TBM |  ✔ |  ✘ |  ✘ |  ✘ |  ✘
Uncertainty reasoning & quantification in DSmT |  ✔ |  ✘ |  ✘ |  ✘ |  ✘
Uncertainty reasoning & quantification in IDM |  ✔ |  ▲ |  ✘ |  ✘ |  ✘
Uncertainty reasoning & quantification in TVL |  ✔ |  ▲ |  ✘ |  ✘ |  ✘
Uncertainty reasoning & quantification in Fuzzy Logic |  ✔ |  ✔ |  ✘ |  ▲ |  ✘
Uncertainty reasoning & quantification in Bayesian Inference |  ✔ |  ✔ |  ✘ |  ▲ |  ▲
Uncertainty reasoning & quantification in Subjective Logic |  ✔ |  ✘ |  ✘ |  ▲ |  ✘
Uncertainty reasoning & quantification in Bayesian Deep Learning |  ✔ |  ✘ |  ▲ |  ✘ |  ✔
Applicability of Belief Models in Deep Learning |  ✔ |  ✘ |  ▲ |  ✘ |  ✘
Discussions of insights, lessons, and limitations of the existing uncertainty-aware approaches |  ✔ |  ▲ |  ▲ |  ▲ |  ✔
Discussions of future research directions |  ✔ |  ✔ |  ✔ |  ✔ |  ✔

✔: Fully addressed; ▲: Partially addressed; ✘: Not addressed at all; DST:
Dempster-Shafer Theory; TBM: Transferable Belief Model, DSmT: Dezert-
Smarandache Theory; TVL: Three-Valued Logic.

###  1.3. Research Questions

In this work, we aim to answer the following research questions:

RQ1.:

What are the key causes and types of uncertainty studied in belief theory and
deep learning?

RQ2.:

How can the ontology of uncertainty be defined based on the multidimensional
aspects of uncertainty studied in belief models and deep learning?

RQ3.:

How has each belief model considered and measured uncertainty?

RQ4.:

How has each belief model been applied in deep learning and vice-versa for
effective decision making under uncertainty?

RQ5.:

What are the key differences of uncertainty reasoning and quantification in
belief theory and deep learning?

RQ6.:

How can belief model(s) be applied in deep learning to solve complicated
decision making problems?

The research questions above will be answered in Section 6.

###  1.4. Scope & Key Contributions

Although uncertainty has been considered in various domains, we limit the
scope of our paper to belief models and their applications in DL algorithms.
Note that when we refer to ‘decision making’, we mean a choice among multiple
alternatives. For example, it can be a certain class in classification tasks
to maximize prediction accuracy, an action chosen among multiple actions
available to maximize a decision utility, or a strategy chosen for optimizing
system performance.

In this paper, we made the following key contributions:

  1. (1)

We are the first conducting an extensive survey on identifying the causes and
types of uncertainty studied in various belief models and deep learning and
provides the ontology of uncertainty.

  2. (2)

We first investigate how various belief theories have considered uncertainty
and quantified it for effective decision making.

  3. (3)

We also first discuss how belief theories can be effectively leveraged for
deep learning-based solutions for decision making.

  4. (4)

We identify the key commonalities and differences about how each belief theory
reasons and quantifies uncertainty and how it is applied in the context of
deep learning or along with it.

  5. (5)

We provide the overall perspectives of insights and lessons learned as well as
the limitations from our extensive survey and suggest promising future
research directions.

###  1.5. Structure of the Paper

The rest of the paper is organized as follows:

  * •

Section 2 provides various classification types of uncertainty, the causes of
different types of uncertainties, and a proposed uncertainty ontology based on
the surveyed multidimensional concepts of uncertainty.

  * •

Section 3 provides the details of eight belief models and discusses belief
formation, causes and types of uncertainty, uncertainty quantification, and
its application as decision making application. The eight belief models
include Dempster Shafer Theory (DST), Transferable Belief Model (TBM), Dezert-
Smarandache Theory (DSmT), Imprecise Dirichlet Model (IDM), Kleene’s Three-
Valued Logic (TVL), Fuzzy Logic (FL), Bayesian Inference (BI), and Subjective
Logic (SL).

  * •

Section 4 discusses how a belief theory can be applied in the context of DL as
decision making applications under uncertainty, particularly in terms of
evidential neural networks, fuzzy deep neural networks, and rough deep neural
networks.

  * •

Section 5 provides the answers to the key research questions raised in Section
1.

  * •

Section 6 concludes our paper by discussing the limitations, insights, and
lessons learned from our survey. In addition, we suggest promising future
research directions in applying belief models to solve DL-based decision
making problems.

Caveat of Mathematical Notations: In Sections 3 and 4, we discuss a set of
belief theories and deep learning theories leveraging belief models, including
Subjective Logic, Fuzzy theory, and rough set theory. The discussion of a
theory needs to use mathematical notations which are only used under the
theory, not other theories. We keep the mathematical notations of original
papers in order to deliver the common notations used to discuss each theory in
the literature.

##  2\. Classification Types, Causes, and Ontology of Uncertainty

In this work, we deal with uncertainty in data or information. We define an
uncertainty type as a perceived state of data or information, such as
fuzziness, discord, non-specificity, ambiguity, and so forth (see Section
2.1). We define the causes of uncertainty by the reason introducing
uncertainty in a decision maker’s judgment. We also discuss the ontology of
uncertainty where ontology is studied as a branch of philosophy which is
defined as the “science of what it is” describing “the structures of objects,
properties, events, processes, and relations in every area of reality (Smith,
2012). In this section, we will describe the ontology of uncertainty in terms
of its types, causes, and outcomes of decision making based on uncertainty
reasoning and quantification.

###  2.1. Classification of Uncertainty Types

In the probabilistic uncertainty research community (Jøsang, 2016; Kiureghian
and Ditlevsen, 2009), two types of uncertainty natures are widely used and
commonly discussed by:

  * •

Aleatoric uncertainty: This refers to statistical uncertainty about the long
term relative frequencies of possible outcomes (Jøsang, 2016). For example, if
we do not know whether a dice is loaded – and thereby unfair – then we are
faced with aleatoric uncertainty. This uncertainty can be reduced to the true
variance about the loaded dice by throwing the dice sufficiently many times.
However, every time a dice is thrown, we cannot predict its outcome exactly
but can have only a long-term probability (Jøsang, 2016). In this sense, the
long-term probability can reduce epistemic uncertainty by more and more
observations. Therefore, aleatoric uncertainty is fundamentally related to the
nature of randomness in which a variable is governed by a frequentist process
(Kiureghian and Ditlevsen, 2009; Jøsang, 2016).

  * •

Epistemic uncertainty: This uncertainty is related to a situation that we
cannot predict exactly an event because of a lack of knowledge. A typical
example is the assassination of President Kennedy in 1963 (Jøsang, 2016),
where the uncertainty is about whether he was killed by Lee Harvey Oswald and
who organized it. The nature of epistemic uncertainty derives from a lack of
knowledge or information (or data). This type of uncertainty is also called
systematic uncertainty or model uncertainty. This means that the outcome of a
specific future or past event can be known, but there is insufficient evidence
to support it. The epistemic uncertainty can be reduced by more evidence, more
advanced technology, and/or scientific principles to interpret the evidence
(e.g., forensic science) (Kiureghian and Ditlevsen, 2009). This epistemic
uncertainty follows a non-frequentist process representing the likelihood of
an event (Jøsang, 2016).

Since the above two natures of uncertainty have been most widely discussed as
the nature of uncertainty, we will discuss how a different type of uncertainty
in different belief models and DL models is related to these two natures of
uncertainty.

Uncertainty reasoning and quantification research has been heavily explored by
several theories, such as probability theory, fuzzy sets theory, possibility
theory, evidence theory, and rough sets theory. These theories can be seen as
complementary as each of them is designed to deal with different types of
uncertainty. In these theories, Dubois (1980) identified three uncertainty
types in terms of fuzziness, discord, and nonspecificity. The latter two
terms, discord and non-specificity, are combined as the term ambiguity. Each
type is represented by a brief common-sense characterization and several
pertinent synonyms as follows:

  * •

Fuzziness: This results from a lack of definite or sharp distinctions and has
pertinent synonyms, such as vagueness, cloudiness, haziness, unclearness,
indistinctness, or sharplessness.

  * •

Ambiguity: In general, there exists ambiguity when an object cannot be
specified due to a lack of certain distinctions or detected as a single class
due to conflicting evidence. Hence, these two cases can be further classified
into the following two subclasses:

    * –

Discord: This is associated with disagreement among several alternatives and
interchangeably used with synonyms including dissonance, incongruity,
discrepancy, or conflict.

    * –

Nonspecificity: This refers to a situation in which two or more alternatives
are left unspecified and is used with pertinent synonyms, such as variety,
generality, diversity, equivocation, and imprecision.

Based on our understanding, fuzziness introduces vagueness (i.e., failing in
distinguishing one from another) while ambiguity belongs to epistemic
uncertainty. In addition to the above, the most common uncertainty is also
derived from a lack of evidence, called vacuity (or ignorance) (Jøsang, 2016)
as we do not know how to make a decision because of insufficient evidence,
which belongs to epistemic uncertainty as more evidence can reduce vacuity.

Figure 1. Classification of uncertainty types.

In the modeling and risk assessment research (Linkov and Burmistrov, 2003;
Walker et al., 2003), uncertainty associated with choices made by modelers has
been studied, such as differences in problem formulation, model
implementation, and parameter selection originating from subjective
interpretation of the problem at hand. We call this ‘modeler uncertainty’
which has been categorized by the following three types:

  * •

Parameter uncertainty: This refers to uncertainty derived from the values of
input parameters in a model, such as measurement errors, sampling errors,
variability, and use of surrogate data. Hence, it is a type of epistemic
uncertainty and can be reduced by collecting more reliable evidence to more
accurately estimate the parameters used in the model.

  * •

Model uncertainty: This indicates uncertainty about a model structure and the
mathematical relationships of components defined in the model. For example,
uncertainty can be introduced by making assumptions and simplifying
mathematical equations in modeling real-world problems. Thus, this uncertainty
is introduced by missing or incomplete information, which makes hard to fully
define the model. This uncertainty belongs to epistemic uncertainty and can be
reduced by gathering necessary, reliable information to accurately define the
model.

  * •

Scenario uncertainty: This represents uncertainty caused by normative choices
made on constructing scenarios, including the choice of a functional unit,
time horizon, geographical scale, and other methodological choices. This
uncertainty arises from uncertain problem formulations and theoretic
assumptions, which are not statistical in nature. Due to this nature of
uncertainty, we understand this uncertainty as epistemic uncertainty. This
uncertainty can be reduced by collecting more evidence, using more advanced
technology, and/or considering scientific principles to interpret the
evidence, such as finding a better choice of a time horizon or a geographical
scale for crime hotspot detection.

###  2.2. Causes of Uncertainty

From the perspective of framing research in decision making, where frames are
heuristic representations of the external world, there are three main causes
of uncertainty, including unpredictability, incomplete knowledge, and multiple
knowledge frames (Brugnach et al., 2008). From an engineering perspective,
these three causes can be introduced by a lack of evidence, limited cognition
to process a large amount of evidence, conflicting evidence, ambiguity,
measurement errors, and subjective beliefs (Zimmermann, 2000). The three
causes are described by:

  * •

Unpredictability: A system (or entity/data) may exhibit chaotic, variable
behavior in space/time. In Statistics, confidence intervals have been used as
a measure of uncertainty (Zimmermann, 2000). Statistical noise is a common
factor triggering uncertainty, leading to unpredictability. Even if the system
learns and adapts to dynamic, new conditions, it exhibits highly variable
behaviors. The variability may be due to unreliability in information, data,
or an entity, which is caused by system/network dynamics, non-stationary
environmental conditions, or adversarial attacks. If this is the case, this
type of uncertainty can be reduced by detecting and excluding the unreliable
sources or data in decision making process (Zimmermann, 2000).

  * •

Incomplete Knowledge: This refers to situations where we do not know enough
about a system to be managed or our knowledge about the system is incomplete
(i.e., epistemic uncertainty) (Zimmermann, 2000). This can be due to a lack of
evidence (e.g., information/data) or a lack of knowledge because we may not
have sufficient theoretical understanding (e.g., ignorance) or reliable
information or data. This uncertainty can be reduced by considering more
evidence or discarding unreliable evidence. In addition, when human decision
makers receive a large amount of information, which is often highly complex,
they cannot process them properly because of their limited cognition and
processing power. To deal with this, people usually transform available data
into information with a rougher ‘granularity’ or focus on important features,
neglecting other less important (or noisy) information or data. This
uncertainty can be reduced by considering relevant information among the
available information (Zimmermann, 2000).

  * •

Multiple Knowledge Frames: This refers to the case when same information
(e.g., evidence or opinions) is interpreted differently, resulting in
different, conflicting views. Dewulf et al. (2005) defined ambiguity as the
presence of multiple, valid beliefs about a certain phenomenon. The ways of
understanding the system (or the external world) can differ in where to put
the boundaries of the system or what and who to put as the focus of attention.
The differences can also emerge from the way in which the information about
the system is interpreted. Another major cause is conflicting evidence,
representing a situation where some of the information available may be
incorrect, simply irrelevant, or the model to observe a system may not be
correct at a given time. Further, multiple observers may provide different
opinions based on their subjective views.

We demonstrated our view about the classifications of uncertainty types based
on the existing classifications in Fig. 1.

###  2.3. Ontology of Uncertainty

Ontology provides “a definitive and exhaustive classification of entities in
all spheres of being” where the classification should be able to fully
describe the details of the entities (Smith, 2012). The information fusion
research community has developed “Uncertainty Representation and Reasoning
Framework (URREF)” (Costa et al., 2018), which considers uncertainty ontology
in the information processing systems. However, its scope was limited only to
the information processing systems and only considered a subset of uncertainty
types studied in the uncertainty research domain. In order to more extensively
understand the concepts of uncertainty and its multiple causes, we develop an
ontology of uncertainty by using W3C Web Ontology Language (OWL) (Stanford
Center for Biomedical Informatics Research (BMIR), 2019), which is
demonstrated in Fig. 2. We do not show the subattributes of ambiguity and
fuzziness (see Fig. 1) in Fig. 2 due to the space constraint. Hence, we
describe more about the key attributes of uncertainty used above in Appendix F
of the supplement document.

Figure 2. An Ontology of Uncertainty.

##  3\. Decision Making under Uncertainty in Belief Theory

Different types of uncertainty affect the assessment and analysis of a
specific situation. Underlying uncertainty comes from how to view and model a
given part of the world which we call a _domain_. A domain is the abstract
representations of states of the world, where analysts or decision makers can
have beliefs about the true states of a domain. Beliefs about domains can be
easily biased by an analyst or a decision maker, which is often called
“framing effect” (Tversky and Kahneman, 1985; Walker et al., 2003), which can
cause subjective beliefs about the world to deviate from ground truth of the
world (e.g., past or future events) (Walker et al., 2003). The way a situation
is formally modeled (i.e., elements in a domain) can also affect the types and
levels of uncertainty perceived by a decision maker.

Belief has been based off for decision making process. In 1930s, Kleene (1938)
proposed Three-Valued Logic (TVL) by defining algebra based on three values
including false, unknown, and truth. Many other theories defining a belief
based on probabilities have been proposed since 1960’s, including Fuzzy Logic
(Zadeh, 1965), Dempster-Shafer Theory (Shafer, 1976), Transferable Belief
Model (Smets and Kennes, 1994), Subjective Logic (Jøsang, 1999, 2001), Dezert-
Smarandache Theory (DSmT) (Dezert and Smarandache, 2004), Bayesian Inference
(Fienberg, 2006), Imprecise Dirichlet Model (IDM) (Walley, 1996). The
Dempster-Shafer Theory (DST) (Shafer, 1976) first defined a “frame of
discernment,” the set of propositions considered. DST generalized Bayesian
theory based on subjective probability (Shafer, 1976). Transferrable Belief
Model (TBM) (Smets and Kennes, 1994) has been proposed to deal with more
knowledge and situations than DST. Zadeh (1965) introduced fuzzy set theory to
represent a uncertain, subjective belief based on a membership function
(Zadeh, 1983) and has been applied in various trust-based systems (Nagy et
al., 2008; Lesani and Bagheri, 2006; Chen et al., 2009; Liao et al., 2009; Luo
et al., 2008; Manchala, 1998; Nefti et al., 2005). DSmT (Dezert and
Smarandache, 2004) extended DST to deal with conflicting evidence in trust
management systems (Wang and Sun, 2007; Deepa and Swamynathan, 2014).

###  3.1. Kleene’s Three-Valued Logic (TVL)

####  3.1.1. Belief Formation.

Kleene (1938) first proposed TVL in 1938 . Its truth table is shown in Table
2, where $p_{1}$ and $p_{2}$ are two logical variables. TVL’s belief
distribution is determined by the logical values of logical variables, i.e.,
$\mathbf{b}(p)_{q}=1$ when $p=q$ and $0$ otherwise. The $p$ is a logical
variable and $q$ is a logical value from $\\{T,U,F\\}$ where $T$ is true, $U$
is unknown, and $F$ is false.

Table 2. Truth Table of the TVL

$p_{1}\wedge p_{2}$ $p_{1}$ $p_{2}$ | $T$ | $U$ | $F$
---|---|---|---
$T$ | $T$ | $U$ | $F$
$U$ | $U$ | $U$ | $F$
$F$ | $F$ | $F$ | $F$

$p_{1}\vee p_{2}$ $p_{1}$ $p_{2}$ | $T$ | $U$ | $F$
---|---|---|---
$T$ | $T$ | $T$ | $T$
$U$ | $T$ | $U$ | $U$
$F$ | $T$ | $U$ | $F$

Kleene Algebras and TVL. Kleene’s TVL is a special case of Kleene algebras.
The properties of Kleene algebra, $\mathcal{K}=(K,\vee,\wedge,\sim,F,T)$, are:

  * •

$\mathcal{K}$ is a bounded distributive lattice; and

  * •

$\forall a,b\in K$, $\sim(a\;\wedge b)=\sim a\;\vee\sim b$, $\sim\sim a=a$,
and $a\;\wedge\sim a\leq b\;\vee\sim b$.

Here we apply the semantics where $\vee$ means logical disjunction, $\wedge$
means logical conjunction, and $\sim$ means negation. It can be easily derived
that Kleene’s TVL, in which $K=\\{T,U,F\\}$, is a Kleene algebra where $T=\sim
F$, $F=\sim T$, and $U=\sim U$.

Rough Sets and Kleene Algebras. Kleene algebras are related to rough sets. The
relationships between these two concepts are as follows. Given an information
system, $I=(S,\mathbb{A})$, where $S$ is a set of objects and $\mathbb{A}$ is
a set of attributes $a:x\mapsto a(x)$ for any $x\in S$, we can define the set
of equivalence relationships, $IND(I)$:
$IND(I)=\\{IND(A):A\subseteq\mathbb{A}\\}$, where $IND(A)=\\{(x,y)\in
S^{2}:\forall a\in A,a(x)=a(y)\\}$. Given any equivalence relationship $R\in
IND(I)$, a rough set $\mathcal{X}\in(S\times S)/R$ is a pair
$(\underline{R}X,\overline{R}X)$, where $\underline{R}X$ and $\overline{R}X$
are called the $R$-lower and $R$-upper approximation of $X$, respectively.
More specifically,

(1) $\displaystyle\underline{R}X=\bigcup\\{Y\in S/R:Y\subseteq
X\\},\;\;\overline{R}X=\bigcup\\{Y\in S/R:Y\cap X\neq\emptyset\\},$

where $S/R$ is the collection of equivalence classes corresponding to $R$.

Given any set $S$ with $|S|\geq 2$, universal equivalence relationship
$R:=S\times S$ and information system $I=(S,\mathbb{A})$, we can induce a
three-valued algebra on a collection of rough sets, $\mathcal{RS}$, with the
Kleene semantics by:

(2) $\mathcal{RS}=\\{(\underline{R}A,\overline{R}A):A\subseteq
S\\}=\\{(S,S),(\emptyset,S),(\emptyset,\emptyset)\\}.$

Here, if we define $\sim\mathcal{X}:=(\underline{R}X^{c},\overline{R}X^{c})$,
we have $(S,S)=\sim(\emptyset,\emptyset)$, $(\emptyset,S)=\sim(\emptyset,S)$,
and $(\emptyset,\emptyset)=\sim(\emptyset,\emptyset)$. This means
$K\cong\mathcal{RS}$. In general, given the set of all logic functions
(propositional formula) denoted by $\mathcal{F}$, the set of all Kleene
algebras by $\mathcal{A}_{\mathcal{K}}$, and the collections of all rough sets
over all possible information systems by $\mathcal{A}_{\mathcal{RS}}$, the
following theorem (Kumar and Banerjee, 2017) is held:

(3)
$\forall\alpha,\beta\in\mathcal{F},\alpha\vDash_{\mathcal{A}_{\mathcal{K}}}\beta\Leftrightarrow\alpha\vDash_{\mathcal{A}_{\mathcal{RS}}}\beta.$

The above can be read by: For any logic functions $\alpha$ and $\beta$ in
$\mathcal{F}$, if $\beta$ is a semantic consequence of $\alpha$ in
$\mathcal{A}_{\mathcal{K}}$, then $\beta$ is a semantic consequence of
$\alpha$ in $\mathcal{A}_{\mathcal{RS}}$. We summarize the uncertainty-aware
decision making process using Kleene’s TVL in Fig. 3.

####  3.1.2. Causes and Types of Uncertainty

Uncertainty is formalized as a logical value $U$ (unknown) and its
relationship with two classical logical values $T$ (true) and $F$ (false) are
shown in Table 2. The stated uncertainty here refers to unpredictability
because of a lack of information or knowledge. For example, in rough sets, due
to unpredictable noises, sets are represented by approximation spaces.

####  3.1.3. Uncertainty Quantification

Figure 3. Uncertainty-aware decision making process using Kleene’s TVL.

Uncertainty in TVL represents an unknown or unspecified state in the decision
making using TVL. This is related to vacuity uncertainty caused by a lack of
information/knowledge or non-specificity. Since uncertainty is regarded as a
logical value, uncertainty value can be quantified through logical operations
of logical variables. In Kleene’s TVL, the three values of $T$, $U$
(uncertainty), and $F$ are often defined by 1, 0, and -1. As seen in Table 2,
uncertainty, $U$, can be ignored under $\wedge$ to decide $T$ or $F$ while it
can be used to support $T$ over $F$.

####  3.1.4. Applications of TVL on Machine/Deep Learning

Kashkevich and Krasnoproshin (1979) defined a function of TVL to solve
classification problems in pattern recognition tasks. They viewed accepted
accurate classification as $T$, accepted incorrect classification as $F$, and
refused classification as $U$. Dahl (1979) leveraged TVL to construct a
database used for natural language consultation. Codd (1986) applied TVL in
the area of SQLs, with “Null” value behaving like the uncertain value $U$ in
TVL. TVL is rarely observed for its applications in recent research. As the
topological generalization of TVL, rough sets are used in ML/DL, as described
in Section 4.3.

###  3.2. Dempster Shafer Theory (DST)

DST is a fusion technique for decision-making based on the belief mass (a.k.a.
evidence) of various detection systems. Each system can be defined as a set of
possible conclusions, called proposition (Shafer, 1976). The set of all
propositions is denoted by $\Theta$ (a.k.a. the frame of discernment (FOD)).
Given the set $\Theta$, we can generate the power set $P(\Theta)$ (a.k.a. the
powerset of FOD), where the $P(\Theta)$ represents all possible combination of
the set $\Theta$, including an empty set $\emptyset$. So, $2^{|\Theta|}$ is
the size of the $P(\Theta)$. As an example of $P(\Theta)$, if
$\Theta=\\{W,Z,L\\}$, then
$P(\Theta)=\\{\emptyset,\\{W\\},\\{Z\\},\\{L\\},\\{W,Z\\},\\{W,L\\},\\{Z,L\\},\\{W,Z,L\\}\\}$.

####  3.2.1. Belief Formation

The belief mass is an observed probability based on evidences. For example,
assuming we have a black ball, a black square, and a red ball. The mass for
focal element $black$ is $m(black)=\frac{2}{3}$, and the mass for focal
element $red$ is $m(red)=\frac{1}{3}$. For a given system, we assign a belief
mass to each element in power set $P(\Theta)$, and defined the mass function
as $m:P(\Theta)\rightarrow[0,1]$. The mass function is also called basic
belief assignment (bba), and the sum of the mass for each element in set
$P(\Theta)$ is equal to one, that is (Shafer, 1976):

(4) $m:\Theta\rightarrow[0,1],\sum_{A\in
P(\Theta)}m(A)=1,\;\;\textsf{where}\;\;m(\emptyset)=0.$

Given the power set $P(\Theta)$ and the corresponding belief mass $m$ for each
focal element (i.e., a subset) $A$ in $P(\Theta)$, we can calculate the belief
interval of each focal element $A$, and represent it as $[{Bel}(A),{pl}(A)]$.
The belief ${Bel}(A)$ is the lower bound and plausibility ${pl}(A)$ indicates
the upper bound (Smarandache et al., 2012). The ${Bel}(A)$ and ${pl}(A)$ are
obtained by:

(5) $\displaystyle{Bel}(A)=\sum_{B|B\subseteq A}m(B),\;\;{pl}(A)=\sum_{B|B\cap
A\neq\emptyset}m(B),\;\;{Dis}(A)=1-{pl}(A).$

Figure 4. Dempster’s Rule of Combination. Figure 5. Belief, Plausibility, and
Disbelief in Dempster Shafer Theory.

For example, given $\Theta=\\{W,Z,L\\}$ and the belief mass $m(W)$, $m(Z)$,
$m(W\text{ or }L)$, we can obtain beliefs of focal element $W$ and ($W$ and
$Z$) by ${Bel}(W)=m(W)$ and ${Bel}(W\text{ and }Z)=m(W)\cdot m(Z)$,
respectively, and plausibility of $W$ by $pl(W)=m(W)+m(W\text{ or }L)$. The
belief interval for $W$ is denoted by $[m(W),m(W)+m(W\text{ or }L)]$.

The disbelief of focal element $A$ is represented as ${Dis}(A)$, which equals
${Bel}(\overline{A})$, where $\overline{A}$ means the complement of $A$ (i.e.,
negation of $A$). The ${Dis}(A)$ is calculated by summing all masses of the
focal elements that do not intersect with $A$. Another way to calculate the
${Dis}(A)$ is ${Dis}(A)=1-{pl}(A)$ where ${pl}(A)$ can be considered as an
estimated uncertainty as a potential credit to increase the given belief.
While uncertainty is often considered risk (van Asselt, 2000), DST uses
uncertainty as a credit to support a particular belief. Fig. 5 describes the
key concept of DST.

Dempster’s Rule of Combination is a belief mass combination function for two
independent detection systems $i$ and $j$ over the same frame. The joint mass
committed to focal element $A$ is given by:

(6) $m(A)=\kappa\sum_{A_{i}\cap B_{j}=A\neq\emptyset}m_{1}(A_{i})\cdot
m_{2}(B_{j}),$

where $A_{i}$ and $B_{j}$ are values in set $\Theta$ of two different systems
$i$ and $j$ that contain target value $A$. The $\kappa$ is a renormalization
constant, defined by $\kappa=(1-\sum_{A_{i}\cap
B_{j}=\varnothing}m_{1}(A_{i})m_{2}(B_{j}))^{-1}$ (Shafer, 1976). For example,
we have $m_{1}(W)$, $m_{1}(Z)$, $m_{1}(W,Z)$, $m_{2}(W)$, $m_{2}(Z)$, and
$m_{2}(W\text{ or }Z)$. The joint mass for focal element $W$ is calculated by
$m(W)=m_{1}(W)\cdot m_{2}(W)+m_{1}(W)\cdot m_{2}(W\text{ or }Z)+m_{1}(W\text{
or }Z)\cdot m_{2}(W)+m_{1}(W\text{ or }Z)\cdot m_{2}(W\text{ or }Z)$.

We summarize the key concept of Dempter’s rule of combination in Fig. 4 based
on our discussion above. Many DST variants have been proposed. Due to the
space constraint, we discuss some variants of DST in Appendix A of the
supplement document.

####  3.2.2. Causes and Types of Uncertainty

DST considers uncertainty in plausibility due to a lack of evidence. This
implies that uncertainty in DST is closely related to epistemic uncertainty or
vacuity. Hence, DST can quantify an uncertain opinion as a subjective belief
in a given proposition (Shafer, 1976).

####  3.2.3. Uncertainty Quantification

Smarandache et al. (2012) measured uncertainty in DST based on its multiple
dimensions, including auto-conflict (i.e., conflict in a belief function with
conjunctive rule), non-specificity (i.e., a generic form of Hartley entropy
with base 2), confusion (i.e., uncertainty by a lack of evidence), dissonance
(i.e., all beliefs are mostly the same), aggregate uncertainty measure (AU)
(i.e., generalized Shannon entropy), and ambiguity measure (AM) (i.e., non-
specificity and discord). We provide the detail of each certainty explained
above in Appendix A of the supplement document. Blasch et al. (2013) defined
the Interval of Uncertainty (IOU) by:

(7) $IOU(A)=pl(A)-Bel(A)=1-Bel(\overline{A})-Bel(A)=1-Dis(A)-Bel(A).$

Klir and Ramer (1990) measured the total uncertainty in DST, denoted by
$U^{T}(A)$, by considering two types of uncertainty, non-specificity and
discord. Both $U^{T}(A)$ and AM consider non-specificity and discord and
differently capture them. AM captures them in a level of each proposition
(i.e., element $\theta\in\Theta$) while $U^{T}(A)$ obtains them at the level
of sets, $A\subset\Theta$. Hence, $U^{T}(A)$ is given by (Klir and Ramer,
1990):

(8)
$U^{T}(A)=\sum_{A\subset\Theta}m(A)\log_{2}\Bigg{(}\frac{|A|}{\sum_{B\subset\Theta}m(B)\frac{|A\cap
B|}{|B|}}\Bigg{)}.$

The key merit of DST is to combine an amount of uncertain evidence from
multiple sources and select elements based on the combined belief mass.
However, the combination rule of DST fails to balance different sources,
especially when sources provide conflicting evidence. Although many
alternative combination rules have been proposed, Dubois and Prade (1988)
argued that no single combination rule could be used as a universal solution
to all encountered situations.

####  3.2.4. Applications of DST on Machine/Deep Learning

DST has been used with deep learning techniques. Soua et al. (2016) proposed a
framework to predict traffic flow using a Deep Belief Network (DBN), a class
of deep neural network (DNN), to make an identical prediction based on two
types of data, including data streams and event data. Then DST was used to
fuse those two predictions. Tong et al. (2021) studied a set-valued
classification (SVC) where a sample can be classified as multiple classes, not
just a single class to identify outliers not represented in a training
dataset. The authors proposed a technique of combining DST with Convolutional
Neural Networks (CNNs) to improve the accuracy of the SVC. Tian et al. (2020)
proposed a new intrusion detection system (IDS) using DST with Long Short-Term
Memory Recurrent Neural Network (LSTM-RNN) to combine the results from
different classes. Zhang et al. (2020b) proposed a new fault diagnosis
technique using an improved DST by fusing data from multiple sensors for fault
classification.

###  3.3. Transferable Belief Model (TBM)

TBM was developed as a variant of DST to resolve unreasonable results of the
DST combination rule (see Section 3.2) when multiple sources provide
conflicting evidence (Smets and Kennes, 1994). TBM is based on the open-world
assumption with two levels of belief reasoning: credal level and pignistic
level. The credal level quantifies and updates a belief through a belief
function. The pignistic level transfers a belief into a probability using the
so-called pignistic probability function for making decision (Smets and
Kennes, 1994).

####  3.3.1. Belief Formation

TBM defines basic belief masses the same as DST (Shafer, 1976) (see Eq. (4)).
The credal level belief function updating a belief upon the arrival of new
evidence is formulated by (Smets and Kennes, 1994):

(9) $m_{B}(A)=\begin{cases}\frac{\sum_{C\subseteq\overline{B}}m(A\cup
C)}{1-\sum_{C\subseteq\overline{B}}m(C)}\;\;\text{for}\;\;(A\subseteq
B)\wedge(A\neq\emptyset);\\\
\;\;\;\;\;\;\;\;0\;\;\;\;\;\;\;\;\;\;\;\text{otherwise.}\end{cases}$

Here $m_{B}(A)$ means the belief mass supporting propositions $A$ when
conditional evidence does not support proposition $B$ as the truth (Smets and
Kennes, 1994). The $\sum_{C\subseteq\overline{B}}m(C)$ refers the sum of
beliefs supporting a set not supporting $B$ and
$\sum_{C\subseteq\overline{B}}m(A\cup C)$ is the sum of beliefs not supporting
$B$ or supporting $A$. For $m_{B}(A):\mathbb{R}\rightarrow[0,1]$, Eq. (9)
needs to hold $\sum_{C\subseteq\overline{B}}m(A\cup
C)<1-\sum_{C\subseteq\overline{B}}m(C)$.

The probability transformed through the pignistic probability function for
decision making is denoted by $BetP$, which is estimated by:

(10) $BetP(x)=\sum_{x\in A\subseteq X}\frac{m(A)}{|A|}=\sum_{x\in A\subseteq
X}m(A)\frac{|x\cap A|}{|A|},$

Figure 6. A belief at the credal level and pignistic level and their
relationships in TBM.

where $\frac{m(A)}{|A|}$ means belief mass, $m(A)$, is evenly distributed into
the atoms of $A$, a set of atoms, and $|A|$ means the number of atoms $x$ in
set $A$ (i.e., $x\in A$). The $X$ is the Boolean algebra of the subset of
$\Omega$, where $\Omega$ is a set of worlds (truth). The probability
distribution calculated from the pignistic probability function is used for
decision making.

####  3.3.2. Causes and Types of Uncertainty

TBM considers epistemic uncertainty caused by a lack of evidence.

####  3.3.3. Uncertainty Quantification

In TBM, uncertainty has not been explicitly discussed. Since a belief at the
pignistic level is for decision making in real-world settings, we can
understand the pignistic probability function gives a belief mass that
considers uncertainty in practice while the credal level belief function
estimates a belief based on observed evidence. We show how a belief is
constructed at the credal level based on the arrival of evidence and how the
credal level belief is transferred to the pignistic level belief for decision
making in Fig. 6.

####  3.3.4. Applications of TBM on Machine/Deep Learning

TBM has been used as a competitive algorithm to solve various types of
classification problems (Zhang et al., 2020a; Guil, 2019; Quost et al., 2005;
Honer and Hettmann, 2018; Henni et al., 2019). However, to the best of our
knowledge, we have not found any prior work that uses TBM along with ML/DL.

###  3.4. Dezert-Smarandache Theory (DSmT)

Dezert and Smarandache (2004) introduced DSmT theory for data and information
fusion problems as a general framework that provides new rules of handling
highly imprecise, vague, and uncertain sources of evidence and making
decisions under them. The main advantages of DSmT over DST are as follows.
First, DSmT has a more general fusion space in a hyper-power set (discussed in
the Appendix B.1 of the supplement document), compared to a power set. Second,
DSmT fits free and a hybrid model compared to a strict DST model (see Appendix
B.1 of the supplement document). Third, DSmT also combines complex classes
based on subsets or complements and introduces better fusion rules, such as
proportional conflict redistribution rule 5 (PCR5), dynamic fusion by hybrid
DSm rule (DSmH), a new probability transformation, qualitative operators for
data with labels (e.g., linguistic labels in natural language), and new belief
conditioning rules (BCRs), or new fusion rules for set-valued imprecise
beliefs.

####  3.4.1. Belief Formation

A generalized basic belief assignment (gbba) is formulated the same as a
belief function of DST in Eqs. (4) and (5), but the domain of DSmT is the
hyper-power set $D^{\Theta}$, compared to a power set $P(\Theta)$ of DST. Note
that $P(\Theta)\overset{\Delta}{=}(\Theta,\cup)$,
$D^{\Theta}\overset{\Delta}{=}(\Theta,\cup,\cap)$, and
$S^{\Theta}\overset{\Delta}{=}(\Theta,\cup,\cap,c(\cdot))$. If
$\Theta=\\{a,b\\}$, $P(\Theta)\overset{\Delta}{=}(\emptyset,a,b,a\cup b)$,
$D^{\Theta}\overset{\Delta}{=}(\emptyset,a,b,a\cup b,a\cap b)$, and
$S^{\Theta}=(\emptyset,a,b,a\cup b,a\cap b,c(\emptyset),c(a),c(b),c(a\cup
b),c(a\cap b))$ where $c(X)$ refers to the complement of $X$. We provide the
details of other various types of belief mass functions introduced in DSmT in
Appendix B.1 of the supplement document.

####  3.4.2. Causes and Types of Uncertainty

DSmT handles various uncertainties as follows (Smarandache and Dezert, 2009):

  * •

Precise, uncertain beliefs from multiple sources: The beliefs from multiple
sources contribute uncertainty even if the belief of each proposition is a
precise $m(X)$, where each $m(X)$ is only represented by one real number in
$[0,1]$ in $D^{\Theta}$. Uncertainty exists when a single source provides
beliefs about partial elements or multiple sources provide conflicting
beliefs. For example, for $\Theta=\\{\theta_{1},\theta_{2},\theta_{3}\\}$, two
independent sources provide beliefs
$m_{1}(\theta_{1})=0.6,m_{1}(\theta_{3})=0.4$ and
$m_{2}(\theta_{2})=0.8,m_{2}(\theta_{3})=0.2$, respectively.

  * •

Highly conflicting evidence from multiple sources: If $k$ multiple sources
have conflicting evidences toward a same event, there is uncertainty for which
source to trust. For example, for
$\Theta=\\{\theta_{1},\theta_{2},\theta_{3}\\}$, two sources provide
$m_{1}(\theta_{1})=0.2,m_{1}(\theta_{2})=0.1,m_{1}(\theta_{3})=0.7$ and
$m_{2}(\theta_{1})=0.5,m_{2}(\theta_{2})=0.4,m_{2}(\theta_{3})=0.1$. The
decision is based on those conflicting evidence.

  * •

Imprecise beliefs: Imprecise beliefs are represented by the admissible
imprecise. Imprecise beliefs can be either quantitative or qualitative.
Quantitative imprecise beliefs $m^{I}(\cdot)$ are real subunitary intervals of
$[0,1]$ or real subunitary sets over $D^{\Theta}$. Qualitative $m^{I}(\cdot)$
is a set of labels $L=\\{L_{0},L_{1},L_{2},\ldots,L_{m},L_{m+1}\\}$ in order.
Imprecise beliefs are common in fusion problems because it is very hard to
generate precise sources of evidence. For example, the set of ordered
sentiment labels are
$L=\\{L_{0},L_{1},L_{2}\\}=\\{negative,neutral,positive\\}$ and the set of
elements is $\Theta=\\{\theta_{1},\theta_{2}\\}$. The two sources can give
qualitative beliefs by sentiment labels as
$qm_{1}(\theta_{1})=L_{1},qm_{1}(\theta_{2})=L_{0}$ and
$qm_{2}(\theta_{1})=L_{2},qm_{2}(\theta_{2})=L_{1}$, respectively.

  * •

Subjective probability (DSm probability or DSmP) transformation, fusion space,
and fusion rules: The criteria (i.e., frame $\Theta$), the set of elements
(i.e., $G^{\Theta}$), the choice of combination rule, the probability
function, and controllable parameter $\epsilon$ for DSmP, all contribute to
the uncertainty that can significantly impact decision making.

####  3.4.3. Uncertainty Quantification

DSmT does not provide its own uncertainty measure. It borrows other methods
and helps decision making using the following uncertainty measures:

  * •

In probability theory, uncertainty in proposition $A$ can be defined as
(Smarandache, 2012):

(11) $U(A)=\sum_{\begin{subarray}{c}B\in
S^{\Theta}\backslash\\{\emptyset\\},B\cap A\neq\emptyset,B\cap
C(A)\neq\emptyset\end{subarray}}m(B),$

where $A$, $B$, and $C$ are three different elements (i.e., propositions),
$\Theta$ is a set of the elements, and $S^{\Theta}$ is a super power set. The
$C(A)$ is the complement of $A$. Uncertainty and IOU can also be defined in
the same way as DST in Eq. (7).

  * •

Degree of uncertainty can be evaluated in the probability transformation.
Normalized Shannon’s entropy is a measure of uncertainty in probability theory
and given by:

(12)
$E_{H}=-\frac{\sum_{i=1}^{n}m(\theta_{i})\log_{2}(m(\theta_{i}))}{H_{max}},$

where $H_{max}$ is the maximal entropy for the uniform distribution.

  * •

Probabilistic information content (PIC) score refers to the degree of
certainty which can be estimated by $PIC=1-E_{H}$. Less uncertainty (or higher
certainty) can lead to a correct and reliable decision.

For decision making, DSmT extends the probability function, called classical
pignistic transformation (CPT) in DST, into two ways. First, CPT can be
generalized to the Generalized pignistic transformation (GPT). Second, it can
be generalized to a subjective probability measure of $m(\cdot)$ by
$\epsilon\geq 0$ in the new probability transformation, $DSmP_{\epsilon}$,
which is a probability transformation with a subjective measure, $\epsilon$.
Due to the space constraint, we discuss these two transformation methods in
Appendix B of the supplement document.

Figure 7. Uncertainty-aware decision making process using DSmT where the
generalized basic belief assignment ($gbba$) is the formal name of $m(\cdot)$
and BetP refers to pignistic transformation in $gbba$ domain.

Fig. 7 demonstrates the following steps/criteria to make a decision. In Fig.
7, (b) and (c) combines the belief masses from different sources. Uncertainty
is not combined into input evidence and is used to make final decisions in
(c).

  1. (1)

Belief functions and models are defined in the proper frame $\Theta$ of a
given problem. In Appendix B of the supplement document, the closed finite set
(i.e., frame), denoted by $\Theta$, has $n$ elements of hypotheses. These
steps decide the elements in the given problem.

  2. (2)

The belief functions are defined in the proper set of $G^{\Theta}$ (e.g.,
power, hyper, or super set) where $G^{\Theta}$ means any set of items,
including power set $P^{\Theta}$, hyper power set $D^{\Theta}$, and super
power set $S^{\Theta}$. DSmT works on any $G^{\Theta}$, but normally
$D^{\Theta}$ is used to distinguish from $P^{\Theta}$ in DST. This step means
the choice of $P^{\Theta}$, $D^{\Theta}$, or $S^{\Theta}$.

  3. (3)

Choose an efficient rule to combine belief functions (in Appendix B.1 of the
supplement document) for a given problem.

  4. (4)

Before making a decision, one must use a probability function (e.g., GPT, see
Eq. (21) in Appendix B.2) or DSmP with a subjective measure) which is from the
belief functions. The maximum of the GPT function can be used as a decision
criterion between two choices.

Making decisions by DSmP can improve the previous probabilistic
transformations and increase the strength of a critical decision from the
total knowledge.

####  3.4.4. Applications of DSmT on Machine/Deep Learning

DSmT covers broad information fusion topics of data and sensors in robotics,
biometrics, image fusion, trust management, situation analysis, or object
tracking. Various applications such as DSmH (hybrid), DSmP (probabilistic),
and DSmT coordinate between machine processing and user coordination
(Smarandache and Dezert, 2009). DSmT serves as an information fusion tool in
binary class and multi-class classification problems in conjunction with ML/DL
models (Abbas et al., 2015; Ji et al., [n.d.]). As an extension to Support
Vector Machine One-Against-All (SVM OAA) model for multi-class classification,
DSmT models partial ignorance by combining conflicting evidence from two
complementary SVM results through PCR6 rule (see Appendix B.1 of the
supplement document) and reduces focal elements in the model. A DSmT-based
multi-classifier (Ji et al., [n.d.]) integrates PCR6 fusion rules into layered
ML model structures, including Convolutional Neural Network (CNN), Long Short-
Term Memory (LSTM), and Random Forests (RF). DSmT is applied to the final
decision making process by combining multi-signal sources of fault
characteristics.

###  3.5. Imprecise Dirichlet Model (IDM)

Walley (1996) proposed IDM to derive beliefs based on objective statistical
inference from multinomial data with no prior information. The inference is
expressed in terms of posterior upper and lower probabilities. A typical
application is predicting the color of the next marble from a bag whose
contents are initially unknown. Objective Bayesian does not satisfy this
principle because predicted outcome is unknown and we cannot formulate the
sample space. In IDM, the inferences are expressed as the posterior upper and
lower probabilities, $\overline{P}(A|n)$ and $\underline{P}(A|n)$, where $A$
refers to an event and $n$ is the number of observations towards event $A$. In
a multinomial sampling (i.e., $k\geq 2$), the sample space, any event of
interest can be identified as a subset of $\Omega$. IDM generates the lower
and upper bounds for each value in Beta/Dirichlet PDFs (Probability Density
Functions).

####  3.5.1. Belief Formation

Figure 8. Derivation of the upper and lower bounds in IDM.

According to Walley (1996), IDM can be defined as the set of all Dirichlet
$(s,t)$ distribution, such that $0<t_{j}<1$ for $j=1,2,\dots,k$ and
$\sum_{j=1}^{k}t_{j}=1$ and $s$ is a specified positive constant that does not
depend on $\Omega$. Walley suggests $s\leq 2$ where $s$ determines how quickly
the upper and lower probabilities converge as the observation data accumulate.
This is a prior set and denoted as $\mu_{0}$ to model the prior ignorance
about chance $\theta$. Given
$\theta=\\{\theta_{1},\theta_{2},\dots,\theta_{k}\\}$, which refers to the
identical probability distribution of observations. The corresponding set of a
posterior distribution, denoted by $\mu_{N}$, is composed of all Dirichlet
$(N+s,\mathbf{t}^{*})$ distribution (i.e.,
$\mathbf{t}^{*}=\\{t_{1}^{*},t_{2}^{*},\ldots,t_{j}^{*},\ldots,t_{k}^{*}\\}$),
where $t^{*}_{j}=\frac{n_{j}+s\times t_{j}}{N+s}$ and $n_{j}$ is the number of
observations of category $\omega_{j}$ in $N$ trials.

For example, let $A_{j}$ be the event with outcome $\omega_{j}$ from next
trial. The predictive probability $P(A_{j}|n)$ under Dirichlet
$(N+s,\mathbf{t}^{*})$ is equal to the posterior mean of $\theta$. By
maximizing and minimizing $t^{*}_{j}$ with respect to $t_{j}$ (i.e.,
$t_{j}\rightarrow 1$ and $t_{j}\rightarrow 0$), the posterior upper and lower
probabilities of $A_{j}$ are given by:

(13) $\displaystyle\overline{P}(A_{j}|n)=\frac{n_{j}+s}{N+s}\;\;\text{for
$t_{j}\rightarrow
1$};\;\;\;\;\underline{P}(A_{j}|n)=\frac{n_{j}}{N+s}\;\;\text{for
$t_{j}\rightarrow 0$.}$

If $s$ is the hidden observation and $N$ is the number of revealed
observations, those values can be interpreted as the upper and lower bound of
relative frequency of $A_{j}$. For example, before making any observation,
$n_{j}=N=0$, so that $\overline{P}(A_{j}|n)=\frac{s}{s}=1$ and
$\underline{P}(A_{j}|n)=\frac{0}{s}=0$. However, the interval of an IDM bound
may be out of range under insufficient evidence ($s$) condition. For example,
if a bag has nine red balls and one black ball, we randomly pick a ball and
obtain a black ball. Now we have evidence $r(black)=1$, which gives
$\underline{P}(black)=\frac{1}{2+1}=\frac{1}{3}$. However, we know that the
actual probability of having a black ball is $p(black)=\frac{1}{10}$. So
$\underline{P}(black)>p(black)$ when the number of trials is not sufficient.
This case shows that actual probability may be outside the range of IDM under
a lack of evidence (Jøsang, 2016).

####  3.5.2. Causes and Types of Uncertainty

In IDM, uncertainty decreases as a more amount of evidence is received. Hence,
it is aligned with the concept of epistemic uncertainty, which can be reduced
by increasing an amount of observations (or evidence).

####  3.5.3. Uncertainty Quantification

In IDM, the uncertainty is associated with the imprecision whose degree is
captured by the difference between the posterior upper and lower probabilities
as:

(14) $\overline{P}(A_{j}|n)-\underline{P}(A_{j}|n)=\frac{s}{N+s}.$

From the above, we conclude that the imprecision does not depend on event
$A_{j}$. That is, uncertainty due to the imprecision is based on the amount of
hidden observations.

####  3.5.4. Applications of IDM on Machine/Deep Learning

Utkin (2015) proposed an algorithm called IDMBoost (Imprecise Dirichlet Model
Boost), which is an improved version of AdaBoost, one of the well-known
machine learning algorithms, particularly to improve the overfitting problem
with a reduced number of iterations. Serafín et al. (2020) solved imprecise
classification problems by proposing an improved algorithm of Credal Decision
Trees (CDTs). CDTs are Decision Trees using imprecise probabilities using the
Non-Parametric Predictive Inference Model (NPI-M) and showed its
outperformance over IDM. Corani and de Campos (2010) presented a tree-
augmented naïve classifier, called a TANC, based on imprecise probabilities.
TANC considered prior near-ignorance using the extreme IDM (or EDM). They
proved their TANC provides an efficient and sensible approximation of the
global IDM via extensive comparative performance analysis.

###  3.6. Fuzzy Logic

Łukasiewicz and Alfred Tarski (Łukasiewicz and Tarski, 1930) first proposed
Łukasiewicz logic, which is the most typical case of many valued logic. Our
discussion focuses on the real-valued semantics of Łukasiewicz logic as the
backbone of fuzzy logic. Assume $\alpha$ and $\beta$ are two propositional
formulas with truth values $v(\alpha)=x$ and $v(\beta)=y$, we adopt these
semantics in the following context:

(15) $\displaystyle
v(\alpha\vee\beta)=\max\\{x,y\\},\;\;v(\alpha\wedge\beta)=\min\\{x,y\\},\;\;v(\sim\alpha)=1-x,$

where $\vee$ refers to logical disjunction, $\wedge$ is logical conjunction,
and $\sim$ indicates negation.

Fuzzy logic (Zadeh, 1975b) is a kind of infinite-valued logic defined on type
$1$ fuzzy sets (Zadeh, 1965). A fuzzy logic truth value set $\mathscr{T}$ is a
set of linguistic truth-values, which is a language generated from a context-
free grammar $G$:

(16) $\mathscr{T}=L(G).$

For each truth value $\tau\in\mathscr{T}$, $\tau$ is defined as a fuzzy subset
of a truth-value set $l_{\tau}$ of Łukasiewicz logic, which is given by:

(17) $\tau=\int_{0}^{1}\frac{\mu_{\tau}(v)}{v},$

where $\mu_{l_{\tau}}:[0,1]\rightarrow c_{\tau}\in[0,1]$ and
$\mu_{\tau}:[0,1]\rightarrow[0,c_{\tau}]$ are defined as the membership
function of $l_{\tau}$ and $\tau$, respectively. Suppose $\tau$ has a finite
support set $\\{v_{1},v_{2},\ldots,v_{n}\\}\subset[0,1]$, then we can write:

(18)
$\displaystyle\tau=\frac{\mu_{1}}{v_{1}}+\frac{\mu_{2}}{v_{2}}+\dots+\frac{\mu_{n}}{v_{n}},$

where $\mu_{i}=\mu_{\tau}(v_{i})$ for $i\in[1,n]$ and ‘$+$’ stands for an
union operation.

Since truth values are fuzzy subsets of truth-value sets of Łukasiewicz logic,
logic operations between them can be similarly defined by:

(19)
$\displaystyle\mu_{\neg\tau_{0}}=1-\mu_{\tau_{0}},\;\;\mu_{\tau_{0}\vee\tau_{1}}=\max\\{\mu_{\tau_{0}},\;\;\mu_{\tau_{1}}\\},\mu_{\tau_{0}\wedge\tau_{1}}=\min\\{\mu_{\tau_{0}},\mu_{\tau_{1}}\\},$

where $\tau_{0},\tau_{1}\in\mathscr{T}$. From this, we can then derive
$\mu_{\tau_{0}\Rightarrow\tau_{1}}=\mu_{\neg\tau_{0}\vee\tau_{1}}=\max\\{1-\mu_{\tau_{0}},\mu_{\tau_{1}}\\}$
based on Kleene-Dienes implication.

In general, a type $n$ fuzzy set has a membership function defined based on
the set of fuzzy sets of type $n-1$, where $n\geq 2$. Fuzzy numbers (Zadeh,
1975a) can also be formulated as instances of fuzzy sets. In other words, each
fuzzy number is attached to a membership function, that defines a fuzzy set.

A decision making process under fuzzy logic often consists of three phases:
fuzzification, inference, and defuzzification. A fuzzifier transforms crispy
data into fuzzy sets. An inference engine does the logical deduction based on
given fuzzy rules. A defuzzifier transforms the fuzzy relationships to crispy
relationships and makes a final decision.

####  3.6.1. Belief Formation

Zadeh (1968) defined $P(A)$ as the probability of a fuzzy event $A$:

(20) $P(A)=\int_{\mathbb{R}^{n}}\mu_{A}(x)dP=E(\mu_{A}),$

where $A\subseteq\mathbb{R}^{n}$, $\mu_{A}:\mathbb{R}^{n}\rightarrow[0,1]$ is
the membership function of $A$, and $P(A)$ represents the belief of a fuzzy
event $A$.

####  3.6.2. Causes and Types of Uncertainty

Uncertainty in fuzzy logic mostly comes from linguistic imprecision or
vagueness, leading to generating unpredictability, multiple knowledge frames,
and/or incomplete knowledge.

####  3.6.3. Uncertainty Quantification

Zadeh (1968) defined two types of fuzzy sets: Type-1 fuzzy set and Type-2
fuzzy set. In Type-1 fuzzy sets, the uncertainty of fuzzy events introduces
unpredictability and multiple knowledge frames. Zadeh formulated the
uncertainty of a fuzzy event $A$ based on the entropy of event $A$,
$H^{P}(A)$, which is given by:

(21) $H^{P}(A)=-\sum_{i=1}^{n}\mu_{A}(x_{i})P(x_{i})\log P(x_{i}),$

where $A=\\{x_{1},x_{2},\dots,x_{n}\\}$, $\mu_{A}$ is the membership function
of $A$, and $P=\\{P(x_{1}),P(x_{2}),\dots,P(x_{n})\\}$. Here, $P(x_{i})$
refers to the probability of occurring event $x_{i}$.

Fuzzy logic research mainly focused on investigating uncertainty measures on
Type-2 fuzzy sets, which can provide a way of accurately and effectively
measuring fuzziness and uncertainty characteristics of fuzzy complex systems
with two membership functions (Zadeh, 1975a). Wu and Mendel (2007) proposed
five novel uncertainty metrics, called centroid, cardinality, fuzziness
(entropy), variance, and skewness, to measure uncertainty in interval Type-2
fuzzy sets. They further evaluated these metrics with inter-uncertainty and
intra-uncertainty raised in words paradigms (Wu and Mendel, 2009). Zhai and
Mendel (2011) extended the five metrics to general Type-2 fuzzy sets.

####  3.6.4. Applications of Fuzzy Logic on Machine/Deep Learning

Figure 9. Uncertainty-aware decision making process in Fuzzy Logic.

Recently, fuzzy deep neural networks (FDNNs) are considered for a system using
both fuzzy logic and deep neural networks (DNNs) to deal with uncertainty or
ambiguity in data (Das et al., 2020). The methods using FDNNs fall into two
categories: integrated models and ensemble models. The integrated models
integrate fuzzy logic as a part of DL models. In particular, Pythagorean Fuzzy
Deep Boltzmann Machine (PFDBM) (Zheng et al., 2017) was developed based upon
the DBM (Holyoak, 1987). PFDBM used the Pythagorean Fuzzy Set (PFS) (Yager,
2013) to replace standard real-valued parameters. El Hatri and Boumhidi (2018)
developed a DL model in which a network architecture was designed based on
stacked-auto-encoders (SAE) where multiple hyper parameters, such as the
learning rate and the momentum, were determined using fuzzy logic systems. The
ensemble models refer to ensembles of DL and fuzzy logic systems with three
models: models with fuzzy inputs, models with fuzzy outputs, and parallel
models. Wang et al. (2016) proposed a DL model that takes fuzzy feature points
for input for damaged fingerprint classification. Zhang et al. (2014) proposed
a model that uses DL with fuzzy granulation features to predict time-series
data. Chopade and Narvekar (2017) proposed an ensemble of fuzzy logic and DL
to predict fuzzy memberships for document summarization. Deng et al. (2016)
proposed a DL architecture with DL layers and fuzzy membership functions
running in parallel. FDNNs have been applied in various application domains,
such as traffic control (Chen et al., 2018; Hernandez-Potiomkin et al., 2018),
surveillance and security (Chen et al., 2015; Zheng et al., 2016), text
processing (Shirwandkar and Kulkarni, 2018; Nguyen et al., 2018), image
processing (Ahmed et al., 2018), and time-series prediction (Luo et al.,
2019).

###  3.7. Bayesian Inference (BI)

Bayesian theory has been evolved for more than a hundred years (Fienberg,
2006). Bayesian inference (BI) is the process of inductive learning using
Bayer’s rule (Hoff, 2009). Inductive learning is the process of estimating
characteristics of a population from a subset of members of the entire
population (Hoff, 2009). Although some literature treats BI as an ML technique
due to its statistical nature (Tipping, 2003), we treat BI as a belief model
because it deals with a subjective probability representing a belief (Hoff,
2009).

####  3.7.1. Belief Formation

Bayes’ rule offers a rational tool of updating beliefs of unknown information,
which connects probabilities and information (Hoff, 2009). Beliefs are the
statements that can have overlapping domains, such as two beliefs $A$ and $B$
and $A\cap B\neq\emptyset$. A higher value returned from a belief function
indicates the higher degree of a given belief. Bayesian inference estimates
population characteristics $\theta$ from a single dataset sample $y$. A belief
is formed via three steps:

  1. (1)

Prior distribution $p(\theta)$ describes that the belief of $\theta$ being
true population characteristics.

  2. (2)

Sampling model $p(y|\theta)$ shows the belief where $y$ means a sample of the
huge sample space $\mathcal{Y}$ if $\theta$ is true and $y$ needs to be
estimated.

  3. (3)

Posterior distribution $p(\theta|y)$ updates the belief about $\theta$ from
Bayes’ rule based on observed datasets $y$ (Hoff, 2009), for the set of all
possible parameter values in the parameter space, $\Theta$:

(22)
$p(\theta|y)=\frac{p(y|\theta)p(\theta)}{\int_{\Theta}p(y|\tilde{\theta})p(\tilde{\theta})d\tilde{\theta}}.$

Bayesian inference includes conjugate (i.e., prior and posterior distribution
are in the same class) prior distributions, posterior inference, predictive
distributions, and confidence regions. There are variants of quantifying the
uncertainty of variables depending on different sampling methods. Due to the
space constraint, we describe them in Appendix C of the supplement document.
The probability of an event can be obtained via the following steps (Hoff,
2009): (1) determine proper parameter $\theta$ and sample spaces; (2) select
sampling model $p(y|\theta)$ and collect samples; (3) observe prior
distribution $p(\theta)$ by experience or select uninformative prior; (4)
calculate posterior distribution $p(\theta|y)$ based on prior and sampling
methods; (5) perform sensitivity analysis for a range of parameter values; and
(6) finalize general estimation of a population mean. The reliable estimate of
$\theta$ contains a best guess and degree of its confidence. We demonstrate
the diagram of uncertainty and belief process of Bayesian inference in Fig.
10.

Figure 10. Uncertainty-aware decision making process using Bayesian inference
where $p(\theta)$ refers to the probability estimation of $\theta$ that causes
uncertainty. $Bel(\cdot)$ is the belief of evidence in Eq. (5). When no model
is identified, Monte Carlo approximation can be used.

####  3.7.2. Causes and Types of Uncertainty

A belief is formed with the unknown values of random variables. In a
population, the parameter of population characteristics $\theta$ may be
unknown. This means that the conjugate prior belief $p(\theta)$ is unknown.
Before obtaining a dataset $y$, the subset of a population is also unknown. A
sample of dataset $y$ can help to reduce the uncertainty about the population
characteristics. This type of uncertainty is caused by a lack of evidence.

####  3.7.3. Uncertainty Quantification

In single-parameter sampling models, such as Binomial, Poisson, and Monte
Carlo approximation, the posterior inference variance of the estimated mean
$\theta$ is a measure of uncertainty from the current belief formation. That
is, the uncertainty is measured by a variance in Binomial model, Poisson
model, and Monte Carlo sampling by:

(23)
$\displaystyle\mathrm{Var}^{Bin}[\theta|y]=\frac{\mathrm{E}[\theta|y]\mathrm{E}[1-\theta|y]}{a+b+n+1},\;\mathrm{Var}^{Poiss}[\theta|y]=\frac{a+y}{b+n},\;\mathrm{Var}^{MC}[\theta|y]=\sum^{S}_{s=1}\frac{\theta^{(s)}-\overline{\theta})^{2}}{(S-1)},$

where $n$ is the number of choices of $y$, $a$ and $b$ are the parameters in
$\mathrm{Beta}(a,b)$ distribution for a Binomial model, $a$ and $b$ are the
parameters of $\Gamma(a,b)$ distribution for a Poisson model, and $\theta$ is
the estimation of parameters and $\overline{\theta}$ is the mean of $\theta$
for Monte Carlo sampling.

In the normal model with mean $\theta$ and variance $\sigma^{2}$, a joint
distribution can be transformed to a conditional probability by Eq. (30) in
the supplement document. The distribution
$p(\theta|\sigma^{2},y_{1},\ldots,y_{n})$ is defined by Eq. (29) in the
supplement document with variance
$\tau_{n}^{2}=1/(\frac{1}{\tau_{0}^{2}}+\frac{n}{\sigma^{2}})$. The posterior
inverse variance
$\frac{1}{\tau_{n}^{2}}=\frac{1}{\tau_{0}^{2}}+\frac{n}{\sigma^{2}}$ indicates
that the posterior inverse variance (a.k.a. precision) $1/\tau_{n}^{2}$
combines sampling precision $1/\sigma^{2}$ and prior precision
$1/\tau_{0}^{2}$.

####  3.7.4. Applications of Bayesian Inference on Machine/Deep Learning

Tipping (2003) introduced how Bayesian inference (BI) is used in ML. BI solves
a non-deterministic relationship between dependent ($Y$) and independent ($X$)
variables. Given $N$ data examples and many parameters w, the model of
probability of $Y$ given $X$ is computed by $P(Y|X)=f(X;\textbf{w})$ (Tipping,
2003). The distribution over parameters w can be inferred from Bayes’ rule.
Approximation techniques are the key points, such as least-square, maximum
likelihood, and regularization. The common choice of a prior is a zero-mean
Gaussian prior. The Bayesian way of estimating Maximum A Posteriori (MAP) is
for posterior inference. Marginalization serves an important role in the
Bayesian framework (Tipping, 2003). Sofman et al. (2006) used improved robot
navigation in a linear Gaussian model to estimate the posterior distribution
of the general Bayesian features and the locale-specific features. Tripathi
and Govindaraju (2007) used relevance vector machines to predict uncertainty
in hydrology. G. Tian and Feng (2011) analyzed brain image segmentation by
applying Gaussian mixture model (GMM) with a genetic algorithm (GA) and
variational expectation-maximization algorithm.

In the ML/DL, the amount of parameters are expanded in Bayesian neural network
(BNN) (Wang and Yeung, 2020). BNN is naturally to capture the uncertainty for
prediction by putting a prior distribution over its weights, such as Gaussian
prior distribution: $\bm{\theta}\sim\mathcal{N}(0,I)$, where $\bm{\theta}$ is
the model weights (parameters). Specifically, given a dataset
$D=\\{X=\\{x_{1},\ldots,x_{N}\\},Y=\\{y_{1},\ldots,y_{N}\\}\\}$, instead of
optimizing the deterministic model weights via maximum likelihood estimation
(MLE), BNN refers to extending standard networks with posterior inference,
which learns a posterior over model weights $p(\bm{\theta}|D)$ such that model
output $f(x,\bm{\theta})$ is stochastic.

###  3.8. Subjective Logic (SL)

As a variant of DST, Jøsang (2016) proposed a belief model, called _Subjective
Logic_ (SL) that describes subjectivity of an opinion in terms of multiple
belief masses and uncertainty.

####  3.8.1. Belief Formation

Since a binomial opinion is a special case of multinomial opinions where the
number of belief masses is two, for brevity, we only provide the descriptions
of multinomial opinions and hyper-opinions.

Multinomial Opinions: In SL, a multinomial opinion in a given proposition $x$
is represented by $\omega_{X}=(\bm{b}_{X},u_{X},\bm{a}_{X})$ where a domain is
$\mathbb{X}$, a random variable $X\in\mathbb{X}$, $\kappa=|\mathbb{X}|>2$ (for
a binomial opinion, $\kappa=|\mathbb{X}|=2$), and the additivity requirement
of $\omega_{x}$ is given as $\sum_{x\in\mathbb{X}}\bm{b}_{X}(x)+u_{X}=1$ where
each parameter refers to: (1) $\bm{b}_{X}$: belief mass distribution over
$\mathbb{X}$; (2) $u_{X}$: uncertainty mass representing vacuity of evidence;
and (3) $\bm{a}_{X}$: base rate distribution over $\mathbb{X}$.

The projected probability distribution of multinomial opinions is given by:

(24) $\mathbf{P}_{X}(x)=\bm{b}_{X}(x)+\bm{a}_{X}(x)u_{X},\;\;\;\forall
x\in\mathbb{X}.$

The probability distribution of a multinomial opinion follows Dirichlet
distribution (Jøsang, 2016).

Hyper-opinions: Hyper-opinions represent multiple choices under a specific
singleton belief $x$ where belief mass is allowed to be assigned to a
composite value $x\in\mathscr{C}(\mathbb{X})$ consisting of a set of singleton
values $x$’s. Belief masses assigned to composite values
$x\in\mathscr{C}(\mathbb{X})$ can be used to estimate the vagueness of an
opinion. Hyperdomain, denoted by $\mathscr{R}(\mathbb{X})$, is the reduced
powerset of $\mathbb{X}$ which is the set of $\mathscr{P}(\mathbb{X})$ that
excludes $\\{\mathbb{X}\\}$ and $\\{\emptyset\\}$. Hyperdomain can be defined
by:

(25)
$\text{Hyperdomain}:\mathscr{R}(\mathbb{X})=\mathscr{P}(\mathbb{X})\backslash\\{\\{\mathbb{X}\\},\\{\emptyset\\}\\}.\vspace{-1mm}$

Given $X$ as a hyper variable in $\mathscr{R}(\mathbb{X})$, a hyper-opinion on
$X$ is represented by $\omega_{X}=(\bm{b}_{X},u_{X},\bm{a}_{X})$ where each
opinion dimension includes: (1) $\bm{b}_{X}$: belief mass distribution over
$\mathscr{R}(\mathbb{X})$; (2) $u_{X}$: uncertainty mass representing vacuity
of evidence; and (3) $\bm{a}_{X}$: base rate distribution over $\mathbb{X}$,
where $\sum_{x\in\mathscr{R}(\mathbb{X})}\bm{b}_{X}(x)+u_{X}=1$.

The projected probability distribution of a hyper-opinion can be given by:

(26)
$\small\mathbf{P}_{X}(x)=\sum_{x_{i}\in\mathscr{R}(\mathbb{X})}\bm{a}_{X}(x|x_{i})\bm{b}_{X}(x_{i})+\bm{a}_{X}(x)u_{X},\;\;\bm{a}_{X}(x|x_{i})=\frac{\bm{a}_{X}(x\cap
x_{i})}{\bm{a}_{X}(x_{i})},\forall x,x_{i}\in\mathscr{R}(\mathbb{X}),$

where $\bm{a}_{X}(x|x_{j})$ is the relative base rate and
$\bm{a}_{X}(x_{i})\neq 0$. For the binomial or multinomial opinions, the
additivity requirement is met (i.e.,
$\sum_{x\in\mathbb{X}}\mathbf{P}_{X}(x)=1$). However, for the hyper-opinion,
the additivity requirement may not be met, but $\mathbf{P}_{X}(x)$ follows
super-additivity (i.e.,
$\sum_{x\in\mathscr{R}(\mathbb{X})}\mathbf{P}_{X}(x)\geq 1$) with a
hyperdomain, $\mathscr{R}(\mathbb{X})$.

Hyper-opinions can be represented by Dirichlet PDFs and the hyper-Dirichlet
distribution (Hankin, 2010). To do so, we can project a hyper-opinion into a
multinomial opinion based on (Jøsang, 2016). The approximation by the
projection of hyper-opinions to multinomial opinions removes vague information
in the representation of opinions. This allows a decision maker to see a
particular opinion without the veil of vagueness, which facilitates a more
direct and intuitive interpretation of the opinion.

####  3.8.2. Causes and Types of Uncertainty

In SL, three types of uncertainties are discussed as follows (Jøsang et al.,
2018): vacuity, vagueness, and dissonance. Vacuity uncertainty is caused by a
lack of evidence or knowledge. Vagueness uncertainty is caused by vague
observations, leading to failing in identifying a distinctive singleton
belief. Dissonance uncertainty is introduced due to conflicting evidence,
resulting in inconclusiveness. Vacuity and dissonance can be understood as
epistemic uncertainty, which can be reduced with more evidence. Vagueness is
related to fuzziness, which triggers aleatoric uncertainty in its nature.

####  3.8.3. Uncertainty Quantification

Uncertainty measures across all belief masses are calculated based on the sum
of uncertainty masses associated with individual belief masses, as discussed
above. They include total vacuity (same as $u_{X}$), total vagueness (
$b^{\mathrm{TV}}_{X}$), and total dissonance ($\dot{b}_{X}^{\mathrm{Diss}}$):

(27) $\displaystyle
u_{X}=\\!\\!\\!\\!\sum\limits_{x\in\mathscr{R}(\mathbb{X})}\mathbf{u}^{F}_{X}(x),\;\;b^{\mathrm{TV}}_{X}=\\!\\!\\!\\!\sum\limits_{x\in\mathscr{R}(\mathbb{X})}\\!\\!\\!\\!\bm{b}_{X}(x),\;\;\dot{b}_{X}^{\mathrm{Diss}}=\sum\limits_{x_{i}\in\mathbb{X}}\bm{b}^{\mathrm{Diss}}_{X}(x_{i}),$

where $\mathbf{u}^{F}_{X}(x)$ refers to a focal uncertainty (vacuity per
belief), $\bm{b}_{X}(x)$ is a belief mass supporting $x$, and
$\bm{b}^{\mathrm{Diss}}_{X}(x_{i})$ indicates dissonance per singleton belief.
All these belief masses are detailed in Appendix D.2 of the supplement
document.

The uncertainty associated with each belief is elaborated in Appendix D of the
supplement document. In addition, Jøsang (2016) proposed a technique called
uncertainty maximization to offset the amount of evidence received in the past
to consider additional new evidence because additional new evidence does not
change the current belief states significantly if uncertainty is very low. We
provided the uncertainty maximization formulation in Appendix D of the
supplement document.

When one makes a decision under uncertainty using SL, we can leverage SL’s
capability to estimate multidimensional uncertainty (i.e., vagueness, vacuity,
and dissonance) to make effective decisions. As in Fig. 11, after estimating
multiple dimensions of uncertainty, one can use the vacuity maximization
technique if more evidence is needed in order to allow considering more
evidence even under low vacuity, representing high certainty due to a large
volume of evidence collected. Recall that SL-based opinion cannot be updated
or is rarely updated significantly if its vacuity is or close to zero. One can
also consider other opinions by using a variety of fusion operators in SL
(Jøsang, 2016), which can generate a single opinion with the updates of
corresponding belief masses and vacuity values. The generated single opinion
can be also assessed based on which decision has the most utility by
normalizing the opinion based on each decision (i.e., belief mass)’s utility.
Most decision making problems can be solved by these processes and allows us
to make a decision with minimum uncertainty and maximum utility. However, if
all decisions have the same uncertainty-aware maximum utility, one can select
a decision at random, which we want to avoid.

Figure 11. Uncertainty-aware decision making process using Subjective Logic.

####  3.8.4. Applications of SL on Machine/Deep Learning

Recently SL has been considered along with machine/deep learning. Uncertainty
reasoning to solve classification tasks has been studied by leveraging SL to
consider vacuity and dissonance uncertainty dimensions (Sensoy et al., 2018;
Zhao et al., 2019b). In addition, SL-based opinion formulation is used to
infer subjective opinions along with DL in the presence of adversarial attacks
(Alim et al., 2019; Zhao et al., 2018a, b). Further, SL-based opinions are
considered along with deep reinforcement learning to propose uncertainty-aware
decision making (Zhao et al., 2019a).

##  4\. Belief Theory Meets Deep Learning

In this section, we review several hybrid frameworks that combine belief
models and neural networks, including evidential (or subjective) neural
networks, fuzzy neural networks, and rough deep neural networks.

###  4.1. Evidential Neural Networks (ENNs)

Evidential neural networks (ENNs) (Sensoy et al., 2018) is a hybrid framework
of subjective belief models and neural networks. They are similar to classic
neural networks for classification. The main difference is that the softmax
layer is replaced with an activation function in ENNs, e.g., ReLU, to ensure
non-negative output (i.e., range of $[0,+\infty]$), which is taken as the
evidence vector for the predicted Dirichlet distribution, or equivalently,
multinomial opinion.

####  4.1.1. Key Formulation of ENNs

Given the feature vector ${\bf x}$ of an input sample, let $f({\bf
x}|{\bm{\theta}})$ represent the evidence vector by the network for the
classification, where ${\bm{\theta}}$ is network parameters. Then the
corresponding Dirichlet distribution has parameters ${\bm{\alpha}}=f({\bf
x}_{i}|{\bm{\theta}})+1$, where the $k$-th parameter ${\alpha}_{k}$ denotes
the effective number of observations of the $k$-th class, and the total number
of classes is $K$. Let $\textbf{p}=(p_{1},\dots,p_{K})^{T}$ be the
probabilities of the $K$ predefined classes. The Dirichlet PDF (i.e.,
$\text{Dir}({\bf p};{\bm{\alpha}})$) with ${\bf p}$ as a random vector is
defined by:

(28)
$\displaystyle\mathrm{Dir}(\bm{p}|{\bm{\alpha}})=\frac{1}{B({\bm{\alpha}})}\prod\nolimits_{k\in\mathbb{Y}}p_{k}^{(\alpha_{k}-1)},$

where
$\frac{1}{B({\bm{\alpha}})}=\frac{\Gamma(\sum_{k\in\mathbb{Y}}\alpha_{k})}{\prod_{k\in\mathbb{Y}}(\alpha_{k})}$,
$\alpha_{k}\geq 0$, and $p_{k}\neq 0$, if $\alpha_{k}<1$. The expected value
of class probabilities $\textbf{p}=(p_{1},\ldots,p_{K})^{T}$ is given by:

(29)
$\displaystyle\mathbb{E}[p_{k}]=\frac{\alpha_{k}}{\sum_{j=1}^{K}\alpha_{j}}=\frac{e_{k}+a_{k}W}{\sum_{j=1}^{K}e_{j}+W}.$

The observed evidence in a Dirichlet PDF $\mathrm{Dir}(\bm{p}|{\bm{\alpha}})$
can be mapped to a multinomial opinion $(b_{1},\cdots,b_{K},u)$ as follows:

(30) $\displaystyle b_{k}=\frac{e_{k}}{S},\;\;u=\frac{W}{S},\text{ for
}k=1,\cdots,K,$

where $S=\sum_{k=1}^{K}\alpha_{k}$ refers to the Dirichlet strength. Without
loss of generality, we set $a_{k}=1/K$ and the non-informative prior weight
(i.e., $W=K$), which indicates that $a_{k}\cdot W=1$ for each
$k\in\\{1,\cdots,K\\}$. Therefore, the output of an ENN can be applied to
measure the subjective uncertainty about the predictive class variable $y$ in
different types, such as vacuity and dissonance as defined based on a
multinomial opinion (See Section 3.8).

The Bayesian framework of ENNs was proposed in (Malinin and Gales, 2018)that
considers a prior distribution on the network parameters ${\bm{\theta}}$,
denoted by $P({\bm{\theta}})$. Let $P({\bm{\theta}}|\mathcal{D})$ be the
posterier PDF, where $\mathcal{D}$ refers to the training set. Let
$\text{Cat}(y|{\bf p})$ be the PDF of the categorical distribution about the
predictive variable $y$, where the class probabilities ${\bf p}$ are the
parameters. We can then show terms associated with different uncertainty as
follows:

(31) $\displaystyle P(y|{\bf
x},\mathcal{D})=\int\int\underbrace{\text{Cat}(y|{\bf
p})}_{Data\;\;uncertainty}\;\;\underbrace{P({\bf p}|{\bf
x},{\bm{\theta}})}_{Subjective\;\;uncertainty}\;\;\underbrace{P({\bm{\theta}}|\mathcal{D})}_{Model\;\;uncertainty}d{\bm{p}}d{\bm{\theta}},$

where $P({\bf p}|{\bf x},{\bm{\theta}})=\text{Dir}({\bf p}|{\bm{\alpha}})$ and
${\bm{\alpha}}=f({\bf x},{\bm{\theta}})$. In this expression, data
(aleatoric), subjective (distributional), and model (epistemic) uncertainty
are modeled by a separate term within an interpretable probabilistic
framework. The data uncertainty is described by the point-estimate categorical
distribution, $\text{Cat}(y|{\bf p})$. The subjective (or distributional)
uncertainty is described by the distribution over predictive class variables
$P({\bf p}|{\bf x},{\bm{\theta}})$. The model uncertainty is described by the
posterior distribution over the parameters given the data. The relationship
between uncertainties is made explicit - model uncertainty affects estimates
of subjective uncertainty, which in turn affects the estimates of data
uncertainty. This forms a hierarchical model with three layers of uncertainty:
the posterior over classes, the per-data Dirichlet prior distribution, and the
global posterior distribution over model parameters. The uncertainty due to
the mismatch between testing and training distributions can be measured by two
methods. First, as the Dirichlet distribution $P({\bf p}|{\bf
x},{\bm{\theta}})$ is equivalent to a subjective multinomial opinion based on
the mapping defined in Eq. (30), we can quantify subjective uncertainty types
directly based on the Dirichlet distribution, such as vacuity and dissonance,
where vacuity captures elements of distributional uncertainty. Second, the
distributional uncertainty can be measured based on mutual information between
the categorical label $y$ and the class probabilities ${\bf p}$ as:

(32) $\displaystyle\underbrace{\mathcal{I}[y,{\bf p}|{\bf
x},\mathcal{D})]}_{Epistemic\;\;uncertainty}=\underbrace{\mathcal{H}[\mathbb{E}_{P({\bf
p}|{\bf x};\mathcal{D})}[\text{Cat}(y|{\bf
p})]]}_{Entropy}-\underbrace{\mathbb{E}_{P({\bf p}|{\bf
x};\mathcal{D})}[\mathcal{H}[\text{Cat}(y|{\bf
p})]]}_{Aleatoric\;\;uncertainty}$

We note that distributional uncertainty and vacuity are negatively correlated,
if the parameters ${\bm{\theta}}$ are deterministic. The former is maximized
(and the latter is minimized) when all categorical distributions are
equiprobable, which occurs when the Dirichlet distribution is flat.

####  4.1.2. Causes and Types of Uncertainty

Since ENNs provide a hybrid framework of subjective belief models and neural
networks, we can estimate evidential uncertainty, such as vacuity (scenario
uncertainty) and dissonance (discord uncertainty), based on a subjective
opinion. Recall that vacuity is due to a lack of evidence introducing
uncertainty by incomplete knowledge. Dissonance is due to conflicting
evidence, resulting in multiple knowledge frames.

####  4.1.3. Uncertainty Quantification

ENNs estimate Dirichlet distribution parameters directly, which can be
transferred to a subjective opinion. After then, we can estimate vacuity ($u$)
and dissonance ($diss$) based on SL-based subjective opinion where there are
$K$ classes and $e_{k}$ number of evidence to support each class $k$ by:

(33) $\displaystyle\small
u=\frac{K}{\sum_{k=1}^{K}e_{k}+K},\;diss=\sum_{k=1}^{K}\Bigg{(}\frac{b_{k}\sum_{j\neq
k}b_{j}\text{Bal}(b_{j},b_{k})}{\sum_{j\neq k}b_{j}}\Bigg{)},$ (34)
$\displaystyle\small{\text{Bal}(b_{j},b_{k})=\begin{cases}1-\frac{|b_{j}-b_{k}|}{b_{j}+b_{k}}&\text{if
$b_{i}b_{j}\neq 0$}\\\ 0&\text{if $\min(b_{i},b_{j})=0$}\end{cases}}$

where $b_{k}$ and $b_{j}$ refer to the belief masses supporting $k$ class and
$j$ class, respectively.

####  4.1.4. Applications of ENNs

There is a whole class of evidential neural networks that have the
interpretation that evidence represents the number of nearby training samples
of various classes relative to the sample under test. This includes the
generative version from (Sensoy et al., 2020), posterior networks based on
density-based pseudo-counts in (Charpentier et al., 2020), and Epistemic
Neural networks (Osband et al., 2021) that allow a general interface to
distinguish epistemic from aleatoric uncertainty. The ENNs have been applied
on several applications in different domains, such as justified true belief
models (Virani et al., 2020; Bhushan et al., 2020), active learning on image
data (Shi et al., 2020), misclassification and out-of-distribution detection
on graph data (Zhao et al., 2020; Hu et al., 2020), event early detection on
time series data (Zhao et al., 2022), and self-training on NLP task (Xu et
al., 2021).

###  4.2. Fuzzy Deep Neural Networks (FDNNs)

Fuzzy deep neural networks (FDNNs) is a hybrid framework of fuzzy logic
systems and deep neural networks (Das et al., 2020). FDNNs is designed to
address the drawback that deep neural networks are sensitive to the
uncertainties and the ambiguities of real-world data. Multiple approaches are
developed to implement an FDNN. Some models, such as Fuzzy Restricted
Boltzmann Machines (FRBMs) (Chen et al., 2015), consider the concept of fuzzy
numbers to represent network weights. Some models use fuzzy logic units to
replace perceptrons in the network (Park et al., 2016). Fuzzy systems are also
used to train the network parameters of a deep neural network (El Hatri and
Boumhidi, 2018). In this section, we use Pythagorean Fuzzy Deep Boltzmann
Machines (PFDBMs) (Zheng et al., 2016), a recent extension of FRBMs, to
demonstrate how fuzzy logic can be integrated as a part of deep neural
networks, such as deep Boltzmann machines.

####  4.2.1. Key Formulation of PFRBMs

We will first introduce the building blocks, including deep Boltzmann machines
and pythagorean fuzzy set, and then introduce the architecture design of
PFRBMs. A deep Boltzmann machine (DBM) is an extension of the restricted
Boltzmann machine (Zhang et al., 2018) and considers multiple hidden layers to
capture more complex correlations of the activities of the preceding layers
(Salakhutdinov and Larochelle, 2010). Considering a $L$ hidden layers DBM
whose set of layers is $\\{\textbf{x},\textbf{h}_{1},\ldots,\textbf{h}_{L}\\}$
where $x$ is a set of visible units $\textbf{x}\in\\{0,1\\}^{D}$, and
$\textbf{h}_{l}$ is $l$-th hidden layerwith a set of hidden units
$\textbf{h}_{l}\in\\{0,1\\}^{P_{l}}$. DBM is an energy-based probabilistic
model which defines a joint probability distribution over $x$ as

(35)
$P(\textbf{x};\theta)=\frac{1}{Z(\theta)}\sum_{\textbf{h}_{1}}\cdots\sum_{\textbf{h}_{L}}e^{-E\left(\textbf{x},\textbf{h}_{1},\ldots,\textbf{h}_{L},\theta\right)}$

where $\theta=[W_{1},\ldots,W_{L}]$ is a vector of the parameters, and
$E(\textbf{x},\textbf{h}_{1},\ldots,\textbf{h}_{L};\theta)$ is the energy
function (Smolensky, 1986) n defined as

(36)
$E(\textbf{x},\textbf{h}_{1},\ldots,\textbf{h}_{L};\theta)=-\textbf{x}^{T}W_{1}\textbf{h}_{1}-\sum_{l=2}^{L}\textbf{h}_{l-1}^{T}W_{l}\textbf{h}_{l},$

and $Z(\theta)$ is the partition function defined as

(37)
$Z(\theta)=\sum_{\textbf{x}}\sum_{\textbf{h}_{1}}\cdots\sum_{\textbf{h}_{L}}e^{-E\left(\textbf{x},\textbf{h}_{1},\ldots,\textbf{h}_{L},\theta\right)}$

DBM aims to maximize the joint probability $P(\textbf{x};\theta)$, which has
the same effect to minimize energy function $E(\cdot)$.

Pythagorean fuzzy sets (PFS) is an extension of the basic fuzzy sets in two
perspectives. First, it introduces a non-membership degree besides the
standard membership degree []. Second, it considers the restriction that the
sum of the squares of the membership degree is between 0 and 1. PFS is defined
by the mathematical object:

(38) $\mathcal{P}=\left\\{\langle x,\mu_{p}(x),v_{p}(x)\rangle\mid x\in
S\right\\},$

where $\mu_{p}(x):S\rightarrow[0,1]$ is the membership degree (how much degree
of $x\in S$) of element $x$ to $S$ in $P$, and $\nu_{p}(x):S\rightarrow[0,1]$
is the non-membership degree (how much degree of $x\notin S$) as well. In
addition, we have $\mu^{2}_{p}(x)+\nu^{2}_{p}(x)\leq 1$, and the hesitant
degree, neither membership nor non-membership degree, may consider as
uncertainty degree. The hesitation degree (uncertainty degree) is the function
that expresses lack of knowledge of whether $x\in S$ or $x\notin S$. It can be
calculated by:

(39) $\pi_{p}(x)=\sqrt{1-\mu_{p}^{2}(x)-v_{p}^{2}(x)}.$

Moreover, to simplify it, $\mathcal{P}(\mu_{p}(x),\nu_{p}(x))$ is called a
Pythagorean fuzzy number (PFN) denoted
$\beta=\mathcal{P}(\mu_{\beta},\nu_{\beta})$, where
$\mu_{\beta},\nu_{\beta}\in[0,1]$ and $\mu_{\beta}^{2}+\nu_{\beta}^{2}\leq 1$.
We can use two metrics to rank a PFN by:

(40) $\displaystyle h(\beta)=\mu_{\beta}^{2}+v_{\beta}^{2},$ $\displaystyle
s(\beta)=\mu_{\beta}^{2}-v_{\beta}^{2},$

where $h(\beta)$ is the accuracy function of $\beta$ and $s(\beta)$ is the
score function of $\beta$. The ranking of two PFNs,
$\beta_{1}=P(\mu_{\beta_{1}},\nu_{\beta_{1}})$ and
$\beta_{2}=P(\mu_{\beta_{2}},\nu_{\beta_{2}})$, is performed by: (1) If
$s\left(\beta_{1}\right)<s\left(\beta_{2}\right)$, then $\beta_{1}<\beta_{2}$;
and (2) If $s\left(\beta_{1}\right)=s\left(\beta_{2}\right)$, then (a) if
$h\left(\beta_{1}\right)<h\left(\beta_{2}\right)$, then $\beta_{1}<\beta_{2}$;
and (b) if $h\left(\beta_{1}\right)=h\left(\beta_{2}\right)$, then
$\beta_{1}=\beta_{2}$.

The Pythagorean Fuzzy Restricted Boltzmann Machine (PFRBM) extends the DBM
model by replacing the standard real-valued parameters with PFNs. PFRBM is
able to handle fuzzy and/or incomplete data and the fuzzy parameters provide a
better representation of the data using fuzzy probability. Fig. 12 describes
the framework of a PFRBM with $L$ layers, denoted as $h_{1},\ldots,h_{L}$. Let
$\widetilde{\theta}=[\widetilde{W_{1}},\ldots,\widetilde{W_{L}}]$ be the fuzzy
parameters and $\textbf{x}=(x_{1},\cdots,x_{D})$ be the input feature vector.
The energy function and probability function of a PFRBM are shown as follows:

(41)
$\displaystyle\widetilde{E}\left(\textbf{x},\textbf{h}_{1},\ldots,\textbf{h}_{L};\widetilde{\theta}\right)=-\textbf{x}^{T}\widetilde{W_{1}}\textbf{h}_{1}-\sum_{l=2}^{L}\textbf{h}_{l-1}^{T}\widetilde{W_{l}}\textbf{h}_{l},$
(42)
$\displaystyle\widetilde{P}(\textbf{x};\bar{\theta})=\frac{1}{\widetilde{Z}(\widetilde{\theta})}\sum_{\textbf{h}_{1}}\cdots\sum_{\textbf{h}_{L}}e^{-\widetilde{E}\left(\textbf{x},\textbf{h}_{1},\ldots,\textbf{h}_{L},\widetilde{\theta}\right)}.$

Therefore, we consider the log likelihood as the objective function,

(43)
$\max_{\tilde{\theta}}\widetilde{\mathcal{L}}(\widetilde{\theta},D)=\sum_{\textbf{x}\in
D}\log(\widetilde{P}(\textbf{x},\widetilde{\theta})).$

As fuzzy optimization problems are intractable, the PFDBM is trained using a
combination of gradient descent and metaheuristic techniques.

Figure 12. Pythagorean Fuzzy Deep Belief Network (PFDBN).

####  4.2.2. Causes and Types of Uncertainty

PFDBMs provides a hybrid framework of fuzzy sets and DNNs where uncertainty
comes from a fuzzy set due to fuzzy and/or incomplete data, leading to
unpredictability. The fuzziness has its root nature in aleatoric uncertainty.

####  4.2.3. Uncertainty Quantification

Unlike traditional DNNs, PFDBMs with fuzzy parameters provide a better
representation of data using a fuzzy probability to represent uncertainty. The
fuzzy parameters can learn new features and allow investigating how much a
certain (or uncertain) feature influences the output.

####  4.2.4. Applications of FDNNs

PFDBMs was proposed to develop an airline passenger profiling (Zheng et al.,
2016) and provide an early warning system for industrial accidents (Zheng et
al., 2017). Besides PFDBMs, Park et al. (2016) developed intra- and inter-
fraction FDNNs to tracking lung-cancer tumor motion. Similar to (El Hatri and
Boumhidi, 2018), fuzzy logic was employed to train the learning parameters in
FDNNs for traffic incident detection. In addition, some models consider fuzzy
logic and deep learning in a sequential or parallel fashion. Wang et al.
(2016) proposed a model that uses deep neural network with fuzzy feature
points for damaged fingerprint classification. Zhang et al. (2014) proposed a
model utilizing fuzzy granulation and deep belief network for predicting time-
series data.

###  4.3. Rough Deep Neural Networks (RDNNs)

Rough neural networks (RNNs) have been studied for a decade by combining a
rough set or rough neuron with DNNs to process the uncertainties and high-
dimensional data (Lingras, 1996). The methods fall into two main categories:
rough neural-based and rough set-based. Due to the space constraint, we
discuss the rough neural-based method while providing the details of the rough
set-based approaches in Appendix E.2 of the supplement document.

Figure 13. Rough neuron with six tunable parameters

####  4.3.1. Key Formulation of RDNNs

Rough neural-based method considers a rough neuron in DNNs to improve the
robustness of learning. For a traditional neural network, if the input feature
is represented by a range, such as the temperature of climate (e.g., daily
maximum and minimum temperature), the neural network cannot learn a good
representation and the prediction error will be relatively large. The neural
network based on a rough neuron can address this issue.

Fig. 13 shows how the rough neuron is applied for rough pattern recognition.
This neuron consists of an upper bound neuron with parameters
$\theta_{U}=\\{W_{U},b_{U},\alpha\\}$, and a lower bound neuron with
parameters $\theta_{L}=\\{W_{L},b_{L},\beta\\}$. Here $W_{U}$ and $b_{U}$ are
the weight and bias of the upper bound, respectively, while $W_{L}$ and
$b_{L}$ are those for the lower bound neuron, respectively. Output
coefficients, $0\leq\alpha$ and $\beta\leq 1$, determine the contribution of
upper bound output $O_{U}$ and lower bound output $O_{L}$ to the overall
neuron’s output $O$. A rough extension of auto-encode, called rough auto-
encoder (RAE), uses rough neurons in its hidden layer and output layer. Here
$W^{k}_{U}$, $b^{k}_{U}$, and $\alpha^{k}$ are the upper bound parameters of
layer $k$ and $W^{k}_{L},b^{k}_{L}$, and $\beta^{k}$ are the lower bound
parameters of layer $k$, respectively.

Given the RAE defined with input vector $h_{0}=X$, the upper bound and lower
bound outputs of the first hidden layer are shown where $W_{U}^{1}$ and
$W_{L}^{1}$ are the learned parameters and $f^{1}W_{L}^{1}X+b_{L}^{1}$ can be
larger than $f^{1}W_{U}^{1}X+b_{U}^{1}$. The $h_{U}^{1}(X)$ and $h_{L}^{1}(X)$
are defined by:

(44) $\displaystyle
h_{U}^{1}(X)=\max\Big{[}f^{1}\left(W_{U}^{1}X+b_{U}^{1}\right),f^{1}\left(W_{L}^{1}X+b_{L}^{1}\right)\Big{]},$
$\displaystyle
h_{L}^{1}(X)=\min\Big{[}f^{1}\left(W_{U}^{1}X+b_{U}^{1}\right),f^{1}\left(W_{L}^{1}X+b_{L}^{1}\right)\Big{]},$

where $f^{1}$ is a sigmoid function. The latent representation in the hidden
layer is computed by:

(45) $h^{1}=\alpha^{1}h^{1}_{U}+\beta^{1}h^{1}_{L}.$

For the rough decoding process in the output layer, the upper bound and lower
bound outputs are computed as:

(46) $\displaystyle h_{U}^{2}$
$\displaystyle=\max\Big{[}f^{2}\left(W_{U}^{2}h^{1}+b_{U}^{2}\right),f^{2}\left(W_{L}^{2}h^{1}+b_{L}^{2}\right)\Big{]},$
$\displaystyle h_{L}^{2}$
$\displaystyle=\min\Big{[}f^{2}\left(W_{U}^{2}h^{1}+b_{U}^{2}\right),f^{2}\left(W_{L}^{2}h^{1}+b_{L}^{2}\right)\Big{]},$

where $f^{2}$ is considered to be a linear function. Therefore, we have the
reconstructed input,

(47) $r=\alpha^{2}h^{2}_{U}+\beta^{2}h^{2}_{L}.$

####  4.3.2. Causes and Types of Uncertainty

In RDNNs, uncertainty is considered in a rough set introducing
unpredictability and rough neuron introducing incomplete knowledge. Hence, the
rough set and neuron can capture vagueness from model input and parameter
uncertainty from model parameters.

####  4.3.3. Uncertainty Quantification

The uncertainty in RDNNs can be estimated based on the rough set theorem
introduced in Section 3.1. The rough set theorem approximates an $M$-boundary
region, which contains a set of objects that cannot be clearly classified by
only employing the set of attributes and represents vagueness.

####  4.3.4. Applications of RDNNs

Most RDNNs are proposed to reduce uncertainty. Zhang and Wang (2006) applied
the fuzzy-rough neural network in vowels recognition. Khodayar et al. (2017)
proposed a rough extension of stacked denoising autoencoder (SDAE) for
ultrashort-term and short-term wind speed forecasting, incorporating a rough
neural network into wind uncertainties. Sinusoidal Rough-Neural Network (SR-
NN) (Jahangir et al., 2020) is proposed to predict wind speed by using rough
neurons to handle the high intermittent behavior of wind speed.

##  5\. Summary of the Key Findings

We summarize the key findings from our survey by answering the key research
questions below: RQ1. What are the key causes and types of uncertainty studied
in belief theory and deep learning?

Answer: The majority of belief models, such as DST, TBM, IDM, SL, TVL, and
Bayesian inference, consider uncertainty caused by a lack of evidence, which
is called vacuity in SL. It is related to aleatoric uncertainty where a long-
term probability can increase as more evidence is received. The second most
common uncertainty type is considered in belief models, such as DSmT, SL, or
Fuzzy Logic, is discord (or dissonance), caused by disagreement or conflicting
evidence from multiple sources or observers, which generates multiple
knowledge frames and results in inconclusiveness in decision making. Lastly,
unpredictability is introduced by unclearness or impreciseness of observations
or beliefs, which are considered as fuzziness in TVL, vagueness in SL, and
imprecise beliefs in DSmT. In DL, two types of uncertainty natures are mainly
considered: epistemic uncertainty and aleatoric uncertainty. Epistemic
uncertainty, also called ‘model or systematic uncertainty,’ represents the
model (parameters) uncertainty due to the limited training data. Aleatoric
uncertainty indicates data uncertainty introduced by the nature of randomness
in data.

RQ2. How can the ontology of uncertainty be defined based on the
multidimensional aspects of uncertainty studied in belief models and deep
learning?

Answer: We demonstrated the ontology of uncertainty in Fig. 2. The source of
uncertainty can be from machines, networks, environmental factors, and humans
that can generate a lot of various types of uncertain data. Uncertainty has a
model of reasoning and quantifying various types of uncertainties to make
effective decision making. We limited the models in belief theory and DL.
Uncertainty has procedures to collect evidence, including both subjective and
objective data or information. Uncertainty has its multiple types including
ambiguity and fuzziness which also have been studied under different
taxonomies (see Fig. 1), such as vagueness, imprecision, unclearness, and so
forth. Uncertainty has its root nature in the most popularly used two types of
uncertainty: aleatoric and epistemic uncertainty.

RQ3. How has each belief model considered and measured uncertainty?

Answer: DST’s combination rule helps the decision by combining beliefs from
multiple information channels. IDM provides a belief range, rather than a
single value, allowing a decision maker to be aware of the magnitude of
uncertainty. In DSmT, Shannon’s entropy and the Probabilistic Information
Content (PIC) score are used to indicate uncertainty caused by a lack of
evidence where a decision is made based on GPT and DSmP. Bayesian inference
theory uses a variance or co-variance to measure uncertainty representing
unpredictability. In SL, one can use a projected belief that interprets
uncertainty (i.e., vacuity) based on its prior belief (i.e., base rate). If
there is very low vacuity but high dissonance, one can maximize vacuity by
offsetting the amount of the smallest belief mass while increasing vacuity to
have a high effect on new evidence. TVL uses an unknown status to model system
uncertainty and defines a set of logical operations to decide the system
status for decision making in system operations. Fuzzy Logic uses fuzzy
entropy to quantify the unpredictability and multiple knowledge frames of
fuzzy events.

RQ4. How has each belief model been applied in deep learning and vice-versa
for effective decision making under uncertainty?

Answer: TVL is used to solve classification problems in pattern recognition
tasks (Kashkevich and Krasnoproshin, 1979) and leveraged to establish a
database for natural language consultation (Dahl, 1979) in 1970s. We rarely
found any recent work using TVL in ML/DL applications. DST is mainly used to
fuse data from multi-sensors before conducting neural network training, or
fuse predictions from two identically trained models (Soua et al., 2016; Tong
et al., 2021; Tian et al., 2020). To our knowledge, TBM is also used to solve
classification problems but not used with ML/DL. DSmT is used along with ML/DL
to solve classification problems where it is integrated with SVM, CNN, LSTM,
and RF (Abbas et al., 2015; Ji et al., [n.d.]). IDM is used to improve ML
algorithms, such as AdaBoost (Utkin, 2015), Decision Tree (Serafín et al.,
2020), or näive classifier (Corani and de Campos, 2010). Fuzzy Logic is
combined with DNNs, named fuzzy DNNs, to deal with ambiguity in data (Das et
al., 2020; Holyoak, 1987; El Hatri and Boumhidi, 2018; Wang et al., 2016;
Zhang et al., 2014). Bayesian inference is mainly used to infer the posterior
distribution of Bayesian features (Sofman et al., 2006; Tripathi and
Govindaraju, 2007; G. Tian and Feng, 2011). SL’s vacuity and dissonance
uncertainty dimensions are considered in evidential neural networks for
uncertainty-aware decision making in classification problems (Sensoy et al.,
2018; Zhao et al., 2019b). Vacuity is used to detect out-of-distribution (OOD)
samples while dissonance is used to detect missclassification samples. Rough
set theory is combined with DNNs, named RDNNs, to deal with imprecise
information and uncertainty in data (e.g., ranges as values for input and/or
output variables) (Zhang, 2007; Yasdi, 1995; Lingras, 1996; Khodayar et al.,
2017).

RQ5. What are the key differences of uncertainty reasoning and quantification
in belief theory and deep learning?

Answer: Deep learning (DL) has received high attention because of its powerful
capability to deal with a large volume of high dimensional data and provide
solutions to maximize decision performance. However, as DL is limited in
dealing with uncertainty explicitly, it often faces the issue of
unexplainability, a well-known issue of explainable AI (XAI), due to its
nature of statistical inference. On the other hand, belief models provide
rigorous mathematical formulation based on a limited number of parameters
which can offer the capability to easily reason and quantify different types
of uncertainties. This merit of quantifiable uncertainty in belief models can
provide reasons to explain a decision made based on mathematical induction.
However, belief models suffer from dealing with a large volume of data, which
can be complemented by DL. Therefore, our work discussed how a belief model
(e.g., SL) has been bridged with DL to improve decision making capability
based on the merits of both approaches to achieve XAI.

RQ6. How can belief model(s) be applied in DL to solve complicated decision
making problems?

Answer: There may be various ways to leverage belief models considered in DL
research. One example we discussed in Section 4 is combining SL’s opinions
with DNNs by constructing evidential NNs (ENNs). That is, ENNs can be built to
generate evidence to formulate a subjective opinion in SL, rather than using a
common activation function, such as softmax, generating class probabilities.
Based on the estimated evidence in ENNs, we can calculate vacuity and
dissonance uncertainty by leveraging the operators in SL. Depending on the
degree of the quantified uncertainty values, such as vacuity, vagueness, or
dissonance, diverse algorithms can be developed for effective decision making.
Based on our prior work (Zhao et al., 2019b), we found vacuity is a promising
uncertainty type to detect out-of-distribution (OOD) samples while dissonance
is an uncertainty type that can effectively detect misclassification samples.

##  6\. Concluding Remarks

###  6.1. Insights, Lessons Learned, and Limitations

  * •

Recent efforts have been made to estimate different types of uncertainty in
our prior work (Jøsang et al., 2018) while vacuity and vagueness have been
mainly considered in the past (Shafer, 1976; Jøsang, 2016; Zadeh, 1965).
However, reasoning and quantification of other dimensions of uncertainty still
remains unaddressed in the literature. Furthermore, the question of how
different types of uncertainty can be helpful for effective decision making
has not been addressed in the literature.

  * •

When both all the belief masses and a prior belief supporting each belief are
the same, decision making becomes more challenging because it leads to
inconclusiveness. Although utility-based belief masses have been studied (Yang
et al., 2009; Jøsang, 2016), their contributions are limited with the
theoretical discussions based on simple examples.

  * •

Most belief/evidence theories and their uncertainty reasoning show high
maturity as they have been explored since the 1960s. However, they are mostly
theoretical and have not been thoroughly validated based on real datasets
and/or applications for effective decision making and learning.

  * •

Bayesian theorem and inference methods are the foundations of the advanced
machine learning and deep learning algorithms. However, the fitting of
Bayesian inference from an one-parameter model to a deep learning model with a
large volume of parameters can introduce non-trivial challenges in quantifying
uncertainties.

  * •

Although belief models have considered various types of uncertainties, as
described in Fig. 1, the terminologies of those types are often found very
similar but their distinctions have not been clarified. Although our survey
paper can help readers better understand the diverse types of uncertainties,
one may want to argue about our clarification of uncertainty types in Section
2. This implies that much more efforts should be made to investigate different
types of uncertainties and their effect on diverse decision making settings
and applications.

  * •

Some efforts leveraging both a belief theory and deep learning have been made,
such as fuzzy deep neural networks (FDNNs) combining fuzzy logic and DNNs (Das
et al., 2020), rough deep neural networks (RDNNs) combining rough logic and
DNNs (Lingras, 1996), and evidential neural networks (ENNs) combining SL and
DNNs (Zhao et al., 2020). However, although belief theories have been explored
for several decades and their uncertainty research has been matured more than
any other fields, their applications in reasoning and quantifying uncertainty
in DL are still in an infant stage.

  * •

Although we can consider misclassification detection and out-of-distribution
detection tasks to evaluate the accuracy of predictive uncertainty measured,
the metrics of predictive uncertainty have not been validated as it is hard to
determine the ground truth of measured uncertainty. To have more valid metrics
of predictive uncertainty, we need to develop a way to evaluate the data
generation process by considering the causes of uncertainty (e.g., how to
generate data with vagueness).

  * •

Uncertainty can be easily introduced by intelligent adversarial attacks taking
highly deceptive poisonous or evasion attacks. Detecting adversarial attacks
with the intent to increase various types of uncertainty should be the first
step to reduce noises or false information that can increase uncertainty
before estimating uncertainty in data for effective decision making.

###  6.2. Future Research Directions

  * •

Uncertainty quantification research can be explored more for the studies using
qualitative labels. Since belief theories, such as DST, DSmT, or TBM, can
provide the capability to fuse qualitative beliefs, their applications in
natural language processing (NLP) are promising. Other conventional NLP
methods can be compared to DSmT in handling uncertainty for the qualitative
beliefs and its effect on application performance.

  * •

When a different belief model, that has a different way of estimating
different types of uncertainty, is combined with deep learning, we can
investigate how the different ways of measuring uncertainty can impact
decision making performance.

  * •

A belief model, such as IDM, offers the ability to derive a belief without any
prior knowledge about a given proposition. In the settings that do not allow
any prior knowledge or information about the proposition, IDM can allow one to
make decisions under uncertainty.

  * •

To solve sequential decision making problems, belief models can be combined
with deep reinforcement learning. In particular, as IDM does not require
having prior knowledge in the decision making process, it can be easily used
for an RL agent to make decisions in the environment with no prior knowledge
and to learn an optimal action via trials and errors.

  * •

Although many different types of uncertainty have been discussed in belief
models, we can capture three main uncertainty types: vacuity caused by a lack
of evidence, vagueness (or fuzziness) by failing in capturing a singleton
belief, and discord (or dissonance) by conflicting evidence. This can be
further examined to propose a unified mathematical belief framework to
quantify various uncertainty types and belief masses for its broader
applicability.

  * •

Most uncertainty measurements approaches are designed for a singleton
prediction, such as image or node classifications. However, they may not be
able to extend for time series application because they ignore temporal
dependencies in uncertainty quantification. It is critical to developing novel
uncertainty metrics considering temporal dependencies of time series data. For
example, we may consider fusion operators in SL to fuse the subjective
opinions from different time steps.

  * •

Most uncertainty estimation research focuses on unstructured tasks, such as
classification and regression tasks. Meanwhile, for the structured prediction
tasks, such as language modeling (e.g., machine translation and named entity
recognition), we can further investigate a general, unsupervised,
interpretable uncertainty framework.

## Acknowledgement

This work is partly supported by the Army Research Office under Grant Contract
Number W91NF-20-2-0140 and NSF under the Grant Number 2107449, 2107450, and
2107451. The views and conclusions contained in this document are those of the
authors and should not be interpreted as representing the official policies,
either expressed or implied, of the Army Research Laboratory or the U.S.
Government. The U.S. Government is authorized to reproduce and distribute
reprints for Government purposes notwithstanding any copyright notation
herein.

## References

  * (1)
  * Abbas et al. (2015) N. Abbas, Y. Chibani, A. Martin, and F. Smarandache. 2015\.  The effective use of the DSmT for multi-class classification.  _Advances and Applications of DSmT for Information Fusion_ (2015), 359.
  * Abdar et al. (2021) M. Abdar, F. Pourpanah, S. Hussain, D. Rezazadegan, L. Liu, M. Ghavamzadeh, P. Fieguth, X. Cao, A. Khosravi, U. R. Acharya, V. Makarenkov, and S. Nahavandi. 2021.  A review of uncertainty quantification in deep learning: techniques, applications and challenges.  _Information Fusion_ (2021).
  * Ahmed et al. (2018) S. A. Ahmed, D. P. Dogra, S. Kar, and P. P. Roy. 2018\.  Surveillance scene representation and trajectory abnormality detection using aggregation of multiple concepts.  _Expert Systems with Applications_ 101 (2018), 43–55.
  * Alim et al. (2019) A. Alim, X. Zhao, J.H. Cho, and F. Chen. 2019\.  Uncertainty-aware opinion inference under adversarial attacks. In _The 2019 IEEE International Conference on Big Data (IEEE Big Data 2019)_. 6–15.
  * Bhushan et al. (2020) C. Bhushan, Z. Yang, N. Virani, and N. Iyer. 2020\.  Variational encoder-based reliable classification. In _2020 IEEE International Conference on Image Processing (ICIP)_. IEEE, 1941–1945.
  * Blasch et al. (2013) E. Blasch, J. Dezert, and B. Pannetier. 2013.  Overview of Dempster-Shafer and belief function tracking methods. In _Proceedings of SPIE_ , Vol. 8745. Baltimore, Maryland, USA.
  * Brugnach et al. (2008) M. Brugnach, A. Dewulf, C. Pahl-Wostl, and T. Taillieu. 2008\.  Toward a relational concept of uncertainty: about knowing too little, knowing too differently, and accepting not to know.  _Ecology and Society_ 13, 2 (2008), 30.
  * Charpentier et al. (2020) B. Charpentier, D. Zügner, and S. Günnemann. 2020\.  Posterior network: Uncertainty estimation without ood samples via density-based pseudo-counts.  _Advances in Neural Information Processing Systems_ 33 (2020), 1356–1367.
  * Chen et al. (2015) CL Philip Chen, Chun-Yang Zhang, Long Chen, and Min Gan. 2015\.  Fuzzy restricted Boltzmann machine for the enhancement of deep learning.  _IEEE Transactions on Fuzzy Systems_ 23, 6 (2015), 2163–2173.
  * Chen et al. (2018) W. Chen, J. An, R. Li, L. Fu, G. Xie, Md. Z. A. Bhuiyan, and K. Li. 2018\.  A novel fuzzy deep-learning approach to traffic flow prediction with uncertain spatial–temporal data features.  _Future Generation Computer Systems_ 89 (2018), 78–88.
  * Chen et al. (2009) W. Chen, Y. Wang, and S. Yang. 2009.  Efficient influence maximization in social networks. In _Proceedings of the 15th ACM SIGKDD_ (Paris, France). New York, NY, USA, 199–208.
  * Chopade and Narvekar (2017) H. A. Chopade and M. Narvekar. 2017.  Hybrid auto text summarization using deep neural network and fuzzy logic system. In _2017 International Conference on Inventive Computing and Informatics (ICICI)_. 52–56.
  * Codd (1986) E. F. Codd. 1986\.  Missing information (applicable and inapplicable) in relational databases.  _ACM Sigmod Record_ 15, 4 (1986), 53–53.
  * Corani and de Campos (2010) G. Corani and C. P. de Campos. 2010.  A tree augmented classifier based on extreme imprecise Dirichlet model.  _International Journal of Approximate Reasoning_ 51, 9 (Nov. 2010), 1053–1068.
  * Costa et al. (2018) Paulo Costa, Anne-Laure Jousselme, Kathryn Laskey, Erik Blasch, Valentina Dragos, Juergen Ziegler, Pieter de Villiers, and Gregor Pavlin. 2018\.  URREF: Uncertainty representation and reasoning evaluation framework for information fusion.  _Journal of Advances in Information Fusion_ 13, 2 (2018), 137–157.
  * Dahl (1979) V. Dahl. 1979.  Quantification in a three-valued logic for natural language question-answering systems. In _Proceedings of the 6th international joint conference on Artificial intelligence-Volume 1_. 182–187.
  * Das et al. (2020) R. Das, S. Sen, and U. Maulik. 2020.  A survey on fuzzy deep neural networks.  _ACM Computing Surveys (CSUR)_ 53, 3 (2020), 1–25.
  * Deepa and Swamynathan (2014) R. Deepa and S. Swamynathan. 2014.  _Recent Trends in Computer Networks and Distributed Systems Security Communications in Computer and Information Science_. Vol. 420.  Springer-Verlag Berlin Heidelberg, Chapter A Trust Model for Directory-Based Service Discovery in Mobile Ad Hoc Networks, 115–126.
  * Deng et al. (2016) Y. Deng, Z. Ren, Y. Kong, F. Bao, and Q. Dai. 2016.  A hierarchical fused fuzzy deep neural network for data classification.  _IEEE Transactions on Fuzzy Systems_ 25, 4 (2016), 1006–1012.
  * Dewulf et al. (2005) A. Dewulf, M. Craps, R. Bouwen, T. Taillieu, and C. Pahl-Wostl. 2005.  Integrated management of natural resources: Dealing with ambiguous issues, multiple actors and diverging frames.  _Water Science and Technology_ 52 (Feb. 2005), 115–24.
  * Dezert and Smarandache (2004) J. Dezert and F. Smarandache. 2004.  _Advances and Applications of DSmT for Information Fusion_.  American Research Press, Rehoboth, NM, USA, Chapter Advances on DSmT.
  * Dubois and Prade (1988) D. Dubois and H. Prade. 1988.  Representation and combination of uncertainty with belief functions and possibility measures.  _Computational Intelligence_ 4, 3 (1988), 244–264.
  * Dubois (1980) D. J. Dubois. 1980\.  _Fuzzy sets and systems: Theory and applications_. Vol. 144.  Academic press.
  * El Hatri and Boumhidi (2018) C. El Hatri and J. Boumhidi. 2018.  Fuzzy deep learning based urban traffic incident detection.  _Cognitive Systems Research_ 50 (2018), 206–213.
  * Fienberg (2006) S. E. Fienberg. 2006\.  When Did Bayesian Inference Become “Bayesian"?  _Bayesian Analysis_ 1, 1 (2006), 1–40.
  * G. Tian and Feng (2011) Y. Zhang G. Tian, Y. Xia and D. Feng. 2011.  Hybrid genetic and variational expectation-maximization algorithm for Gaussian-mixture-model-based brain MR image segmentation.  _IEEE transactions on information technology in biomedicine_ 15, 3 (2011), 373–380.
  * Gawlikowski et al. (2021) Jakob Gawlikowski, Cedrique Rovile Njieutcheu Tassi, Mohsin Ali, Jongseok Lee, Matthias Humt, Jianxiang Feng, Anna Kruspe, Rudolph Triebel, Peter Jung, Ribana Roscher, et al. 2021\.  A survey of uncertainty in deep neural networks.  _arXiv preprint arXiv:2107.03342_ (2021).
  * Guil (2019) F. Guil. 2019.  Associative classification based on the transferable belief model.  _Knowledge-Based Systems_ 182 (2019), 104800.
  * Hankin (2010) R. K. S. Hankin. 2010\.  A Generalization of the Dirichlet Distribution.  _Journal of Statistical Software_ 33, 11 (Feb. 2010).
  * Hariri et al. (2019) R. H. Hariri, E. M. Fredericks, and K. M. Bowers. 2019\.  Uncertainty in big data analytics: Survey, opportunities, and challenges.  _Journal of Big Data_ 6 (2019), 1–16.
  * Henni et al. (2019) A. H. Henni, R. B. Bachouch, O. Bennis, and N. Ramdani. 2019\.  Enhanced multiplex binary PIR localization using the transferable belief model.  _IEEE Sensors Journal_ 19, 18 (2019), 8146–8159.
  * Hernandez-Potiomkin et al. (2018) Y. Hernandez-Potiomkin, M. Saifuzzaman, E. Bert, R. Mena-Yedra, T. Djukic, and J. Casas. 2018\.  Unsupervised incident detection model in urban and freeway networks. In _2018 21st International Conference on Intelligent Transportation Systems (ITSC)_. IEEE, 1763–1769.
  * Hoff (2009) P. D Hoff. 2009\.  _A first course in Bayesian statistical methods_. Vol. 580.  Springer.
  * Holyoak (1987) K. J Holyoak. 1987\.  Parallel distributed processing: explorations in the microstructure of cognition.  _Science_ 236 (1987), 992–997.
  * Honer and Hettmann (2018) J. Honer and H. Hettmann. 2018.  Motion state classification for automotive LIDAR based on evidential grid maps and transferable belief model. In _2018 21st International Conference on Information Fusion (FUSION)_. IEEE, 1056–1063.
  * Hu et al. (2020) Y. Hu, Y. Ou, X. Zhao, J.-H. Cho, and F. Chen. 2020.  Multidimensional uncertainty-aware evidential neural networks.  _In Proceeding of the Thirty-fifth AAAI Conference on Artificial Intelligence_ (2020).
  * Hüllermeier and Waegeman (2021) Eyke Hüllermeier and Willem Waegeman. 2021.  Aleatoric and epistemic uncertainty in machine learning: An introduction to concepts and methods.  _Machine Learning_ 110, 3 (2021), 457–506.
  * Jahangir et al. (2020) H. Jahangir, M. A. Golkar, F. Alhameli, A. Mazouz, A. Ahmadian, and A. Elkamel. 2020\.  Short-term wind speed forecasting framework based on stacked denoising auto-encoders with rough ANN.  _Sustainable Energy Technologies and Assessments_ 38 (2020), 100601\.
  * Ji et al. ([n.d.]) X. Ji, Y. Ren, H. Tang, and J. Xiang. [n.d.].  DSmT-based three-layer method using multi-classifier to detect faults in hydraulic systems.  _Mechanical Systems and Signal Processing_ 153 ([n. d.]), 107513.
  * Jøsang (1999) A. Jøsang. 1999\.  An algebra for assessing trust in certification chains,” Proc. Network and Distributed Systems Security. In _Proceedings of Network and Distributed Systems Security (NDSS’99) Symposium_.
  * Jøsang (2001) A. Jøsang. 2001\.  A logic for uncertain probabilities.  _International Journal of Uncertainty, Fuzziness and Knowledge-based Systems_ 9, 3 (Jun. 2001).
  * Jøsang (2016) A. Jøsang. 2016\.  _Subjective Logic: A Formalism for Reasoning Under Uncertainty_.  Springer Publishing Company.
  * Jøsang et al. (2018) A. Jøsang, J. Cho, and F. Chen. 2018.  Uncertainty Characteristics of Subjective Opinions. In _2018 21st International Conference on Information Fusion (FUSION)_. 1998–2005.
  * Kabir et al. (2018) H. D. Kabir, A. Khosravi, M. A. Hosen, and S. Nahavandi. 2018\.  Neural network-based uncertainty quantification: A survey of methodologies and applications.  _IEEE access_ 6 (2018), 36218–36234.
  * Kashkevich and Krasnoproshin (1979) S. Kashkevich and V. V. Krasnoproshin. 1979.  A two-level automated pattern recognition complex.  _U. S. S. R. Comput. Math. and Math. Phys._ 19, 6 (1979), 227–239.
  * Khodayar et al. (2017) M. Khodayar, O. Kaynak, and M. Khodayar. 2017.  Rough deep neural architecture for short-term wind speed forecasting.  _Transactions on Industrial Informatics_ 13, 6 (2017), 2770–2779.
  * Kiureghian and Ditlevsen (2009) A. D. Kiureghian and O. Ditlevsen. 2009.  Aleatory or epistemic? Does it matter?  _Structural Safety_ 31, 2 (2009), 105–112.  Risk Acceptance and Risk Communication.
  * Kleene (1938) S. Kleene. 1938\.  On notation for ordinal numbers.  _The Journal of Symbolic Logic_ 3, 4 (1938), 150–155.
  * Klir and Ramer (1990) G. Klir and A. Ramer. 1990\.  Uncertainty in the dempster-shafer theory: a critical re-examination.  _International Journal of General System_ 18, 2 (1990), 155–166.
  * Kumar and Banerjee (2017) A. Kumar and M. Banerjee. 2017.  Kleene algebras and logic: boolean and rough set representations, 3-valued, rough set and perp semantics.  _Studia Logica_ 105, 3 (2017), 439–469.
  * Lesani and Bagheri (2006) M. Lesani and S. Bagheri. 2006.  Fuzzy trust inference in trust graphs and its application in semantic web social networks. In _World Automation Congress (WAS 2006)_. 1–6.
  * Li et al. (2012) Y. Li, J. Chen, and L. Feng. 2012.  Dealing with uncertainty: A survey of theories and practices.  _Transactions on Knowledge and Data Engineering_ 25, 11 (2012), 2463–2482.
  * Liao et al. (2009) H. Liao, Q. Wang, and G. Li. 2009.  A fuzzy logic-based trust model in grid. In _International Conference on Networks Security, Wireless Communications and Trusted Computing (NSWCTC 2009)_. Wuhan, Hubei, China, 608–614.