# Negative Shannon Information Hides Networks

Ming-Xing Luo  School of Information Science and Technology, Southwest
Jiaotong University, Chengdu 610031, China;
CSNMT, International Cooperation Research Center of China, Chengdu 610031,
China;

###### Abstract

Shannon information was defined for characterizing the uncertainty information
of classical probabilistic distributions. As an uncertainty measure it is
generally believed to be positive. This holds for any information quantity
from two random variables because of the polymatroidal axioms. However, it is
unknown why there is negative information for more than two random variables
on finite dimensional spaces. We first show the negative tripartite Shannon
mutual information implies specific Bayesian network representations of its
joint distribution. We then show that the negative Shannon information is
obtained from general tripartite Bayesian networks with quantum realizations.
This provides a device-independent witness of negative Shannon information. We
finally extend the result for general networks. The present result shows new
insights in the network compatibility from non-Shannon information
inequalities.

##  I Introduction

For a given discrete Markov process a fundamental problem is how to
characterize the information produced in such a statistical process. Suppose
the statistical outcomes of a set of possible events
$\\{x_{1},\cdots,x_{n}\\}$ with respectively occurrence frequencies as
$p_{1},\cdots,p_{n}\in[0,1]$. Is there a measure of how uncertain is of the
outcome except for its distribution? For any such a quantity denotes as
$H(p_{1},p_{2},\cdots,p_{n})$, it is reasonable to satisfy the following
axioms: (1) $H$ is continuous function in each variable $p_{i}$ of the
probability. (2) $H$ is an increasing function of sample number for the
uniform distribution. (3) $H$ is weighted summation of its single values if
any one choice is changed into two. These axioms imply the unique entropy
given by Shannon Shannon  as

$\displaystyle H(p_{1},p_{2},\cdots,p_{n})=-\sum_{i=1}^{n}p_{i}\log{}p_{i}$
(1)

This features the average uncertainty of a given statistical process.

Figure 1: (Color online) Schematic network configuration compatible with
negative Shannon mutual information. (a) A classical semantic example. Alice
and Bob who have not shared any relationship make friends assisted by
Matchmaker who is familiar with both. (b) A quantum realization of Bayesian
network in (a). There are two entangled states EPR  $\rho_{1}$ and $\rho_{2}$
which are shared by three parties, where
$\rho_{1}=\rho_{2}=\frac{1}{\sqrt{2}}(|00\rangle+|11\rangle)$ on two-
dimensional Hilbert space $\mathcal{H}$ spanned by the orthogonal basis
$\\{|0\rangle,|1\rangle\\}$. Under local quantum measurements with positive-
operator-value matrices $\\{M_{x}=|x\rangle\langle
x|,x\in\\{0,1\\}\\},\\{M_{y}=|y\rangle\langle y|,y\in\\{0,1\\}\\}$ and
$\\{M_{z}=|z\rangle\langle z|,z\in\\{0,1\\}\\}$, three parties can generate a
joint distribution
$P_{xyz}=\frac{1}{4}[000]+\frac{1}{4}[011]+\frac{1}{4}[101]+\frac{1}{4}[110]$.
This follows $I(X;Y)=0$ and $I(X;Y|Z)=1$. This means Alice and Bob who are
initially independent of each other (e.g., $I(X;Y)=0$) can build new
correlations conditional on local measurements of Matchmaker (e.g.,
$I(X;Y|Z)=0$).

The Shannon entropy shows a remarkable application for mutual information of
two discrete random variables as

$\displaystyle I(X;Y)=H(X)+H(Y)-H(X,Y)$ (2)

where the mutual information means the uncertainty between two variables $X$
and $Y$. It is a measure of the information to which knowledge of one variable
reduces uncertainty about the other. These entropy functions satisfy the
polymatroidal axioms Fuji  of $H(X),I(X;Y)\geq 0$ for any finite dimensional
variables $X$ and $Y$. Shannon actually shows general information inequalities
which are the ”physical laws” for characterizing the fundamental limits in
classical communications and compression Shannon . However, this intrigues a
surprising feature of negative information for three or more discrete
variables beyond the polymatroidal axioms. Each one in fact presents a so-
called non-Shannon inequality Zhang1997 . One primitive example is from the
mutual information contained in three discrete variables $X,Y$ and $Z$ on
finite dimensional spaces as

$\displaystyle I(X;Y;Z)$ $\displaystyle=$ $\displaystyle I(X;Y)-I(X;Y|Z)$ (3)
$\displaystyle=$ $\displaystyle H(X)+H(Y)+H(Z)+H(X,Y,Z)$
$\displaystyle-H(X,Y)-H(Y,Z)-H(X,Z)$

where $I(X;Y|Z)=H(Y,Z)+H(X,Z)-H(Z)-H(X,Y,Z)$ denotes the mutual information
conditional on the outcome of variable $Z$. One example is shown in Fig.1. The
proper local measurements on the network (a) may generate a joint probability
distribution $P_{xyz}$ with the notation of
$P_{xyz}=\frac{1}{4}[000]+\frac{1}{4}[011]+\frac{1}{4}[101]+\frac{1}{4}[110]$,
where $[xyz]$ denotes joint event $(X=x,Y=y,Z=z)$, and the probability of
joint outcome $[xyz]=[000],[011],[101]$ or $[110]$ is $\frac{1}{4}$. This
probability distribution yields to a negative information of $I(X;Y;Z)=-1$.
Especially, the new correlations are built for two independent parties Alice
and Bob assisted by the other’s local operations, that is, the mutual
information conditional on the outcome of $Z$ is given by $I(X;Y|Z)=1$ while
the mutual information $I(X;Y)$ is zero. This may imply a simple explanation
of the Shannon negative information from Bayesian networks Pearl . This
intrigues a natural problem for characterizing general negative Shannon
information.

In Hilbert space formulation of Bayesian network in Fig.1(a), each bipartite
edge is replaced by an entanglement EPR ; NC , as shown in Fig.1(b). Under the
proper postulates of quantum state representation, quantum measurement, Born
rule and tensor decomposition of compose systems, the quantum probability is
given by $p(x,y,z)={\rm Tr}((M_{x}\otimes M_{y}\otimes
M_{z})\rho_{1}\otimes\rho_{2})$, where $\\{M_{x}\\}$, $\\{M_{y}\\}$, and
$\\{M_{z}\\}$ denotes respective quantum measurements of Alice, Bob and
Matchmaker, and ${\rm Tr}(\cdot)$ denotes the trace operation of matrix. This
quantum probability shows not only the similar features of classical
statistics of $I(X;Y;Z)\leq 0$, but also the quantumness of entanglement EPR ;
Bell , i.e., two independent parties Alice and Bob can build quantum
entanglement assisted by the other’s local operations and classical
communication Entswap1993 ; Cava . This provides a simple physical model for
verifying Bayesian network and the new quantumness of entanglement assisted by
other party from its statistical distribution.

Our motivation in this work is to investigate a general problem of the
Bayesian network compatibility of the negative Shannon mutual information. For
a given tripartite joint probability distribution with negative mutual
information on finite sample spaces, we firstly classify all the compatible
Bayesian networks. We show there are intrinsic network configurations for
these distributions, that is, chain network consisting of two edges or
triangle network consisting of three edges. This implies a device-independent
verification of negative Shannon mutual information using quantum networks.
The feature of negative Shannon mutual information is generic for any
tripartite quantum entangled network or general multipartite networks.

##  II Result

###  II.1 Negative Shannon mutual information in Bayesian network model

.

We first introduce some notations of Bayesian networks Pearl . A graph
$\mathcal{G}$ consists of a vertex (or node) set $V$, and an edge (or link)
set $E$. The vertices in a given graph are corresponding to measurable
variables, and the edges denote certain relationships that hold in pairs of
variables. A bi-directed edge denotes the existence of unobserved common
causes. These edges will be marked as curved arcs with two arrowheads, as
shown in Fig.2. If all edges are directed, we then have a directed graph.

Directed graph may include directed cycles. One example is given by $X\to Y\to
Z\to X$ with $X,Y,Z\in V$, which represents mutual causation or feedback
processes. The self-loops (e.g., $X\to X$) are not allowed in what follows. A
graph that contains no directed cycle is called acyclic. A graph that is both
directed and acyclic is called a directed acyclic graph (DAG). A family in a
graph is a set of nodes containing a node and all its parents, where the
parents of one node mean all nodes which are connected to it.

Denote a probability distribution as $P_{x}=\tilde{\Sigma}_{x}p_{x}[x]$ with
random variable $X$ on a finite sample space $\mathcal{X}$, where $[x]$
denotes the event of $X=x$ and $p_{x}$ denotes the probability of the outcome
$x$. The notation $\tilde{\Sigma}$ does not mean the summation but a notation
of union of all possible events of a given probability distribution. Similar
notations will be used for multivariate joint distributions
$P_{x_{1}\cdots{}x_{n}}$ on finite sample space
$\times_{i=1}^{n}\mathcal{X}_{i}$. Consider the task of specifying an
arbitrary joint distribution
$P_{x_{1}\cdots{}x_{n}}=\tilde{\Sigma}_{x_{1}\cdots{}x_{n}}p_{x_{1}\cdots{}x_{n}}[x_{1}\cdots{}x_{n}]$
for $n$ random variables, $X_{1},\cdots,X_{n}$ on finite sample spaces
$\mathcal{X}_{1},\cdots,\mathcal{X}_{n}$, respectively, and
$[x_{1}\cdots{}x_{n}]$ denotes the joint event of
$X_{1}=x_{1},\cdots,X_{n}=x_{n}$. The basic Bayes rule allows us decompose
$p_{x_{1}\cdots{}x_{n}}$ into

$\displaystyle p_{x_{1}\cdots{}x_{n}}=\prod_{j}p_{x_{j}|x_{1},\cdots,x_{j-1}}$
(4)

where $p_{x_{j}|x_{1},\cdots,x_{j-1}}$ denotes the probability of outcome
$x_{j}$ conditional on the outcomes of predecessors $x_{1},\cdots,x_{j-1}$.
Suppose that each $x_{j}$ is dependent of a small subset $pa(x_{j})$ of its
predecessors. We have the following definition.

Definition 1. (Markovian Parents) Pearl  Let $V=\\{X_{1},\cdots,X_{n}\\}$ be
an ordered set of measurable variables, and let $P_{x_{1},\cdots,x_{n}}$ be
the joint probability distribution on these variables. A set of variables
$PA_{j}$ is said to be Markovian parents of $X_{j}$ if $PA_{j}$ is a minimal
set of predecessors of $X_{j}$ that renders $X_{j}$ independent of all its
other predecessors, that is,

$\displaystyle p_{x_{j}|pa_{j}}=p_{x_{j}|x_{1},\cdots,x_{j-1}}$ (5)

and such that no proper subset of $PA_{j}$ satisfies (5).

Definition 1 implies for each $X_{j}$ there is a set $PA_{j}$ of preceding
variables for determining its probability. This can be represented by DAG,
where $PA_{j}$ denotes all the parent nodes toward the node $X_{j}$ Pearl .
Definition 1 provides a simple recursive algorithm for constructing such a DAG
for a given
$P_{x_{1}\cdots{}x_{n}}=\tilde{\sum}_{x_{1},\cdots,x_{n}}p_{x_{1}\cdots{}x_{n}}[x_{1}\cdots{}x_{n}]$
as follows.

  * Algorithm 1

  * (i)

Starting with the pair $(X_{1},X_{2})$, we draw an arrow from $X_{1}$ to
$X_{2}$ if and only if the two variables are dependent.

  * (ii)

For $X_{3}$, we draw an arrow from either $X_{1}$ or $X_{2}$ to $X_{3}$ if
$X_{3}$ is dependent of $X_{1}$ or $X_{2}$;

  * (iii)

For $j\geq 3$, one can select any minimal set $PA_{j}$ of $X_{j}$’s possible
predecessors. And then, draw an arrow from each member in $PA_{j}$ to $X_{j}$.

This follows an iterative algorithm to get a DAG of Bayesian network
compatible with the given distribution $P_{x_{1}\cdots{}x_{n}}$. It has been
shown that $PA_{j}$ is unique for a given distribution
$P_{x_{1}\cdots{}x_{n}}$ Pearl . From the Reichenbach’s common cause principle
Pearl ; Markov , it allows a Markovian decomposition as

$\displaystyle p_{x_{1}\cdots{}x_{n}}=\prod_{j}p_{x_{j}|pa(x_{j})}$ (6)

The Markovian dependence is represented by the directed acyclic graph (DAG) of
Bayesian networks Pearl .

The DAG shows probabilistic and statistical importance for data mining and
efficient inferences. A basic problem in statistics theory is to explore the
related DAG for a given statistical distribution. The most common way to
explore possible DAGs from observations is based on the Markov decomposition
in Eq.(6) and the faithfulness assumption Pearl ; SG .

Definition 2 Pearl  (Markov Compatibility) If a probability function
$P_{x_{1}\cdots{}x_{n}}$ admits the factorization of (6) relative to DAG
$\mathcal{G}$, then $\mathcal{G}$ and $P_{x_{1}\cdots{}x_{n}}$ are compatible.

In classical realization of Bayesian networks, each edge is represented by one
measurable variable on proper measurable space, and each outcome depends on
all the related variables Pearl . The joint distribution is a multivariate
function of all outcomes. This allows us to decompose $P_{x_{1}\cdots{}x_{n}}$
with one measurable variable $\lambda$ Bell  as:

$\displaystyle
p_{x_{1}\cdots{}x_{n}}=\int_{\Omega}p(x_{1}|\lambda)\cdots{}p(x_{n}|\lambda)\mu(\lambda)d\lambda$
(7)

where $(\Omega,\mu(\lambda))$ denotes the measurable space of unobservable
latent variable $\lambda$, and $\mu(\lambda)$ denotes the probability of
$\lambda$, and $p(x_{n}|\lambda)$ denotes the characteristic function of
outcome $x_{n}$ conditional on the variable $\lambda$. This kind of Bayesian
networks with latent variables shows non-trivial constraints on its
correlations TP ; KT .

For a joint distribution of two random variables $X$ and $Y$ on finite sample
spaces $\mathcal{X}\times\mathcal{Y}$, Eq.(1) is used for featuring the common
uncertainty of both variables Shannon . The nonnegative of $H$ and $I$ from
the polymatroidal axioms Fuji  is useful for solving the Markov compatibility
with single latent variable in Eq.(7). Instead, the entropy function in Eq.(3)
shows the tripartite mutual uncertainty only for Markov chains Markov , that
is,

$\displaystyle I(X;Y;Z)\geq 0$ (8)

if $\\{X,Y,Z\\}$ (under any order) consists of a Markov chain Pearl  (see
proofs in Appendix A). This inspires a generalized unordered Markov condition
of $I(X;Y;Z)\geq 0$. The joint probability distribution can be generated from
one single latent variable as Eq.(7). Remarkably, there are joint
distributions implying negative Shannon mutual information, that is,
$I(X;Y;Z)<0$. One example is shown in Fig.2(a). The coarse-grained single-
variable model (7) does not imply any intrinsic feature of this case. Instead,
we prove new Markov compatibilities for these distributions using Bayesian
networks Pearl .

Figure 2: (Color online) Schematic Markov compatibility of negative Shannon
mutual information. (a) Classical chain network consisting of two latent
variables $\lambda$ and $\gamma$. (b) Quantum chain network consisting of two
bipartite states $\rho_{1}$ and $\rho_{2}$ on Hilbert space $\mathcal{H}$. (c)
Classical triangle network consisting of three latent variables $\lambda$,
$\gamma$ and $\eta$. (d) Classical tripartite network consisting of one latent
variable $\lambda$.

Case one. From the definition in Eq.(3) the first scenarios satisfies the
following constrains:

$\displaystyle I(X;Y)$ $\displaystyle=$ $\displaystyle 0,I(X;Y|Z)>0.$ (9)

Different from the Markov conditional independence of $p(x,y|z)=p(x|z)p(y|z)$
Pearl , the present condition in Eq.(9) implies a conditional dependence, that
is, two independent random variables $X$ and $Y$ on
$\mathcal{X}\times\mathcal{Y}$ can build new correlations conditional on the
outcome of variable $Z$. It can be mathematically formulated as
$p(x,y)=p(x)p(y)$ and $p(x,y|z)\not=p(x|z)p(y|z)$. The so-called anti-Markov
condition provides a primitive explanation of nonnegative Shannon mutual
information with $I(X;Y;Z)<0$.

Especially, consider the example shown in Fig.2(a). Combined with classical
Birkhoff transformation Birk  (e.g., a doubly stochastic matrix $(a_{ij})$
which satisfies each column or each row consists of a probability
distribution, i.e., $\sum_{i}a_{ij}=\sum_{j}a_{ij}=1$ and $a_{ij}\geq 0$), the
joint distribution
$P_{xyz}=\frac{1}{4}[000]+\frac{1}{4}[011]+\frac{1}{4}[101]+\frac{1}{4}[110]$
can be obtained from the following joint distribution of four variables as

$\displaystyle
P_{xyz_{1}z_{2}}=\frac{1}{4}[0000]+\frac{1}{4}[0101]+\frac{1}{4}[1010]+\frac{1}{4}[1111]$
(10)

where the outcomes of the variable $Z$ are classically encoded as:
$\mathcal{F}:z_{1}z_{2}\mapsto z=z_{1}\oplus{}z_{2}$. Different from the joint
distribution $P_{xyz}$, the new distribution in Eq.(10) allows the following
Markovian decomposition of $P_{xyz_{1}z_{2}}=P_{xz_{1}}P_{yz_{2}}$, where
$P_{xz_{1}}$ and $P_{yz_{2}}$ are joint distributions of two variables given
by $P_{xz_{1}}=P_{yz_{2}}=\frac{1}{2}[00]+\frac{1}{2}[11]$. This means both
the variables $X$ and $Y$ are independent. Moreover, it is easy to get
$H(X)=H(Y)=1,H(X,Y)=H(X,Y,Z)=2$ and $H(X,Z)=H(Y,Z)=1$. This implies the
distribution $P_{xyz_{1}z_{2}}$ satisfies the inequality (9).

Now, we continue the proof. Combining with Eq.(7) the distribution $P_{xyz}$
allows a classical chain network decomposition as shown in Fig.2(a). This can
be represented by

$\displaystyle p_{xyz}$ $\displaystyle=$
$\displaystyle\int_{\Omega_{1}\otimes\Omega_{2}}p(x|\lambda)p(y|\gamma)p(z|\lambda,\gamma)$
(11) $\displaystyle\times\mu(\lambda)\mu(\gamma)d\lambda{}d\gamma$

where $(\Omega,\mu(\lambda))$ denotes the measurable space of the latent
variable $\lambda$, and $\mu(\lambda)$ denotes the probability of $\lambda$,
and similar definitions for the latent variable $\gamma$. This example can be
extended for general probability distribution satisfying the condition in
Eq.(9). Interestingly, a further analysis shows the rigidity of this Bayesian
network compatibility with any joint distribution satisfying the condition in
Eq.(9) (see Appendix B).

Result 1. Any joint distribution satisfying the condition in Eq.(9) is
compatible with a chain Bayesian network.

In Hilbert space formulation, a finite-dimensional pure state is represented
by a normalized vector $|\phi\rangle$ in Hilbert space $\mathcal{H}_{A}$ NC .
An ensemble of pure states $|\phi_{i}\rangle$ with a mixing probability
$p_{i}$ is represented by a density matrix
$\rho=\sum_{i}p_{i}|\phi_{i}\rangle\langle\phi_{i}|$ on Hilbert space
$\mathcal{H}_{A}$. Here, $\rho$ is positive semidefinite matrix with unit
trace. The multipartite quantum system is defined on the tensor of local
states, i.e., the tensor of Hilbert space as
$\otimes_{i=1}^{n}\mathcal{H}_{A_{i}}$. Any measurement acting on
$\mathcal{H}_{A}$ consists of an ensemble $\\{M_{x_{i}}\\}$ of projection
operators or generalized positive semidefinite operators satisfying
$\sum_{x_{i}}M_{x_{i}}=\openone_{A_{i}}$ with the identity operator
$\openone_{A_{i}}$. After all the local measurements on a given state $\rho$
on Hilbert space $\otimes_{i=1}^{n}\mathcal{H}_{A_{i}}$, from Born rule, the
quantum joint probability is given by

$\displaystyle p_{x_{1}\cdots x_{n}}={\rm Tr}[(M_{x_{1}}\otimes\cdots\otimes
M_{x_{n}})\rho]$ (12)

for the joint outcome $x_{1},\cdots,x_{n}$.

For special case of two particles $A$ and $B$, a state $\rho$ on Hilbert space
$\mathcal{H}_{A}\otimes\mathcal{H}_{B}$ is entangled if it cannot be
decomposed into

$\displaystyle\rho=\sum_{i}p_{i}\rho_{A}^{(i)}\otimes\rho_{B}^{(i)}$ (13)

where $\\{p_{i}\\}$ is a probability distribution, $\rho_{A}^{(i)}$ and
$\rho_{B}^{(i)}$ are states of respective particle $A$ and $B$. Similar
definitions may be extended for multiple particles HHH .

Instead of classical chain network in Fig.2(a), for any joint distribution
$P_{xyz}$ satisfying the condition (9) it is compatible with quantum networks
consisting of two generalized Einstein-Podolsky-Rosen (EPR) states EPR :
$|\phi_{1}\rangle_{AB}=\frac{1}{\sqrt{2}}(|00\rangle+|11\rangle)$ and
$|\phi_{2}\rangle_{B^{\prime}C}=\frac{1}{\sqrt{2}}(|00\rangle+|11\rangle)$, as
shown in Fig.2(b). A simple local measurement strategy implies a quantum joint
distribution

$\displaystyle
P_{xyz}=\frac{1}{4}[0000]+\frac{1}{4}[0101]+\frac{1}{4}[1010]+\frac{1}{4}[1111]$
(14)

where Alice and Bob performs respective projection measurements
$\\{M_{x},x\in\\{0,1\\}\\}$ and $\\{M_{y},y\in\\{0,1\\}\\}$ while the other
performs local measurement $\\{M_{z}=|z\rangle\langle
z|,z\in\\{00,\cdots,11\\}\\}$. This provides a possible experimental
verification of any negative Shannon information satisfying Eq.(9). As a
directive result, any joint distribution in Eq.(14) generated from local
measurements on the quantum chain network in Fig.2(b) with any bipartite
states $\rho_{1}$ and $\rho_{2}$ is compatible with the classical distribution
in Eq.(11), that is,

$\displaystyle\mathcal{S}_{c}=\mathcal{S}_{q}$ (15)

where $\mathcal{S}_{c}$ consists of all classical probability distributions in
Eq. (11) with respect to any two measurable variables $\lambda$ and $\gamma$,
or equivalently represented by
$\mathcal{S}_{c}=\\{P_{xyz}^{(c)}|I(X;Y)=0,I(X;Y|Z)>0\\}$; $\mathcal{S}_{q}$
consists of all the quantum probability distributions in Eq.(14) derived from
any two quantum states $\rho_{1}$ and $\rho_{2}$, that is,
$\mathcal{S}_{q}=\\{P_{xyz}^{(q)}|I(X;Y)=0,I(X;Y|Z)>0\\}$. The equality (15)
means that there is no quantum nonlocality beyond classical networks for
quantum chain network if each party has only one set of local measurements.
This means that for a tripartite chain network its classical realization with
two independent variables can simulate all quantum correlations from any
realization of two independent entangled states with one measurement setting
per party. This is different from previous results with more than one
measurement settings on chain networks Cava ; 15 . It is also different from
the triangle network consisting of three variables with one measurement
setting Marc2019 . This may inspire another interesting problem for what the
network ingredients may inspire the quantum nonlocality for a general network.

Case two. For dependent random variables $X$ and $Y$ on respective finite
sample spaces $\mathcal{X}$ and $\mathcal{Y}$, it suggests the second
scenarios for $I(X;Y;Z)\leq 0$ as

$\displaystyle I(X;Y)>0,I(X;Y|Z)>0.$ (16)

One example is shown as

$\displaystyle P_{xyz}$ $\displaystyle=$
$\displaystyle\frac{1}{8}[000]+\frac{1}{8}[011]+\frac{1}{8}[102]+\frac{1}{8}[113]$
(17)
$\displaystyle+\frac{1}{8}[220]+\frac{1}{8}[231]+\frac{1}{8}[322]+\frac{1}{8}[333]$

which has $I(X;Y)=1$ and $I(X;Y|Z)=2$. Here, the conditional mutual
information $I(X;Y|Z)$ is defined in Eq.(3). By using the binary
representation of $i$, $g:i\mapsto i_{1}i_{2}$, we may construct a compatible
network model Pearl  by two steps. One is to split $P_{xyz}$ into two joint
distributions of $P_{x_{1}y_{1}}=\frac{1}{2}[00]+\frac{1}{2}[11]$ and
$P_{x_{2}y_{2}z}$ in Eq.(10) by using local classical transformation $g$, that
is,

$\displaystyle P_{g(x)g(y)z}$ $\displaystyle=$ $\displaystyle
P_{x_{1}x_{2}y_{1}y_{2}z}=P_{x_{1}y_{1}}\times P_{x_{2}y_{2}z}$ (18)
$\displaystyle=$
$\displaystyle\frac{1}{8}[00000]+\frac{1}{8}[00011]+\frac{1}{8}[01002]+\frac{1}{8}[01013]$
$\displaystyle+\frac{1}{8}[10100]+\frac{1}{8}[10111]+\frac{1}{8}[11102]+\frac{1}{8}[11113]$
$\displaystyle{g^{-1}\atop\to}$ $\displaystyle P_{xyz}$

under the inverse mapping $g^{-1}$ of $g$ for $x$ and $y$, where the joint
distribution $P_{x_{1}y_{1}}$ satisfies $I(X_{1};X_{2})=I(X;Y)$, and the joint
distribution $P_{x_{2}y_{2}z}$ satisfies $I(X_{2};Y_{2})=0$ and
$I(X_{2};Y_{2}|Z)=2$. Note that the joint distribution $P_{x_{1}y_{1}}$ can be
generated by one latent variable $\lambda_{3}$ from Eq.(7) while
$P_{x_{2}y_{2}z}$ is compatible with the chain network using Result 1. This
fact implies a new compatible network consisting of three latent variables, as
shown in Fig.2(c).

In general, for a given distribution $P_{xyz}$ satisfying the condition in
Eq.(16), suppose there is classical transformation $g:x\mapsto x_{1}x_{2}$
such that

$\displaystyle P_{g(x)g(y)z}$ $\displaystyle=$ $\displaystyle
P_{x_{1}x_{2}y_{1}y_{2}z}=P_{x_{1}y_{1}}\times P_{x_{2}y_{2}z},$ (19)

where the joint distribution $P_{x_{2}y_{2}z}$ satisfies the condition in Eq.
(9). Under this assumption, we can obtain the following result.

Result 2. For any joint distribution on finite sample spaces satisfying the
condition in Eq.(16), there is a compatible triangle network if the
decomposition in Eq.(19) holds.

A recent result shows triangle quantum network shows nonlocal correlations
beyond all classical realizations under local measurement assumptions Marc2019
, that is, by performing one set of local measurements on a triangle quantum
network consisting of three entangled states there are quantum tripartite
joint probability $P_{xyz}$ which cannot be generated from any classical
triangle network consisting of three measurable variables and local
measurements. Interestingly, all of their quantum distributions satisfy the
condition in Eq.(16). This implies the inequivalence of quantum and classical
realizations of triangle network in Fig.2(c). Thus the decomposition in
Eq.(19) provides a sufficient condition to verify triangle network, and may be
evaluated by using numeric methods Spirt .

The other is a tripartite network consisting of one latent variable as shown
in Fig.2(d). One example is the distribution
$P_{xyz}=\frac{1}{3}[001]+\frac{1}{3}[010]+\frac{1}{3}[100]$, which cannot be
generated from all networks in Fig.2(a)-(c) Blou ; NW ; Dur .

This yields to a further problem to distinguish different configurations of
triangle networks. Here, we present an informational method as (see proofs in
Appendix C):

$\displaystyle I(X;Y;Z)$ $\displaystyle\leq$
$\displaystyle\min\\{H(X|Y,Z),H(Y|X,Z),H(Z|X,Y)\\}$ $\displaystyle I(X;Y;Z)$
$\displaystyle>$ $\displaystyle\frac{3}{2}(H(X)+H(Y)+H(Z))$ (21)
$\displaystyle-H(X,Y)-H(X,Z)-H(Y,Z)$

which hold for the network in Fig.2(c). Instead, for the network in Fig.2(d)
it follows

$\displaystyle I(X;Y;Z)$ $\displaystyle\leq$ $\displaystyle 4H(X,Y,Z)-H(X,Y)$
(22) $\displaystyle-H(X,Z)-H(Y,Z)$

and

$\displaystyle I(X;Y;Z)$ $\displaystyle>$
$\displaystyle\frac{4}{3}(H(X)+H(Y)+H(Z))$ (23)
$\displaystyle-H(X,Y)-H(X,Z)-H(Y,Z)$

Example 1. Consider a Greenberger-Horne-Zeilinger (GHZ)-type distribution:

$\displaystyle P_{ghz}=a[000]+(1-a)[111]$ (24)

where $a\in[0,1]$. This distribution can be generated by local projection
measurements under the computation basis $\\{|0\rangle,|1\rangle\\}$ on a GHZ
state GHZ : $\sqrt{a}|000\rangle+\sqrt{1-a}|111\rangle$. It is easy to prove
that $I(X;Y;Z)\geq 0$ for $a\geq 0$. Moreover, we show that this distribution
is generated from a triangle network in Fig.2(d) by violating the inequality
(21) for $a>0$.

Example 2. Consider a mixture of GHZ-type distribution and W-type distribution
as

$\displaystyle P_{xyz}=pP_{ghz}+(1-p)P_{w}$ (25)

where $P_{ghz}=\frac{1}{2}[000]+\frac{1}{2}[111]$,
$P_{w}=\frac{1}{3}[001]+\frac{1}{3}[010]+\frac{1}{3}[100]$, and $p\in[0,1]$.
Here, the W-type distribution $P_{w}$ can be generated by local projection
measurements under the computation basis $\\{|0\rangle,|1\rangle\\}$ on a W
state Dur : $\frac{1}{\sqrt{3}}(|001\rangle+|010\rangle+|100\rangle)$. It
follows from $I(X;Y;Z)\leq 0$ for $p\leq 0.814$ assisted by numeric
evaluations. Moreover, the distribution is generated from a triangle network
in Fig.2(d) by violating the inequality (LABEL:eqn16a) or (21) for $p\geq
0.836$.

Example 3. Consider a generalized W-type distribution Dur :

$\displaystyle P_{w}=a[001]+b[010]+(1-a-b)[100],$ (26)

where $a,b\geq 0$ and $0\leq a+b\leq 1$. We show that this distribution
implies a negative Shannon mutual information of $I(X;Y;Z)<0$ for any $a,b$
(Appendix D). Unfortunately, this distribution cannot be verified by using the
inequalities (LABEL:eqn16a) and (21). Instead, we verify any one of the
following distributions (Appendix E)

$\displaystyle P_{xyz}=a[001]+b[010]+c[100]+d[011],$ (27) $\displaystyle
P_{xyz}=a[001]+b[011]+c[100]+d[110],$ (28)

and its permutations using the second-order inflation method Blou , as shown
in Fig.3, where $a,b,c,d\geq 0$ and $a+b+c+d=1$. Here, the triangle network
consisting of $X_{1},Y_{1},Z_{1}$ and $\lambda_{1},\gamma_{1},\eta_{1}$ is the
first-order inflation of the initial network consisting of $X,Y,Z$ and
variables $\lambda,\gamma,\eta$. Moreover, the triangle network consisting of
$X_{2},Y_{2},Z_{2}$ and $\lambda_{2},\gamma_{2},\eta_{2}$ is the second-order
inflation of the initial network.

Figure 3: (Color online) (a) Triangle network consisting of three latent
variables $\lambda,\gamma$ and $\eta$. (b) Second-order inflation of triangle
network. Here, the network consisting of $\lambda_{1},\gamma_{1}$ and
$\eta_{1}$ denotes the first-order inflation while $\lambda_{2},\gamma_{2}$
and $\eta_{2}$ consist of the second-order.

###  II.2 Generic negative Shannon mutual information

Results 1 and 2 show specific network decompositions can be arisen from of
negative Shannon mutual information. Our consideration here is for exploring
its converse problem, that is, what information can be learned from a given
tripartite network configuration? In general, for a given tripartite Bayesian
network $\mathcal{N}$ (consisting of one, two, or three independent measurable
variables), define the minimal tripartite mutual information as

$\displaystyle I_{min}(X;Y;Z)=\min_{P_{xyz}}\\{I(X;Y;Z)\\}$ (29)

where the minimum is over all the possible distributions $P_{xyz}$ generated
from $\mathcal{N}$. Informally, any one in a tripartite Bayesian network can
locally generate additional information for others than its absence. This
shows generic negative Shannon mutual information for tripartite networks
(Appendix F).

Result 3. Any distribution from tripartite Bayesian networks satisfies

$\displaystyle I_{min}(X;Y;Z)\leq 0$ (30)

Result 3 intrigues an interesting indicator of any tripartite network by
defining the tripartite information increasing as

$\displaystyle\Delta$ $\displaystyle:=$
$\displaystyle\max\\{I(X;Y|Z)-I(X;Y)\\}$ (31) $\displaystyle=$ $\displaystyle-
I_{min}(X;Y;Z)$

This can be used to characterize how much information can be built by local
operations and classical communication (LOCC) of one party. It is of a
fundamental rule of management science. One example is the organizational
theory Kem , where the indicator $\Delta$ can be used to characterize the
increasing information by group behaviors associated with the network
relationship. This may be extended and applied for general organizational
theory beyond the scope of this paper.

###  II.3 Negative Shannon mutual information from general networks

Results 1-3 shows the negative Shannon mutual information of three variables
imply the compatible Bayesian networks. A natural problem is to explore
general networks. Our method here is from so-called multipartite independent
networks Luo2018 , that is, there are some nodes which have not shared any
entanglement. Especially, consider an $n+m$-partite quantum network
$\mathcal{N}_{n}$ consisting of generalized EPR states EPR :
$|\phi_{1}\rangle=\cos\theta_{i}|00\rangle+\sin\theta_{i}|11\rangle$ with
$\theta_{i}\in(0,\frac{\pi}{2})$, $i=1,\cdots,N$. Our goal here is to consider
the multipartite independent network Luo2018 . Especially, assume that there
are $n$ number of nodes $\textsf{A}_{1},\cdots,\textsf{A}_{n}$ in
$\mathcal{N}_{n}$ with $n\geq 2$ such that each pair of them has not shared
any entanglement, as shown in Fig.4.

Denote $X_{i}\in\mathcal{X}_{i}$ and $Y_{j}\in\mathcal{Y}_{j}$ as the
respective outcomes of $\textsf{A}_{i}$ and $\textsf{B}_{j}$ under local
projection measurements, $i=1,\cdots,n;j=1,\cdots,m$. Our main result here is
to prove that any general $n$-independent quantum network $\mathcal{N}_{n}$
(with $n\geq 3$) shows different features beyond $\mathcal{N}_{2}$, that is,
the chain network in Fig.2(b). Specially, we show that both negative and
positive mutual information can be generated from $\mathcal{N}_{n}$ with
$n\geq 3$. Here, by using Eq.(3) iteratively and the equality
$H(X_{1},\cdots,X_{j}|X_{j+1})=H(X_{1},\cdots,X_{j},X_{j+1})-H(X_{j+1})$ the
mutual information of multivariate is defined as

$\displaystyle
I(X_{1};\cdots;X_{n};Y)=I(X_{1};\cdots;X_{n})-I(X_{1};\cdots;X_{n}|Y)$ (32)

where $I(X_{1};\cdots;X_{n})$ denotes the mutual information of variables
$X_{1},\cdots,X_{n}$ and can be defined by
$I(X_{1};\cdots;X_{n})=\sum_{\mbox{\small odd }i\leq n}\sum_{1\leq
j_{1}<\cdots<j_{i}\leq n}H(X_{j_{1}},\cdots,X_{j_{i}})-\sum_{\mbox{\small even
}s\leq n}\sum_{1\leq\ell_{1}<\cdots<\ell_{i}\leq
n}H(X_{\ell_{1}},\cdots,X_{\ell_{i}})$, and $I(X_{1};\cdots;X_{n}|Y)$ denotes
the mutual information of $X_{1},\cdots,X_{n}$ conditional on the outcomes of
$Y=Y_{1}\cdots{}Y_{m}$ and can be defined as
$I(X_{1};\cdots;X_{n})=\sum_{\mbox{\small odd }i\leq n}\sum_{1\leq
j_{1}<\cdots<j_{i}\leq n}H(X_{j_{1}},\cdots,X_{j_{i}}|Y)-\sum_{\mbox{\small
even }s\leq n}\sum_{1\leq\ell_{1}<\cdots<\ell_{i}\leq
n}H(X_{\ell_{1}},\cdots,X_{\ell_{i}}|Y)$.

Figure 4: (Color online) Schematic $n$-independent quantum networks. Here,
each pair of the nodes $\textsf{A}_{1},\cdots,\textsf{A}_{n}$ has not shared
any entanglement. While each node $\textsf{A}_{i}$ may share some entanglement
with the node $\textsf{B}_{j}$.

We firstly show some local operations may generate positive Shannon
information for others as (Appendix G)

$\displaystyle I(X_{1};\cdots;X_{n})=0,$ $\displaystyle
I(X_{1};\cdots;X_{n}|Y)>0,$ (33)

where the joint probability distribution $P_{X_{1}\cdots{}X_{n}Y}$ is obtained
from local measurements on the network $\mathcal{N}_{k}$. This implies
negative Shannon mutual information as

$\displaystyle I(X_{1};\cdots;X_{n};Y)<0$ (34)

Moreover, there are some local operations may generate negative Shannon mutual
information for others as (Appendix G)

$\displaystyle I(X_{1};\cdots;X_{n})=0,$ (35) $\displaystyle
I(X_{1};\cdots;X_{n}|Y)<0.$ (36)

This implies positive Shannon mutual information as

$\displaystyle I(X_{1};\cdots;X_{n};Y)>0,$ (37)

which is different from Eq.(34). Thus the general $k$-independent quantum
network with $k\geq 3$ can generate both negative and positive Shannon mutual
information beyond Result 1 for tripartite chain network even if both have
similar Bell nonlocality Luo2018 . This intrigues new kinds of non-Shannon-
type information for any independent set.

Example 4. One example is the joint distribution in Eq.(10). Another example
is given by

$\displaystyle P_{xyz}=\tilde{\sum}_{i,j}p_{i}q_{j}[x_{i}y_{j}z_{ij}],$ (38)

with three finite-dimensional random variables $X,Y,Z$, where $\\{p_{i}\\}$
and $\\{q_{j}\\}$ are probability distributions. Consider classical
transformation $g:z_{ij}\mapsto x_{i}y_{j}$, it follows

$\displaystyle
P_{xyg(z)}=\tilde{\sum}_{i,j}p_{i}q_{j}[x_{i}y_{j}x_{i}y_{j}]=\tilde{\sum}_{i}p_{i}[x_{i}x_{i}]\tilde{\sum}_{j}q_{j}[y_{j}y_{j}],$
(39)

which raises two independent joint distributions of
$\tilde{\sum}_{i}p_{i}[x_{i}x_{i}]$ and $\tilde{\sum}_{j}q_{j}[y_{j}y_{j}]$.
It is easy to check that the joint distribution $P_{xyg(z)}$ satisfying the
constriction in Eq.(9), i.e., a negative information of $I(X;Y;Z)<0$. From
Result 1, there is a tripartite chain network compatible with the distribution
in Eq.(38). From Eq.(33) the present example can be extended for general star
network which is compatible with the joint distribution

$\displaystyle
P_{x_{1}\cdots{}x_{n}y}=\tilde{\sum}_{i_{1},\cdots,i_{n}}p_{i_{1}}\cdots
p_{i_{n}}[x_{i_{1}}\cdots x_{i_{n}}y_{i_{1}\cdots i_{n}}],$ (40)

with $n+1$ finite-dimensional variables $X_{1},\cdots,X_{n},Y$, where
$\\{p_{i_{j}}\\}$ are probability distributions, $j=1,\cdots,n$.

Example 5. For the triangle network one example is given by Eq.(17). Another
is given by

$\displaystyle
P_{xyz}=\tilde{\sum}_{i,j,k}p_{i}q_{j}r_{k}[x_{ij}y_{jk}z_{ki}]$ (41)

with three finite-dimensional variables $X,Y,Z$, where
$\\{p_{i}\\},\\{q_{j}\\}$ and $\\{r_{k}\\}$ are probability distributions.
Consider local mappings $g_{1}:x_{ij}\mapsto(i,j)$, $g_{2}:y_{jk}\mapsto(j,k)$
and $g_{3}:z_{k,i}\mapsto(k,i)$. It follows a new joint distribution

$\displaystyle
P_{g_{1}(x)g_{2}(y)g_{3}(z)}=\tilde{\sum}_{i,j,k}p_{i}q_{j}r_{k}[ij,jk,ki]=\tilde{\sum}_{i}p_{i}[ii]\tilde{\sum}_{j}q_{j}[jj]\tilde{\sum}_{k}r_{k}[kk]$
(42)

which raises three independent joint distributions of
$\tilde{\sum}_{i}p_{i}[ii]$, $\tilde{\sum}_{j}q_{j}[jj]$ and
$\tilde{\sum}_{k}r_{k}[kk]$. This means the joint distribution in Eq.(42)
satisfy the conditions in Eqs.(16) and (19). From Result 2, there is a
tripartite triangle network consisting of three variables compatible with the
distribution in Eq.(41).

##  III Conclusions

For tripartite Shannon mutual information Results 1 and 2 imply a compatible
chain network for special decomposition of its joint distribution in two
cases. This means the negative Shannon mutual information hide different
network configurations compatible with specific decompositions of joint
distributions. This is different from recent results for featuring higher-
order statistical correlations YSL  which takes use of Euler diagram
corresponding to Shannon information. The main reason is that negative Shannon
mutual information can be featured by using non-Shannon inequalities beyond
Euler diagram. Here, the set bounded by Shannon-type information inequalities
is denoted as polymatroidal region. A general problem is to determine whether
all the polymatroids are entropic. Especially, the negative Shannon
information of three random variables implies the existence of non-entropic
polymatroids on the boundary Zhang1997 ; Matus ; Yeung . This is further
extended for four or more random variables, which even allow unconstrained
non-Shannon-type information inequalities Zhang1997 . A natural problem is
then to explore the Markov compatibility of general non-Shannon-type
information. Another problem is to explore different formations of Shannon-
type information Ay .

In summary, we provided an operational characterization of negative Shannon
mutual information. The main idea is inspired by Bayesian networks. We have
investigated the intrinsic network compatibility of all tripartite joint
distributions. Similar results are proved for its quantum realizations. This
provided a general method for experimentally verifying negative Shannon
information in a device-independent manner. These results should be
interesting in the information theory, deep learning, quantum nonlocality, and
quantum networks.

## Acknowledgements

We are grateful to Jingyun Fan, Yali Mao and Zhengda Li for discussions. This
work was supported by the National Natural Science Foundation of China
(Nos.62172341,61772437), and Shenzhen Institute for Quantum Science and
Engineering, Southern University of Science and Technology (Grant No.
SIQSE202105).

## References

  * (1) C. E. Shannon, A mathematical theory of communication, Bell Syst. Tech. J. 27, 379-423, 623-656 (1948).
  * (2) J. von Neumann, Thermodynamik quantummechanischer Gesamheiten, Gott. Nach. 1, 273-291(1927).
  * (3) E. T. Jaynes, Information theory and statistical mechanics Part I. Phys. Rev. 106, 620-630; ibid, 107, 171-190 (1957).
  * (4) J. Honerkamp, Statistical Physics, Springer, Berlin, 1998.
  * (5) S. Fujishige, Polymatroidal dependence structure of a set of random variables, Info. Contr. 39, 55-72 (1978).
  * (6) Z. Zhang and R. W. Yeung. A non Shannon-type conditional inequality of information quantities, IEEE Trans. Inf. Theory 43, 1982-1986, 1997.
  * (7) J. Pearl, Causality, Models, Reasoning, and Inference, 2nd edition, Cambridge University Press, 2009.
  * (8) A. Einstein, B. Podolsky, N. Rosen, Can quantum mechanical description of physical reality be considered complete? Phys. Rev. 47, 777-780 (1935).
  * (9) M. A. Nielsen and I. L. Chuang, Quantum Computation and Quantum Information, Cambridge University Press, 10th Anniversary Edition, 2010.
  * (10) J. S. Bell, On the Einstein-Podolsky-Rosen paradox, Phys. 1, 195 (1964).
  * (11) M. Zukowski, A. Zeilinger, M. A. Horne, and A. K. Ekert, ”Event-ready-detectors” Bell experiment via entanglement swapping, Phys. Rev. Lett. 71, 4287 (1993).
  * (12) D. Cavalcanti, M. L. Almeida, V. Scarani & A. Acín, Quantum networks reveal quantum nonlocality. Nature Commun. 2, 184 (2011).
  * (13) C. Branciard, N. Gisin and S. Pironio, Characterizing the nonlocal correlations created via entanglement swapping. Phys. Rev. Lett. 104, 170401 (2010).
  * (14) W. Feller, An Introduction to Probability Theory and Its Applications, Vol. 1, J. Wiley & Sons, 3rd dition, New York, 1968.
  * (15) P. Spirtes, N. Glymour, and R. Scheienes, Causation, Prediction, and Search, 2nd ed. MIT Press, 2001.
  * (16) J. Tian and J. Pearl. On the testable implications of causal models with hidden variables, In A. Darwiche and N. Friedman (Eds.), Uncertainty in Artificial Intelligence, Proceedings of the Eighteenth Conference, Morgan Kaufmann: San Francisco, CA, pp.519–527, 2002.
  * (17) C. Kang and J. Tian, Inequality constraints in causal models with hidden variables, In Proceedings of the Seventeenth Annual Conference on Uncertainty in Artificial Intelligence (UAI-06), MIT, Cambridge, MA, pp.233–240, 2006.
  * (18) G. Birkhoff, Tres observaciones sobre el algebra lineal, Univ. Nac. Tucuman Rev. Ser. A 5, 147-150(1946).
  * (19) R. Horodecki, P. Horodecki, M. Horodecki, and K. Horodecki, Quantum entanglement, Rev. Mod. Phys. 81, 865 (2009).
  * (20) M.-O. Renou, E. Bäumer, S. Boreiri, N. Brunner, N. Gisin, and S. Beigi, Genuine quantum nonlocality in the triangle network, Phys. Rev. Lett. 123, 140401 (2019).
  * (21) P. Spirtes, R. Scheines, C. Meek, T. Richardson, C. Glymour, H. Hoijtink and A. Boomsma, TETRAD 3: Tools for Causal Modeling, Program, Psychology Press, 1996.
  * (22) W. Dür, G. Vidal, and J. I. Cirac, Three qubits can be entangled in two inequivalent ways, Phys. Rev. A 62, 062314 (2000).
  * (23) E. Wolfe, R. W. Spekkens, and T. Fritz, The inflation technique for causal inference with latent variables, J. Causal Infer. 7, 20170020 (2019).
  * (24) M. Navascués and E. Wolfe, The inflation technique completely solves the causal compatibility problem, J. Causal Infer. 8, 70-91(2020).
  * (25) D. M. Greenberger, M. A. Horne, and A. Zeilinger, in Bell’s Theorem, Quantum Theory and Conceptions of the Universe, edited by M. Kafatos (Kluwer, Dordrecht, 1989), pp. 69-72.
  * (26) J. L. Kmetz, The Information Processing Theory of Organization Managing Technology Accession in Complex Systems, 1st Edition, Ashgate Pub Ltd, 1998.
  * (27) M.-X. Luo, Computationally efficient nonlinear Bell inequalities for quantum networks, Phys. Rev. Lett. 120, 140402 (2018).
  * (28) R. Yeung, A framework for linear information inequalities, IEEE Trans. Inf. Theory 43, 1924-1934 (1997).
  * (29) F. Matus, Piecewise linear conditional information inequality, IEEE Trans. Inf. Theory 44, 236-238 (2006).
  * (30) N. Ay, A refinement of the common cause principle, Discrete Appl. Math. 157, 2439-2457 (2009).
  * (31) D. Bohm, A Suggested Interpretation of the Quantum Theory in ”Hidden” Variables. I, Phys. Rev. 85, 166 (1952).
  * (32) T. Fritzm Beyond Bell’s theorem I: correlation scenarios. New J. Phys. 14, 103001(2012).
  * (33) M.-O. Renou, Y. Wang, S. Boreiri, S. Beigi, N. Gisin, and N. Brunner, Limits on correlations in networks for quantum and no-signaling resources, Phys. Rev. Lett. 123, 070403 (2019).
  * (34) M.-X. Luo, Network configuration theory for all networks, arXiv:2107.05846, 2021.
  * (35) F. Lad, G. Sanfilippo and G. Agró, Extropy: Complementary Dual of Entropy, Statistical Science 30, 40-58 (2015).
  * (36) V. S. Yepez, R. P. Sagar & H. G. Laguna, Higher-order statistical correlations and mutual information among particles in a quantum well, Few-Body Systems 58, 158 (2017).

##  Appendix A The nonnegative tripartite Shannon mutual information

In this section, we prove $I(X;Y;Z)\geq 0$ if three random variables
$\\{X,Y,Z\\}$ on finite sample spaces consist of a Markov chain, that is,
$X\to Y\to Z$, $X\to Z\to Y$, or $Y\to X\to Z$. The following proof holds for
each case because of the symmetry of $I(X;Y;Z)$, that is,
$I(X;Y;Z)=I(X;Z;Y)=I(Y;X;Z)$.

Assume that $X,Y$ and $Z$ consist of a Markov chain $X\to Y\to Z$. This
implies that $X$ and $Z$ are independent conditional on $Y$. It follows that
$p_{xyz}=p_{x|y}p_{z|y}$, where $P_{xyz}=\tilde{\sum}_{xyz}p_{xyz}[xyz]$ is
the joint distribution of $X,Y$ and $Z$, and $p_{x|y}$ and $p_{z|y}$ are
marginal conditional distributions. This implies

$\displaystyle I(X;Z|Y)$ $\displaystyle=$ $\displaystyle
H(YZ)+H(XZ)-H(Z)-H(XYZ)$ (43) $\displaystyle=$ $\displaystyle 0.$

Combined with the definition of $I(X;Y;Z)$ we have

$\displaystyle I(X;Y;Z)$ $\displaystyle=$ $\displaystyle I(X;Z)-I(X;Z|Y)$ (44)
$\displaystyle=$ $\displaystyle I(X;Z)$ $\displaystyle\geq$ $\displaystyle 0.$

This has proved the result.

##  Appendix B Proof of Result 1

###  B.1 Classical Bayesian networks

Figure 5: (Color online) Simulating a tripartite chain network. (a) Classical
chain network consisting of two latent variables. (b) Chain Bayesian network
for representing the distribution $P_{xyz}$ satisfying the decomposition (48).
Here, $g:\mathcal{Z}\to\mathcal{Z}_{1}\times\mathcal{Z}_{2}$ is classical
post-processing function for any $z\in\mathcal{Z}$. (c) Chain network for
representing the distribution $P_{xyz}$ satisfying the decomposition (47).

The ”hidden variable” (or latent variable) is firstly introduced in Ref.EPR
for arguing the incompleteness of quantum mechanics. These additional
variables were from the assumptions of the causality and locality. Instead,
Bohm Bohm  construct a hidden variable interpretation of elementary quantum
theory. That particular interpretation has indeed a grossly nonlocal
structure, according to the result to be proved by Bell Bell . Especially, the
outcome $A$ of measurement $M_{A}$ is depending on its physical observable and
hidden variable, that is,
$p(M_{A})=\int_{\lambda}A(M_{A},\lambda)\mu(\lambda)d\lambda$, where
$\mu(\lambda)$ is the probability distribution of $\lambda$. Thus the
expectation value of two measurements $M_{A}$ and $M_{B}$ is given by

$\displaystyle P_{h}(M_{A},M_{B})=\int
d\lambda\mu(\lambda)A(M_{A},\lambda)B(M_{B},\lambda).$ (45)

This is generally inconsistent with the quantum mechanical expectation value
$P_{q}(M_{A},M_{B})$ on compose system $\rho_{AB}$ for more than one set of
measurements, that is, $P_{h}(M_{A},M_{B})\not=P_{q}(M_{A},M_{B})$, which is
firstly verified with Bell inequality Bell . Nevertheless, they are consistent
with each other for single set of measurements EPR .

Our goal is to prove Result 1 by using classical chain network, as shown in
Fig.5(a). Specially, this classical realization is rigid which includes two
subcases as follows:

  * C1.

Any distribution $P_{xyz}$ derived from local measurement on classical
networks satisfies

$\displaystyle I(X;Y)=0,I(X;Y|Z)\geq 0,$ (46)

which includes negative and zero information beyond Result 1.

  * C2.

For any distribution $P_{xyz}=\tilde{\sum}_{xyz}p_{xyz}[xyz]$ satisfying the
condition in Eq.(46) there exists local post-processing function
$g:\mathcal{Z}\to\mathcal{Z}_{1}\times\mathcal{Z}_{2}$ satisfying

$\displaystyle P_{xyz}=P_{xyg(z_{1}z_{2})},$ (47)

where $P_{xyz_{1}z_{2}}$ is compatible with a chain network, that is,
$P_{xyz_{1}z_{2}}=P_{xz_{1}}P_{yz_{2}}$,
$P_{xz_{1}}=\tilde{\sum}_{xz_{1}}p_{xz_{1}}[xz_{1}]$ and
$P_{yz_{2}}=\tilde{\sum}_{yz_{2}}p_{yz_{2}}[yz_{2}]$.

The proof is as follows.

Case 1. For a given chain network in Fig.5(a), consider its classical
realization with two independent measurable variables $\lambda_{1}$ and
$\lambda_{2}$. This implies a network decomposition of any joint distribution
$P_{xyz}=\tilde{\sum}_{xyz}p_{xyz}[xyz]$ as

$\displaystyle
p_{xyz}=\int_{\Omega}p(x|\lambda)p(y|\gamma)p(z|\lambda,\gamma)\mu(\lambda)\mu(\gamma)d\lambda{}d\gamma,$
(48)

where $\Omega=\Omega_{1}\times\Omega_{2}$. It is forward to prove that

$\displaystyle p_{xy}$ $\displaystyle=$ $\displaystyle\sum_{z}p_{xyz}$ (49)
$\displaystyle=$
$\displaystyle\sum_{z}\int_{\Omega}p(x|\lambda)p(y|\gamma)p(z|\lambda,\gamma)\mu(\lambda)\mu(\gamma)d\lambda{}d\gamma$
$\displaystyle=$
$\displaystyle\int_{\Omega}p(x|\lambda)p(y|\gamma)\mu(\lambda)\mu(\gamma)d\lambda{}d\gamma$
$\displaystyle=$
$\displaystyle\int_{\Omega_{1}}p(x|\lambda)\mu(\lambda)d\lambda\int_{\Omega_{2}}p(y|\gamma)\mu(\gamma)d\gamma$
$\displaystyle=$ $\displaystyle p_{x}p_{y},$

where $p_{x}=\sum_{yz}p_{xyz}$ and $p_{y}=\sum_{xz}p_{xyz}$. This implies that
the distribution $P_{xyz}$ defined in Eq.(48) satisfies the condition in
Eq.(46).

Case 2. For a specific joint distribution
$P_{xyz}=\tilde{\sum}_{x,y,z}p_{xyz}[xyz]$ with three random variables $X,Y$
and $Z$ on finite sample spaces, assume that $P_{xyz}$ satisfies
$I(X;Y)=I(X;Y|Z)=0$, that is, $X$ and $Y$ are both independent and conditional
independent on $Z$. In this case, we have Markovian decomposition of $p_{xyz}$
as

$\displaystyle p_{xyz}=p_{x}p_{y}p_{z|xy},\forall,x,y,z,$ (50) $\displaystyle
p_{xyz}=p_{z}p_{x|z}p_{y|z},\forall,x,y,z.$ (51)

From Algorithm 1, we can obtain two chain networks: one represents the
distribution $P_{xyz}$ with two edges from $X$ to $Z$ and $Y$ to $Z$, the
other represents the distribution $P_{xyz}$ with two edges from $Z$ to $X$ and
$Z$ to $Y$. Both imply an undirected Bayesian network. This has completed the
proof.

In what follows, we prove the result by considering $P_{xyz}$ satisfies
$I(X;Y)=0$ and $I(X;Y|Z)>0$. This implies a Markovian decomposition of
Eq.(50). From Algorithm 1, we can obtain a chain network consisting of two
edges for representing the distribution $P_{xyz}$, one is from $X$ to $Z$ and
the other is $Y$ to $Z$, as shown in Fig.5(b), that is, two independent
variables $X$ and $Y$ are the predecessors of $Z$.

We will prove the converse-directed chain Bayesian network shown in Fig.5(c)
for representing the distribution $P_{xyz}$ satisfying Eq.(47). The
distribution $P_{xyz}$ is rewritten into

$\displaystyle P_{xyz}=\tilde{\sum}_{xy}p_{xy}[xy]\times Q_{xy},$ (52)

where $p_{xy}=\sum_{z}p_{xyz}$, and $Q_{xy}$ denotes the probability
distribution of $Z$ conditional on $X=x$ and $Y=y$ and is defined by

$\displaystyle Q_{xy}=\tilde{\sum}_{z}p_{z|xy}[z]$ (53)

with $p_{z|xy}=\frac{p_{xyz}}{p_{xy}}$.

Consider a joint distribution from the chain network as

$\displaystyle P_{xyz_{1}z_{2}}$ $\displaystyle=$ $\displaystyle
P_{xz_{1}}P_{yz_{2}}$ (54) $\displaystyle=$
$\displaystyle\tilde{\sum}_{x=z_{1},y=z_{2}}p_{x}p_{y}[xyz_{1}z_{2}],$

where $P_{xz_{1}}=\tilde{\sum}_{x,z_{1}}p_{xz_{1}}[xz_{1}]$ and
$P_{yz_{2}}=\tilde{\sum}_{y,z_{2}}p_{yz_{2}}[yz_{2}]$. From the condition in
Eq.(46), it follows from Eq.(52) that

$\displaystyle P_{xyz}=\tilde{\sum}_{xy}p_{x}p_{y}[xy]\times Q_{xy}.$ (55)

Hence, it is sufficient to show that the distribution in Eq.(55) can be
obtained from the distribution in Eq.(54) using local classical post-
processing of $Z_{1}$ and $Z_{2}$. Note that $Q_{xy}$ in Eq.(53) is a
probability distribution, that is
$\sum_{z}\frac{p_{xyz}}{p_{xy}}=\sum_{z}p_{z|xy}=1$. Define the following
classical mapping

$\displaystyle g:[z_{1}=x,z_{2}=y]\mapsto Q_{xy},\forall x,y,$ (56)

that is, the output of $z_{1}=x$ and $z_{2}=y$ is mapping into a probability
distribution $Q_{xy}$. $g$ is a classical post-processing function of
probability. In fact, define a nonnegative matrix as

$\displaystyle S_{xy}=\left(\begin{array}[]{cccc}\mathrm{g}_{1,1}(\vec{v})\\\
\vdots\\\ \mathrm{g}_{x,y}(\vec{v})\\\ \vdots\\\
\mathrm{g}_{n,n}(\vec{v})\end{array}\right),$ (62)

where $\vec{v}=(p_{z=1|xy},\cdots,p_{z=n|xy},0,\cdots,0)$ denotes
$n^{2}$-dimensional nonnegative vector associated with the probability
distribution $Q_{xy}$, $\mathrm{g}_{i,j}$ denote $n^{2}$ different
permutations of vector $\vec{v}$ such that $\mathrm{g}_{x,y}(\vec{v})=\vec{v}$
for each pair of $x$ and $y$. Here, we assume there are $n$ samples of $x,y$.
This implies

$\displaystyle\vec{e}_{xy}\cdot{}S_{xy}=\vec{v},\forall x,y,$ (63)

where $\vec{e}_{xy}$ denotes the $n^{2}$-dimensional unit vector with the
$xy$-th component being $1$ and others being $0$. It is easy to prove that
$S_{xy}$ are double stochastic matrices, that is, the entries are non-
negative, and each row and column sums to 1. From Birkhoff’s theorem Birk , it
shows that the set of doubly stochastic matrices is a convex set whose extreme
points are the permutation matrices. This means that all the transformations
$S_{xy}$ can be realized in statistics. Moreover, all the outputs $\\{xy\\}$
are distinguishable. This means that each $S_{xy}$ can be locally performed
for each pair of outputs $x$ and $y$. So, we have proved that the distribution
in Eq.(55) can be obtained from the distribution in Eq.(54) by using local
classical post-processing of $Z_{1}$ and $Z_{2}$. From Algorithm 1, we obtain
the Bayesian network in Fig.5(c).

In classical realization of Bayesian network, since the variables $x$ and $y$
are independent, from Eq.(7) in the maintext there are two independent
measurable variables $\lambda_{1}$ and $\lambda_{2}$ such that

$\displaystyle p_{x}=\int_{\Omega_{1}}p(x|\lambda)\mu(\lambda)d\lambda,$ (64)
$\displaystyle p_{y}=\int_{\Omega_{2}}p(y|\gamma)\mu(\gamma)d\gamma,$ (65)

where $p(x|\lambda)$ and $p(y|\gamma)$ are characteristic functions, that is,
$p(x|\lambda),p(y|\gamma)\in\\{0,1\\}$. Note that the function in Eq.(56) is
measurable. Combined with Eqs.(4) and (47) we get from the independence of $X$
and $Y$ that

$\displaystyle
p_{xyz}=\int_{\Omega}p(x|\lambda)p(y|\gamma)g(p((z_{1}z_{2})|\lambda,\gamma))\mu(\lambda)d\lambda\mu(\gamma)d\gamma,$
(66)

where $\Omega=\Omega_{1}\times\Omega_{2}$, as shown in Fig.5(a). This has
completed the proof.

###  B.2 Quantum chain networks

Our goal in this section is to prove Result 1 by using quantum chain network,
as shown in Fig.2(b). Specially, the quantum realization is also rigid, that
is, any distribution $P_{xyz}$ derived from local measurements on this quantum
network satisfies Eq.(46). Moreover, for any distribution
$P_{xyz}=\tilde{\sum}_{xyz}p_{xyz}[xyz]$ satisfy the condition in Eq.(46)
there exists a quantum chain network consisting of two bipartite states
$\rho_{AC_{1}}$ on Hilbert space $\mathcal{H}_{A}\otimes\mathcal{H}_{C_{1}}$
and $\varrho_{BC_{2}}$ on Hilbert space
$\mathcal{H}_{B}\otimes\mathcal{H}_{C_{2}}$, and local measurements
$\\{M_{x}^{(A)}\\}$, $\\{M_{y}^{(B)}\\}$ and $\\{M_{z}^{(C_{1}C_{2})}\\}$ such
that

$\displaystyle p_{xyz}={\rm Tr}[(M_{x}^{(A)}\otimes M_{y}^{(B)}\otimes
M_{z}^{(C_{1}C_{2})})\rho_{AC_{1}}\otimes\varrho_{BC_{2}}].$ (67)

In fact, consider a quantum network consisting of any bipartite states
$\rho_{AC_{1}}$ on Hilbert space $\mathcal{H}_{A}\otimes\mathcal{H}_{C_{1}}$
and $\varrho_{BC_{2}}$ on Hilbert space
$\mathcal{H}_{B}\otimes\mathcal{H}_{C_{2}}$. For any POVM measurements
$\\{M_{x}^{(A)}\\}$, $\\{M_{y}^{(B)}\\}$ and $\\{M_{z}^{(C_{1}C_{2})}\\}$, we
have

$\displaystyle p_{xy}={\rm Tr}[(M_{x}^{(A)}\otimes
M_{y}^{(B)})\rho_{A}\otimes\varrho_{B}],$ (68)

where $\rho_{A}$ and $\varrho_{B}$ are reduced density matrices of the systems
$A$ and $B$, respectively. It follows that $p_{xy}=p_{x}p_{y}$ which satisfies
the condition (46).

In fact, the proof in Appendix B.1 above has suggested a proof by using mixed
states for quantum networks, where we can define
$\rho_{AC_{1}}=\sum_{x,z_{1}}p_{xz_{1}}|xz_{1}\rangle\langle xz_{1}|$ and
$\rho_{BC_{2}}=\sum_{y,z_{2}}p_{yz_{2}}|yz_{2}\rangle\langle yz_{2}|$. Each
particle is measured under the computation basis $\\{|x\rangle\\}$,
$\\{|y\rangle\\}$ or $\\{|z_{i}\rangle\\}$. The output of $z_{1}z_{2}$ is
transformed into $z$ by using classically post-proposing function in Eq. (56).
Note that the post-proposing function is depending of the output $z_{1}z_{2}$.
This generally cannot be realized by local quantum measurements on $C_{1}$ and
$C_{2}$. Instead, in what follow we consider a quantum network consisting of
two entangled states. With new quantum realization, any distribution
satisfying the condition in Eq.(46) can be generated by local quantum
measurements without classical post-progressing.

For a given distribution $P_{xyz}=\tilde{\sum}_{xyz}p_{xyz}[xyz]$ satisfying
the condition in Eq. (46), we get marginal distributions
$P_{xy}=\tilde{\sum}_{xy}p_{xy}[xy]$, $P_{x}=\tilde{\sum}_{x}p_{x}[x]$ and
$P_{y}=\tilde{\sum}_{y}p_{y}[y]$, where $p_{xy}=\sum_{z}p_{xyz}$,
$p_{x}=\sum_{y,z}p_{xyz}$, and $p_{y}=\sum_{x,z}p_{xyz}$. Define a general
joint state

$\displaystyle|\Phi\rangle_{ABC}=\sum_{x,y,z}\sqrt{p_{xyz}}|x,y,z\rangle$ (69)

from the joint distribution $P_{xyz}$. It follows that

$\displaystyle p_{xyz}={\rm Tr}[(|x\rangle\langle x|\otimes|y\rangle\langle
y|\otimes|z\rangle\langle z|)|\Phi\rangle\langle\Phi|],$ (70)

that is, the quantum probability under local projection measurement is
compatible with the given distribution $P_{xyz}$.

From two distributions $P_{x}$ and $P_{y}$, define two entangled states as

$\displaystyle|\phi_{1}\rangle_{A_{1}C_{1}}=\sum_{x}\sqrt{p_{x}}|x,x\rangle,$
(71)
$\displaystyle|\phi_{2}\rangle_{B_{1}C_{2}}=\sum_{y}\sqrt{p_{y}}|y,y\rangle.$
(72)

It follows that

$\displaystyle|\phi_{1}\rangle_{A_{1}C_{1}}|\phi_{2}\rangle_{B_{1}C_{2}}$
$\displaystyle=$
$\displaystyle\sum_{x,y}\sqrt{p_{x}p_{y}}|x,y\rangle_{A^{\prime}B^{\prime}}|x,y\rangle_{C_{1}C_{2}}$
(73) $\displaystyle=$
$\displaystyle\sum_{x,y}\sqrt{p_{xy}}|x,y\rangle_{A^{\prime}B^{\prime}}|x,y\rangle_{C_{1}C_{2}}$

from the equality of $p_{xy}=p_{x}p_{y}$ because of the condition in Eq.(46).
The reduced density matrix of the joint system $A^{\prime}B^{\prime}$ is given
by

$\displaystyle\rho_{A^{\prime}B^{\prime}}=\rho_{A^{\prime}}\otimes\rho_{B^{\prime}},$
(74)

where $\rho_{A^{\prime}}=\sum_{x}p_{x}|x\rangle\langle{}x|$ and
$\rho_{B^{\prime}}=\sum_{y}p_{y}|y\rangle\langle{}y|$.

Now, from Eq.(73) we get the Schmidt decomposition of
$|\phi_{1}\rangle_{A_{1}C_{1}}|\phi_{2}\rangle_{B_{1}C_{2}}$ as

$\displaystyle|\phi_{1}\rangle_{A_{1}C_{1}}|\phi_{2}\rangle_{B_{1}C_{2}}=\sum_{x,y}\sqrt{p_{xy}}|x,y\rangle_{A^{\prime}B^{\prime}}|x,y\rangle_{C_{1}C_{2}},$
(75)

where $\\{|x,y\rangle\\}$ are orthogonal states from the distinguishability of
samples $\\{x\\}$ and $\\{y\\}$.

From Eq.(69) we have

$\displaystyle|\Phi\rangle_{ABC}=\sum_{x,y}\sqrt{p_{xy}}|xy\rangle_{AB}|\varphi_{xy}\rangle_{C},$
(76)

where
$|\varphi_{xy}\rangle_{C}=\sum_{z}\sqrt{\frac{p_{xyz}}{p_{xy}}}|z\rangle$
denote some states of the system $Z$. We complete the proof with two subcases,
that is, $\\{|\varphi_{xy}\rangle_{C}\\}$ are orthogonal states or not.

  * •

Assume that $|\varphi_{xy}\rangle_{C}$ are orthogonal states. In this case,
there is a local unitary mapping given by
$U:|\varphi_{xy}\rangle_{C}\mapsto|xy=z\rangle_{C_{1}C_{2}}$ because Eq.(76)
is the Schmidt decomposition of $|\Phi\rangle$. This implies

$\displaystyle(\openone_{AB}\otimes
U_{C}^{-1})|\phi_{1}\rangle_{A_{1}C_{1}}|\phi_{2}\rangle_{B_{1}C_{2}}=|\Phi\rangle_{ABC}.$
(77)

By performing local projection measurements under bases
$\\{M_{x}=|x\rangle\langle x|_{A_{1}}\\}$, $\\{M_{y}=|y\rangle\langle
y|_{B_{1}}\\}$ and
$\\{M_{\varphi_{xy}}=|\varphi_{xy}\rangle\langle\varphi_{xy}|_{C}\\}$, we get
a joint distribution $P_{xyz}=\tilde{\sum}_{xyz}q_{xyz}[xyz]$ with

$\displaystyle q_{xyz}$ $\displaystyle=$ $\displaystyle{\rm Tr}[(M_{x}\otimes
M_{y}\otimes M_{\varphi_{xy}})\rho_{1}\otimes\rho_{2}]$ (78) $\displaystyle=$
$\displaystyle{\rm Tr}[(M_{x}\otimes M_{y}\otimes M_{z})(\openone_{AB}\otimes
U_{C}^{-1})(\rho_{1}\otimes\rho_{2})]$ $\displaystyle=$ $\displaystyle
p_{xyz}$

from Eqs.(70) and (77), where $\rho_{i}=|\phi_{i}\rangle\langle\phi_{i}|$, and
$\\{\Pi_{z}=|z\rangle\langle z|_{C_{1}C_{2}}\\}$. This has completed the
proof.

  * •

Assume that $|\varphi_{xy}\rangle_{C}$ are not orthogonal states. In this
case, we define local transformations $U_{xy}$ on the systems $C_{1}C_{2}$ and
$C^{\prime}$ as follows

$\displaystyle
U_{xy}:|xy\rangle_{C_{1}C_{2}}|0\rangle_{C^{\prime}}\mapsto|xy\rangle_{C_{1}C_{2}}|\varphi_{xy}\rangle_{C^{\prime}}$
(79)

for any $x,y$, where $C^{\prime}$ is an auxiliary system in the state
$|0\rangle$. After these local operations being performed on the joint system
$C_{1}C_{2}$, the total state of chain quantum network is changed into

$\displaystyle|\Psi\rangle_{A_{1}B_{1}C_{1}C_{2}C^{\prime}}:$ $\displaystyle=$
$\displaystyle(\openone_{A_{1}B_{1}}\otimes_{xy}U_{xy})|\phi_{1}\rangle_{A_{1}C_{1}}|\phi_{2}\rangle_{B_{1}C_{2}}|0\rangle_{C^{\prime}}$
(80) $\displaystyle=$
$\displaystyle\sum_{x,y}\sqrt{p_{xy}}|xy\rangle_{A_{1}B_{1}}|xy\rangle_{C_{1}C_{2}}|\varphi_{xy}\rangle_{C^{\prime}}.$

By performing local measurements under projections $\\{M_{x}=|x\rangle\langle
x|_{A_{1}}\\}$, $\\{M_{y}=|y\rangle\langle y|_{B_{1}}\\}$,
$\\{M_{xy}=|xy\rangle\langle xy|_{C_{1}C_{2}}\\}$ and
$\\{M_{\varphi_{xy}}=|\varphi_{xy}\rangle\langle\varphi_{xy}|_{C}\\}$, we get
a joint distribution $P_{xyz}=\tilde{\sum}_{xyz}q_{xyz}[xyz]$ with

$\displaystyle q_{xyz}$ $\displaystyle=$ $\displaystyle{\rm Tr}[(M_{x}\otimes
M_{y}\otimes M_{xy}\otimes
M_{\varphi_{xy}})\rho_{1}\otimes\rho_{2}\otimes|0\rangle\langle
0|_{C^{\prime}}]$ (81) $\displaystyle=$ $\displaystyle{\rm Tr}[(M_{x}\otimes
M_{y}\otimes M_{xy}\otimes M_{z})(\openone_{A_{1}B_{1}}\otimes_{xy}U_{xy})$
$\displaystyle\times(\rho_{1}\otimes\rho_{2}\otimes|0\rangle\langle
0|_{C^{\prime}})]$ $\displaystyle=$ $\displaystyle{\rm Tr}[(M_{x}\otimes
M_{y}\otimes M_{xy}\otimes M_{z})|\Psi\rangle\langle\Psi|]$ $\displaystyle=$
$\displaystyle p_{xyz}$

from Eqs.(70) and (80). This has completed the proof.

##  Appendix C Proofs of Inequalities (19)-(23)

For a triangle network consisting of three independent random variables $X,Y$
and $Z$ on finite sample spaces the joint distribution satisfies the following
inequality Fritz  as

$\displaystyle H(X)+H(Y)+H(Z)\leq H(X,Y)+H(Y,Z).$ (82)

Combining with the definition of $I(X;Y;Z)$ it implies that

$\displaystyle I(X;Y;Z)$ $\displaystyle=$ $\displaystyle
H(X)+H(Y)+H(Z)+H(X,Y,Z)$ (83) $\displaystyle-H(X,Y)-H(Y,Z)-H(X,Z)$
$\displaystyle\leq$ $\displaystyle H(X,Y,Z)-H(X,Z)$ $\displaystyle=$
$\displaystyle H(Y|X,Z).$

Similarly, by considering the cyclic permutation of three random variables in
the inequality (90), it follows that

$\displaystyle I(X;Y;Z)$ $\displaystyle\leq$ $\displaystyle H(X|Y,Z),$ (84)
$\displaystyle I(X;Y;Z)$ $\displaystyle\leq$ $\displaystyle H(Z|X,Y).$ (85)

This implies the upper bound in the inequality (19).

Now, we prove the lower bound of triangle mutual information by using the
Finner inequality Marc2019a  of

$\displaystyle p_{xyz}\leq\sqrt{p_{x}p_{y}p_{z}},$ (86)

where $P_{xyz}=\tilde{\sum}_{x,y,z}p_{xyz}[xyz]$ is a given joint distribution
derived from triangle network consisting of three independent variables
$\lambda,\gamma$ and $\eta$.
$P_{x}=\tilde{\sum}_{x}p_{x}[x],P_{y}=\tilde{\sum}_{y}p_{y}[y]$ and
$P_{z}=\tilde{\sum}_{z}p_{z}[z]$ are marginal distributions. The inequality
(86) implies

$\displaystyle H(X,Y,Z)$ $\displaystyle=$
$\displaystyle\sum_{x,y,z}p_{xyz}\log\frac{1}{p_{xyz}}$ (87)
$\displaystyle\geq$
$\displaystyle\sum_{x,y,z}p_{xyz}\log\frac{1}{\sqrt{p_{x}p_{y}p_{z}}}$
$\displaystyle=$
$\displaystyle\frac{1}{2}\sum_{x,y,z}p_{xyz}(\log\frac{1}{p_{x}}+\log\frac{1}{p_{y}}+\log\frac{1}{p_{z}})$
$\displaystyle=$
$\displaystyle\frac{1}{2}(\sum_{x}p_{x}\log\frac{1}{p_{x}}+\sum_{y}p_{y}\log\frac{1}{p_{y}}+\sum_{z}p_{z}\log\frac{1}{p_{z}})$
$\displaystyle=$ $\displaystyle\frac{1}{2}(H(X)+H(Y)+H(Z))$

from the increasing function of $f(x)=\log x$. Combining with the definition
of $I(X;Y;Z)$, it follows the inequality (20).

For a triangle network consisting of one variable $\lambda$, its joint
distribution $P_{xyz}$ satisfies as

$\displaystyle I(X;Y;Z)$ $\displaystyle\leq$ $\displaystyle 4H(X,Y,Z)-H(X,Y)$
(88) $\displaystyle-H(X,Z)-H(Y,Z)$

and

$\displaystyle I(X;Y;Z)$ $\displaystyle>$
$\displaystyle\frac{4}{3}(H(X)+H(Y)+H(Z))$ (89)
$\displaystyle-H(X,Y)-H(X,Z)-H(Y,Z)$

In fact, we have recently proved Luo2021  that

$\displaystyle p_{xyz}\leq(p_{x}p_{y}p_{z})^{\frac{1}{3}}$ (90)

for any $p_{xyz}$. Similar to the inequality (87) we have

$\displaystyle H(X,Y,Z)$ $\displaystyle\leq$
$\displaystyle\frac{1}{3}(H(X)+H(Y)+H(Z))$ (91)

This implies the inequalities (22) and (23) from the definition of $I(X;Y;Z)$.

##  Appendix D Negative Shannon information of the W-type distribution

Consider the following W-type distribution

$\displaystyle P_{xyz}=a[001]+b[010]+c[100]$ (92)

with $a+b+c=1$ and $a,b,c\geq 0$. It yields to marginal distributions as

$\displaystyle P_{x}$ $\displaystyle=$ $\displaystyle(1-c)[0]+c[1]$
$\displaystyle P_{y}$ $\displaystyle=$ $\displaystyle(1-b)[0]+b[1]$
$\displaystyle P_{z}$ $\displaystyle=$ $\displaystyle(1-a)[0]+a[1]$
$\displaystyle P_{xy}$ $\displaystyle=$ $\displaystyle a[00]+b[01]+c[10]$
$\displaystyle P_{yz}$ $\displaystyle=$ $\displaystyle c[00]+a[01]+b[10]$
$\displaystyle P_{xz}$ $\displaystyle=$ $\displaystyle b[00]+a[01]+c[10]$ (93)

From the definitions of Shannon entropy in Eq.(1) we get

$\displaystyle H(X)$ $\displaystyle=$
$\displaystyle-c\log_{2}c-(1-c)\log_{2}(1-c),$ $\displaystyle H(Y)$
$\displaystyle=$ $\displaystyle-b\log_{2}b-(1-b)\log_{2}(1-b),$ $\displaystyle
H(Z)$ $\displaystyle=$ $\displaystyle-a\log_{2}a-(1-a)\log_{2}(1-a),$
$\displaystyle H(X,Y)$ $\displaystyle=$ $\displaystyle H(X,Z)=H(Y,Z)=H(X,Y,Z)$
(94) $\displaystyle=$ $\displaystyle-a\log_{2}a-b\log_{2}b-c\log_{2}c$

This implies Shannon mutual information as

$\displaystyle I_{w}(X;Y;Z)$ $\displaystyle=$ $\displaystyle
H(X)+H(Y)+H(Z)+H(X,Y,Z)$ (95) $\displaystyle-H(X,Y)-H(X,Z)-H(Y,Z)$
$\displaystyle=$ $\displaystyle a\log_{2}a+b\log_{2}b+c\log_{2}c$
$\displaystyle-(1-a)\log_{2}(1-a)-(1-b)\log_{2}(1-b)$
$\displaystyle-(1-c)\log_{2}(1-c)$ $\displaystyle<$ $\displaystyle 0$

from (Dual, , Appendix A). Here, the total Shannon mutual information of the
W-type distribution is shown in Fig.S6.

Figure 6: (Color online) The total mutual information $I_{w}(X;Y;Z)$ of the
W-type distribution $P_{xyz}=a[001]+b[010]+c[100]$. Here,
$a=\cos^{2}\theta_{1},b=\sin^{2}\theta_{1}\cos^{2}\theta_{2}$ and
$c=\sin^{2}\theta_{1}\sin^{2}\theta_{2}$,
$\theta_{1},\theta_{2}\in(0,\frac{\pi}{2})$.

Moreover, consider classical transformations of the W-type distribution in
Eq.(92) with local doubly stochastic matrices defined by

$\displaystyle
U_{x}=\left(\begin{array}[]{ccc}\cos^{2}\gamma_{1}&\sin^{2}\gamma_{1}\\\
\sin^{2}\gamma_{1}&\cos^{2}\gamma_{1}\end{array}\right),$ (98) $\displaystyle
U_{y}=\left(\begin{array}[]{ccc}\cos^{2}\gamma_{2}&\sin^{2}\gamma_{2}\\\
\sin^{2}\gamma_{2}&\cos^{2}\gamma_{2}\end{array}\right),$ (101) $\displaystyle
U_{z}=\left(\begin{array}[]{ccc}\cos^{2}\gamma_{3}&\sin^{2}\gamma_{3}\\\
\sin^{2}\gamma_{3}&\cos^{2}\gamma_{3}\end{array}\right)$ (104)

Define two quantities

$\displaystyle
I_{1}(X;Y;Z)=\max_{\gamma_{1},\gamma_{2},\gamma_{3}}\\{I(X;Y;Z)\\},$ (105)
$\displaystyle
I_{2}(X;Y;Z)=\min_{\gamma_{1},\gamma_{2},\gamma_{3}}\\{I(X;Y;Z)\\},$ (106)

Interestingly, assisted by numeric simulations the maximal Shannon mutual
information $I_{1}(X;Y;Z)$ over all probabilistic transformations in
Eqs.(98)-(104) is almost constant for any $a,b,c$, that is,
$I_{1}(X;Y;Z)\approx 0.5307$. Meanwhile, the W-type distribution in Eq.(92)
achieves the minimal Shannon mutual information according to the local
probabilistic transformations in Eqs.(98)-(104), that is,

$\displaystyle I_{w}(X;Y;Z)\approx I_{2}(X;Y;Z)$ (107)

up to the error scale of $10^{-7}$.

##  Appendix E Generalized W-type distributions in Eqs.(27) and (28)

In this section we firstly prove how to verify generalized W-type
distributions in Eqs.(27) and (28) by using the network inflation Blou . And
then, we prove the negative Shannon mutual information for these
distributions.

###  E.1 Witnessing generalized W-type distribution

In this section we prove that any one of the following W-type distributions

$\displaystyle P_{xyz}=a[001]+b[010]+c[100]+d[011]$ (108) $\displaystyle
P_{xyz}=a[001]+b[010]+c[100]+d[101]$ (109) $\displaystyle
P_{xyz}=a[001]+b[010]+c[100]+d[110]$ (110) $\displaystyle
P_{xyz}=a[010]+b[011]+c[100]+d[101]$ (111) $\displaystyle
P_{xyz}=a[001]+b[010]+c[101]+d[110]$ (112) $\displaystyle
P_{xyz}=a[001]+b[011]+c[100]+d[110]$ (113)

is compatible with tripartite network consisting of one variable for any
$a,b,c,d\in(0,1)$ and $a+b+c+d=1$ using the second-order inflation method Blou
. Here, we only consider the distributions in Eqs.(108) and (111) while
similar proof holds for other distributions from the permutations of three
variables.

Case 1. We prove the distribution $P_{xyz}$ in Eq.(108) which is compatible
with the tripartite network consisting of three latent variables, as shown in
Fig.3(a).

From Eq.(108) we get the marginal distributions as follows

$\displaystyle P_{xy}$ $\displaystyle=$ $\displaystyle a[00]+c[10]+(b+d)[01],$
(114) $\displaystyle P_{yz}$ $\displaystyle=$ $\displaystyle
c[00]+b[10]+a[01]+d[11],$ (115) $\displaystyle P_{xz}$ $\displaystyle=$
$\displaystyle c[10]+b[00]+(a+d)[01],$ (116) $\displaystyle P_{x}$
$\displaystyle=$ $\displaystyle(b+a+d)[0]+c[1],$ (117) $\displaystyle P_{y}$
$\displaystyle=$ $\displaystyle(a+c)[0]+(b+d)[1],$ (118) $\displaystyle P_{z}$
$\displaystyle=$ $\displaystyle(b+c)[0]+(a+d)[1]$ (119)

for any $a,b,c\in(0,1)$ and $a+b+c+d=1$. Suppose that $P_{xyz}$ is compatible
with the triangle network consisting of random variables $X,Y$ and $Z$
(generated from latent variables $\lambda,\gamma$ and $\eta$), as shown in
Fig.3(a). A second-order spiral inflation is shown in Fig.3(b). Consider the
first-order inflation network consisting of random variables $X_{1},Y_{1}$ and
$Z_{1}$ (generated from latent variables $\lambda_{1},\gamma_{1}$ and
$\eta_{1}$). From Lemma 4 Blou , it follows all the marginal distributions of
$\\{ABC\\},\\{XZ_{1}\\},\\{Y_{1}Z\\},\\{X_{1}Y\\},\\{X_{1}\\},\\{Y_{1}\\}$ and
$\\{Z_{1}\\}$ which have the same marginal distribution with Eqs.(114)-(119)
should be compatible with the first-order inflation network in Fig.3(b), that
is,

$\displaystyle P_{xz_{1}}$ $\displaystyle=$ $\displaystyle
c[10]+b[00]+(a+d)[01],$ (120) $\displaystyle P_{y_{1}z}$ $\displaystyle=$
$\displaystyle c[00]+b[10]+a[01]+d[11],$ (121) $\displaystyle P_{x_{1}y}$
$\displaystyle=$ $\displaystyle a[00]+c[10]+(b+d)[01],$ (122) $\displaystyle
P_{x_{1}}$ $\displaystyle=$ $\displaystyle
P_{x},P_{y_{2}}=P_{y},P_{z_{1}}=P_{z}$ (123)

From Eq.(120) it implies that $X=0$ whenever $Z_{1}=1$. Similarly, we get
$Y=0$ whenever $X_{1}=1$ from Eq.(122), that is,

$\displaystyle{\rm Pr}[X=0|Z_{1}=1]$ $\displaystyle=$ $\displaystyle{\rm
Pr}[Y=0|X_{1}=1]=1$ (124)

Consider the second-order inflation network consisting of random variables
$X_{2},Y_{2}$ and $Z_{2}$ (generated from latent variables
$\lambda_{2},\gamma_{2}$ and $\eta_{2}$). From Lemma 4 Blou , it follows all
the marginal distributions of
$\\{XYZ\\},\\{YZ_{2}\\},\\{XY_{2}\\},\\{X_{2}Z\\},\\{X_{2}\\},\\{Y_{2}\\}$ and
$\\{Z_{2}\\}$ which have the same marginal distribution with Eqs.(114)-(119)
should be compatible with the second-order inflation network in Fig.3(b), that
is,

$\displaystyle P_{yz_{2}}$ $\displaystyle=$ $\displaystyle
c[00]+b[10]+a[01]+d[11],$ (125) $\displaystyle P_{xy_{2}}$ $\displaystyle=$
$\displaystyle a[00]+c[10]+(b+d)[01],$ (126) $\displaystyle P_{x_{2}z}$
$\displaystyle=$ $\displaystyle c[10]+b[00]+(a+d)[01],$ (127) $\displaystyle
P_{x_{2}}$ $\displaystyle=$ $\displaystyle
P_{x},P_{y_{2}}=P_{y},P_{z_{2}}=P_{z}$ (128)

From Eq.(126) it implies that $X=0$ whenever $Y_{2}=1$. Similarly, we get
$Z=0$ whenever $X_{2}=1$ from Eq.(127), that is,

$\displaystyle{\rm Pr}[X=0|Y_{2}=1]$ $\displaystyle=$ $\displaystyle{\rm
Pr}[Z=0|X_{2}=1]=1$ (129)

From Fig.3(b) the inflation shows the random variables
$X_{1},Y_{1},Z_{1},X_{2},Y_{2},Z_{2}$ should be marginally independent in any
compatible distribution. From Eqs.(119) and (123) it follows that

$\displaystyle{\rm Pr}[X_{1}=Z_{1}=X_{2}=Y_{2}=1]\not=0$ (130)

Combined with Eqs.(124) and (129), it follows that

$\displaystyle{\rm Pr}[X=Y=Z=0]\not=0$ (131)

which is contradicted to Eq.(108). This has proved that the distribution in
Eq.(108) is incompatible with the tripartite network in Fig.3(a).

Case 2. We prove the distribution in Eq. (111) is compatible with the
tripartite network consisting of three latent variables, as shown in Fig.3(a).

From Eq.(111) we get the marginal distributions as follows

$\displaystyle P_{xy}$ $\displaystyle=$ $\displaystyle(a+b)[01]+(c+d)[10],$
(132) $\displaystyle P_{yz}$ $\displaystyle=$ $\displaystyle
c[00]+a[01]+d[10]+b[11],$ (133) $\displaystyle P_{xz}$ $\displaystyle=$
$\displaystyle a[00]+b[01]+c[10]+d[11],$ (134) $\displaystyle P_{x}$
$\displaystyle=$ $\displaystyle(a+b)[0]+(c+d)[1],$ (135) $\displaystyle P_{y}$
$\displaystyle=$ $\displaystyle(c+d)[0]+(a+b)[1],$ (136) $\displaystyle P_{z}$
$\displaystyle=$ $\displaystyle(a+c)[0]+(b+d)[1]$ (137)

for any $a,b,c,d\in(0,1)$ and $a+b+c+d=1$. Suppose that $P_{xyz}$ is
compatible with the triangle network consisting of three latent variables, as
shown in Fig.3(a). From Lemma 4 Blou  it follows all the marginal
distributions of sets
$\\{ABC\\},\\{XZ_{1}\\},\\{Y_{1}Z\\},\\{X_{1}Y\\},\\{X_{1}\\},\\{Y_{1}\\}$ and
$\\{Z_{1}\\}$ which have the same marginal distribution with Eqs.(132)-(137)
should be compatible with the first-order inflation network in Fig.3(b), that
is,

$\displaystyle P_{y_{1}z}$ $\displaystyle=$ $\displaystyle
c[00]+a[01]+d[10]+b[11],$ (138) $\displaystyle P_{x_{1}y}$ $\displaystyle=$
$\displaystyle(a+b)[01]+(c+d)[10],$ (139) $\displaystyle P_{xz_{1}}$
$\displaystyle=$ $\displaystyle a[00]+b[01]+c[10]+d[11],$ (140) $\displaystyle
P_{x_{1}}$ $\displaystyle=$ $\displaystyle
P_{x},P_{y_{1}}=P_{y},P_{z_{1}}=P_{z}.$ (141)

From Eq.(139) it implies that $Y=0$ whenever $X_{1}=1$, that is,

$\displaystyle{\rm Pr}[Y=0|X_{1}=1]=1.$ (142)

Moreover, all the marginal distributions of
$\\{XYZ\\},\\{YZ_{2}\\},\\{XY_{2}\\},\\{X_{2}Z\\},\\{X_{2}\\},\\{Y_{2}\\}$,
and $\\{Z_{2}\\}$ which have the same marginal distribution with
Eqs.(132)-(137) should be compatible with the second-order inflation network
in Fig.3(b), that is,

$\displaystyle P_{yz_{2}}$ $\displaystyle=$ $\displaystyle
c[00]+a[01]+d[10]+b[11],$ (143) $\displaystyle P_{xy_{2}}$ $\displaystyle=$
$\displaystyle(a+b)[01]+(c+d)[10],$ (144) $\displaystyle P_{x_{2}z}$
$\displaystyle=$ $\displaystyle a[00]+b[01]+c[10]+d[11],$ (145) $\displaystyle
P_{x_{2}}$ $\displaystyle=$ $\displaystyle
P_{x},P_{y_{2}}=P_{y},P_{z_{2}}=P_{z}.$ (146)

From Eq.(144) it implies that $X=0$ whenever $Y_{2}=1$, that is,

$\displaystyle{\rm Pr}[X=0|Y_{2}=1]=1$ (147)

From Eqs.(137) and (141) it follows that

$\displaystyle{\rm Pr}[X_{1}=Z_{2}=1]\not=0.$ (148)

Combined with Eqs.(142) and (147), it follows that

$\displaystyle{\rm Pr}[X=Y=0]\not=0,$ (149)

which is contradicted to Eq.(111). This has proved that the distribution in
Eq.(111) is incompatible with the tripartite network in Fig.3(a).

###  E.2 Negative Shannon mutual information

Here, we firstly consider the W-type distribution in Eq.(108). We show that
its Shannon mutual information satisfies $I_{ws}(X;Y;Z)\leq 0$ for the subcase
of $a=c$ or $b=c$. Similar proofs hold for other two distributions in
Eqs.(109) and (110). In fact, from the marginal distributions in
Eqs.(114)-(119) we get

$\displaystyle H(X)$ $\displaystyle=$
$\displaystyle-c\log_{2}c-(1-c)\log_{2}(1-c),$ $\displaystyle H(Y)$
$\displaystyle=$ $\displaystyle-(b+d)\log_{2}b-(a+c)\log_{2}(a+c),$
$\displaystyle H(Z)$ $\displaystyle=$
$\displaystyle-(a+d)\log_{2}(a+d)-(b+c)\log_{2}(b+c),$ $\displaystyle H(X,Y)$
$\displaystyle=$ $\displaystyle-a\log_{2}a-(b+d)\log_{2}(b+d)-c\log_{2}c,$
$\displaystyle H(X,Z)$ $\displaystyle=$
$\displaystyle-b\log_{2}b-(a+d)\log_{2}(a+d)-c\log_{2}c,$ $\displaystyle
H(Y,Z)$ $\displaystyle=$ $\displaystyle H(X,Y,Z)=-a\log_{2}a-b\log_{2}b$ (150)
$\displaystyle-c\log_{2}c-d\log_{2}d.$

This implies the Shannon mutual information as

$\displaystyle I_{ws}(X;Y;Z)$ $\displaystyle=$ $\displaystyle
H(X)+H(Y)+H(Z)+H(X,Y,Z)$ (151) $\displaystyle-H(X,Y)-H(X,Z)-H(Y,Z)$
$\displaystyle=$ $\displaystyle a\log_{2}a+b\log_{2}b+c\log_{2}c$
$\displaystyle-(1-c)\log_{2}(1-c)-(a+c)\log_{2}(a+c)$
$\displaystyle-(b+c)\log_{2}(b+c).$

Now, consider the subcase of $a=c$. From Eq.(151) it follows that

$\displaystyle I_{ws}(X;Y;Z)$ $\displaystyle=$ $\displaystyle
b\log_{2}(b)-(1-a)\log_{2}(1-a)-2a$ (152) $\displaystyle-(b+a)\log_{2}(b+a).$

The first-order of partial derivatives are given by

$\displaystyle\frac{\partial I_{ws}(X;Y;Z)}{\partial a}$ $\displaystyle=$
$\displaystyle\log_{2}b-\log_{2}(a+b)$ (153) $\displaystyle<$ $\displaystyle
0,$ $\displaystyle\frac{\partial I_{ws}(X;Y;Z)}{\partial b}$ $\displaystyle=$
$\displaystyle\log_{2}(1-a)-\log_{2}(a+b)-2$ (154) $\displaystyle<$
$\displaystyle 0$

for any $a,b\geq 0$. This implies that $I_{ws}(X;Y;Z)\leq 0$ from
$I_{ws}(X;Y;Z)=0$ with $a=b=0$. From the symmetry of $a$ and $b$ in Eq.(151),
we can prove the result for the subcase of $b=c$.

Figure 7: (Color online) The numeric difference of $I_{w}(X;Y;Z)$ and
$I_{2}(X;Y;Z)$ for the W-type distribution
$P_{xyz}=a[001]+b[010]+a[100]+d[011]$. Here,
$a=\frac{1}{2}\cos^{2}\theta_{1},b=\sin^{2}\theta_{1}\cos^{2}\theta_{2}$ and
$d=\sin^{2}\theta_{1}\sin^{2}\theta_{2}$,
$\theta_{1},\theta_{2}\in(0,\frac{\pi}{2})$.

Moreover, under the probabilistic transformations in Eqs.(98)-(104) the
maximal Shannon mutual information satisfies $I_{1}(X;Y;Z)\geq 0.5294$ for any
$a,b$. Different from the W-type distribution in Eq.(92) the probabilistic
transformations in Eqs.(98)-(104) can change the minimal Shannon mutual
information $I_{2}(X;Y;Z)$. Specially, we show that

$\displaystyle I_{ws}(X;Y;Z)\preceq I_{2}(X;Y;Z)$ (155)

for any $0\leq a,b\leq 1$. This shows a different feature of the W-type
distribution in Eq. (108) from in Eq.(92).

Now, we consider the W-type distribution in Eq.(111). Different from the
distributions in Eqs.(108)-(110) its Shannon mutual information satisfies
$I_{ws}(X;Y;Z)\geq 0$ for any $a,b,c,d$. In fact, from the marginal
distributions in Eqs.(132)-(137) we get

$\displaystyle H(X)$ $\displaystyle=$ $\displaystyle H(Y)=H(X,Y)$ (156)
$\displaystyle=$ $\displaystyle-(a+b)\log_{2}(a+b)$
$\displaystyle-(c+d)\log_{2}(c+d),$ $\displaystyle H(Z)$ $\displaystyle=$
$\displaystyle-(a+c)\log_{2}(a+c)$ (157) $\displaystyle-(b+d)\log_{2}(b+d),$
$\displaystyle H(X,Z)$ $\displaystyle=$ $\displaystyle H(Y,Z)=H(X,Y,Z)$ (158)
$\displaystyle=$ $\displaystyle-a\log_{2}a-b\log_{2}b$
$\displaystyle-c\log_{2}c-d\log_{2}d.$

This implies the Shannon mutual information satisfying

$\displaystyle I_{ws}(X;Y;Z)$ $\displaystyle=$ $\displaystyle
H(X)+H(Z)-H(X,Z)$ (159) $\displaystyle\geq$ $\displaystyle 0.$

Similar proofs hold for other two distributions in Eqs.(112) and (113).

##  Appendix F Proof of Result 3

The proof of Result 3 is completed using quantum realization of Bayesian
network, which includes any classical realization. We firstly prove the result
for a tripartite quantum network consisting of any pure three-qubit state
$|\Phi\rangle_{ABC}$ on Hilbert space
$\mathcal{H}_{A}\otimes\mathcal{H}_{B}\otimes\mathcal{H}_{C}$. Consider its
Schmidt decomposition with respect to the bipartition $A$ and $BC$ as

$\displaystyle|\Phi\rangle=\sum_{i=1}^{s}\lambda_{i}|\phi_{i}\rangle_{A}|\psi_{i}\rangle_{BC},$
(160)

where $\lambda_{i}$’s are Schmidt coefficients satisfying
$\sum_{i}\lambda_{i}^{2}=1$, and $\\{|\phi_{i}\rangle\\}$ are orthogonal
states of the particle $A$ while $\\{|\psi_{i}\rangle\\}$ are orthogonal
states of joint system $BC$.

If $s=1$, $|\Phi\rangle$ is a product state. In this case, for any local POVM
operators $\\{M_{x}\\},\\{N_{y}\\},\\{L_{z}\\}$, we have

$\displaystyle p_{xyz}$ $\displaystyle=$ $\displaystyle{\rm
Tr}[|\Phi\rangle\langle\Phi|(M_{x}\otimes{}N_{y}\otimes{}L_{z})]$ (161)
$\displaystyle=$ $\displaystyle{\rm
Tr}(|\phi_{1}\rangle\langle\phi_{1}|M_{x}){\rm
Tr}(|\psi_{1}\rangle\langle\psi_{1}|(N_{y}\otimes{}L_{z}))$ $\displaystyle=$
$\displaystyle p_{x}p_{yz}$

for any $x,y,z$. This implies that

$\displaystyle I(X;Y;Z)=I(Y;Z)-I(Y;Z|X)=0.$ (162)

It follows that $I_{min}(X;Y;Z)=0$.

If $s=2$, $|\Phi\rangle$ is an entanglement. In this case, by using proper
local unitary transformations for each particle, we obtain three entangled
states as

$\displaystyle|\Phi_{1}\rangle$ $\displaystyle=$
$\displaystyle\lambda_{1}|000\rangle+\lambda_{2}|111\rangle,$ (163)
$\displaystyle|\Phi_{2}\rangle$ $\displaystyle=$
$\displaystyle\lambda_{1}|000\rangle+\lambda_{2}|1\rangle(\gamma_{1}|01\rangle$
(164) $\displaystyle+\gamma_{2}|10\rangle+\gamma_{3}|11\rangle),$
$\displaystyle|\Phi_{3}\rangle$ $\displaystyle=$
$\displaystyle\lambda_{1}|0\rangle(\alpha|00\rangle+\beta|11\rangle)+\lambda_{2}|1\rangle(\gamma\beta|00\rangle$
(165)
$\displaystyle-\gamma\alpha|11\rangle+\gamma_{1}|01\rangle+\gamma_{2}|10\rangle).$

For the GHZ-type state in Eq.(163) each party performs the projection
measurement under the computation basis $\\{|0\rangle,|1\rangle\\}$. They get
the tripartite probability distribution as

$\displaystyle P_{xyz}=\lambda_{1}^{2}[000]+\lambda_{1}^{2}[111],$ (166)

which can be classically transformed into

$\displaystyle P_{xyz}^{\prime}=\sum_{x,y,z=0,1}\frac{1}{8}[xyz]$ (167)

by using a post-processing transformation

$\displaystyle\mathcal{T}:[i]\mapsto\frac{1}{2}[0]+\frac{1}{2}[1],i=0,1$ (168)

for each party. For the distribution $P_{xyz}^{\prime}$ in Eq.(167) we get

$\displaystyle I(X;Y;Z)=0.$ (169)

This implies that $I_{min}(X;Y;Z)\leq 0$. Moreover, we can show that there are
local POVM measurements such that $I_{min}(X;Y;Z)<0$ for any
$\lambda_{1}\in(0,1)$ assisted by numeric evaluations.

For the entanglement in Eq. (164), each party performs the projection
measurement under the computation basis $\\{|0\rangle,|1\rangle\\}$. They get
a joint probability distribution as

$\displaystyle P_{xyz}$ $\displaystyle=$
$\displaystyle\lambda_{1}^{2}[000]+\lambda_{2}^{2}\gamma_{1}^{2}[101]+\lambda_{2}^{2}\gamma_{2}^{2}[110]$
(170) $\displaystyle+\lambda_{2}^{2}\gamma_{3}^{2}[111],$

which can be classically transformed into the distribution in Eq.(167) by
using the post-processing transformation in Eq. (168) for each party. This
implies that $I_{min}(X;Y;Z)\leq 0$. Moreover, we can prove there are local
measurements such that $I_{min}(X;Y;Z)<0$ for almost all
$\lambda_{i},\gamma_{j}\in(0,1)$ assisted by numeric evaluations.

For the entanglement in Eq.(165) each party performs the projection
measurement under the computation basis $\\{|0\rangle,|1\rangle\\}$. And then,
they can transform the joint probability distribution into the distribution
Eq.(167) by using the post-processing transformation (168). This implies that
$I_{min}(X;Y;Z)\leq 0$. Moreover, there are local measurements such that
$I_{min}(X;Y;Z)<0$ for almost all
$\lambda_{i},\alpha,\beta,\gamma_{j},\in(0,1)$ assisted by numeric
evaluations.

Similarly result holds for any high-dimensional entanglement $|\Phi\rangle$ on
Hilbert space $\mathcal{H}_{A}\otimes\mathcal{H}_{B}\otimes\mathcal{H}_{C}$.
Here, each joint distribution under local projection measurements can be
changed into a separable distribution by using the post-processing
transformation in Eq.(168) for any outcome.

##  Appendix G Shannon mutual information from general networks

Consider a general $n+m$-partite quantum network $\mathcal{N}_{n}$, as shown
in Fig.4. Our goal here is to prove both negative and positive mutual
information can be generated from $\mathcal{N}_{n}$ with $n\geq 3$.

###  G.1 Generating positive mutual information

In this subsection, we show some local operations may build positive Shannon
mutual information for others as

$\displaystyle I(X_{1};\cdots;X_{n})=0$ $\displaystyle
I(X_{1};\cdots;X_{n}|Y)>0$ (171)

where the joint probability distribution $P_{X_{1}\cdots{}X_{n}Y}$ is obtained
from local measurements on the network $\mathcal{N}_{k}$. This implies
negative Shannon mutual information as

$\displaystyle I(X_{1};\cdots;X_{n};Y)$ $\displaystyle:=$ $\displaystyle
I(X_{1};\cdots;X_{n})$ (172) $\displaystyle-I(X_{1};\cdots;X_{n}|Y)$
$\displaystyle\leq$ $\displaystyle 0,$

where all the random variables $Y:=Y_{1}\cdots{}Y_{m}$ are regarded as one
high-dimensional variable.

In fact, define local measurements of $\textsf{A}_{i}$ as
$\\{M_{x_{i}}=|x_{i}\rangle\langle x_{i}|\\}$ which satisfy
$\sum_{x_{i}}M_{x_{i}}=\openone_{A_{i}}$ with the identity operator
$\openone_{A_{i}}$ on the state space of $\textsf{A}_{i}$, $i=1,\cdots,n$. Let
$\\{M_{y_{j}}\\}$ with
$M_{y_{j}}\in\\{|i\rangle\pm|2^{s_{j}}-i-1\rangle,i=0,\cdots,2^{s_{j}-1}\\}$
be local measurements of $\textsf{B}_{j}$ and satisfy
$\sum_{y_{j}}M_{y_{j}}=\openone_{B_{j}}$ with the identity operator
$\openone_{B_{j}}$ on the state space of $\textsf{B}_{j}$, where $s_{j}$
denotes the number of particle contained in the node $\textsf{B}_{j}$,
$j=1,\cdots,n$. Since all the nodes $\textsf{A}_{1},\cdots,\textsf{A}_{n}$ are
independent, the joint probability of the outcome $x_{1}\cdots{}x_{n}$ is
given by

$\displaystyle p_{x_{1}\cdots{}x_{n}}$ $\displaystyle=$ $\displaystyle{\rm
Tr}(\otimes_{i=1}^{n}M_{x_{i}}\rho_{A_{1}\cdots{}A_{n}})$ (173)
$\displaystyle=$ $\displaystyle\otimes_{i=1}^{n}{\rm
Tr}(M_{x_{i}}\rho_{A_{i}})$ $\displaystyle=$
$\displaystyle\prod_{i=1}^{n}p_{x_{i}},$

where $\rho_{A_{1}\cdots{}A_{n}}$ denotes the reduced density matrix of
$\textsf{A}_{1},\cdots,\textsf{A}_{n}$, $\rho_{A_{i}}$ denotes the reduced
density matrix of $\textsf{A}_{i}$, and $p_{x_{i}}$ denotes the probability of
the outcome $x_{i}$ by $\textsf{A}_{i}$. From Eq.(173) it follows that

$\displaystyle I(X_{1};\cdots;X_{n})=0.$ (174)

Now, consider the projection measurement $\\{M_{y_{j}}\\}$ of
$\textsf{B}_{1},\cdots,\textsf{B}_{m}$, and send out the measurement outcomes
for $\textsf{A}_{1},\cdots,\textsf{A}_{n}$. It is easy to check that the joint
state of $\textsf{A}_{1},\cdots,\textsf{A}_{n}$ is changed into an $n$-qubit
GHZ state

$\displaystyle|\Phi\rangle_{A_{1}\cdots{}A_{n}}=\cos\varphi|0\rangle^{\otimes
n}+\sin\varphi|1\rangle^{\otimes n}$ (175)

under proper local unitary operations Entswap1993 , where $\varphi$ depends on
parameters $\theta_{1},\cdots,\theta_{n}$. Now, under the local projection
measurement $\\{M_{x_{i}}\\}$, it follows the joint distribution

$\displaystyle
P_{x_{1}\cdots{}x_{n}}=\cos^{2}\varphi[0\cdots{}0]+\sin^{2}\varphi[1\cdots{}1].$
(176)

We obtain that

$\displaystyle I(X_{1};\cdots;X_{n}|Y)$ $\displaystyle=$
$\displaystyle-\cos^{2}\varphi\log(\cos^{2}\varphi)$ (177)
$\displaystyle-\sin^{2}\varphi\log(\sin^{2}\varphi)$ $\displaystyle>$
$\displaystyle 0$

for any $\theta_{1},\cdots,\theta_{n}\in(0,\frac{\pi}{2})$. From Eqs.(174) and
(177) we get that

$\displaystyle I(X_{1};\cdots;X_{n};Y)<0.$ (178)

This has proved the result. One example is long chain network or star network
Luo2018 .

###  G.2 Generating positive Shannon mutual information

In this subsection, we show some local operations may generate negative
Shannon mutual information for others as

$\displaystyle I(X_{1};\cdots;X_{n})=0$ (179) $\displaystyle
I(X_{1};\cdots;X_{n}|Y)<0$ (180)

where the joint probability distribution
$P_{X_{1}\cdots{}X_{n}Y_{1}\cdots{}Y_{m}}$ is obtained from local measurements
on the network $\mathcal{N}_{k}$. This implies positive Shannon mutual
information as

$\displaystyle I(X_{1};\cdots;X_{n};Y)$ $\displaystyle=$ $\displaystyle
I(X_{1};\cdots;X_{n})$ (181) $\displaystyle-I(X_{1};\cdots;X_{n}|Y)$
$\displaystyle>$ $\displaystyle 0$

which is different from Eq.(172). This means that the $n$-independent network
shows different correlations beyond the chain network in Fig.2(b) for any
$n\geq 3$.

The proof is constructed as follows. Consider a special case of
$\mathcal{N}_{q}$ as $n=3$ and $m=1$. Here, each pair of $\textsf{A}_{i}$ and
B shares one EPR state $|\phi_{i}\rangle$. In this case, the total state of
$\mathcal{N}_{q}$ is given by

$\displaystyle|\Phi\rangle=\sum_{i_{1}i_{2}i_{2}=0,1}a_{i_{1}i_{2}i_{3}}|i_{1}i_{2}i_{3}\rangle_{A_{1}A_{2}A_{3}}|i_{1}i_{2}i_{3}\rangle_{B}$
(182)

where
$a_{i_{1}i_{2}i_{3}}=\prod_{j=1}^{3}\cos^{i_{j}+1\\!\\!\mod\\!2}\theta_{j}\sin^{i_{j}}\theta_{j}$.
B performs a local measurement under the quantum Fourier basis $\\{\Pi_{y}\\}$
with $\Pi_{y}\in\\{|\varphi_{i}\rangle\langle\varphi_{i}\\}$, where
$|\varphi_{i}\rangle$ are defined as

$\displaystyle|\varphi_{i}\rangle$ $\displaystyle=$
$\displaystyle\frac{1}{2}(|001\rangle+\omega^{i}|010\rangle$
$\displaystyle+\omega^{2i}|100\rangle+\omega^{3i}|011\rangle),$
$\displaystyle|\varphi_{4+i}\rangle$ $\displaystyle=$
$\displaystyle\frac{1}{2}(|110\rangle+\omega^{i}|101\rangle$ (183)
$\displaystyle+\omega^{2i}|000\rangle+\omega^{3i}|111\rangle$

with an unit root $\omega=\sqrt{-1}$, $i=1,\cdots,4$. After the local
measurement, by proper local operation of $\textsf{A}_{1}$, the joint state of
$\textsf{A}_{1},\textsf{A}_{2}$ and $\textsf{A}_{3}$ is changed into

$\displaystyle|\Phi_{i}\rangle$ $\displaystyle=$ $\displaystyle
a_{001}|001\rangle+\omega^{-i}a_{010}|010\rangle$
$\displaystyle+\omega^{-2i}a_{100}|100\rangle+\omega^{-3i}a_{011}|011\rangle),$
$\displaystyle|\Phi_{4+i}\rangle$ $\displaystyle=$ $\displaystyle
a_{110}|010\rangle+\omega^{-i}a_{101}|001\rangle$ (184)
$\displaystyle+\omega^{-2i}a_{000}|100\rangle+\omega^{-3i}a_{111}|011\rangle,$

where $i=1,\cdots,4$. Under the local projection of all parties
$\textsf{A}_{j}$’s, it implies a joint distribution as

$\displaystyle P_{x_{1}x_{2}x_{3}}$ $\displaystyle=$ $\displaystyle
a^{2}_{001}[001]+a^{2}_{010}[010]$ (185)
$\displaystyle+a_{100}^{2}[100]+a^{2}_{011}[011],$

or

$\displaystyle P_{x_{1}x_{2}x_{3}}$ $\displaystyle=$ $\displaystyle
a^{2}_{110}[010]+a^{2}_{101}[001]$ (186)
$\displaystyle+a^{2}_{000}[100]+a^{2}_{111}[011]$

Combined with Appendix E.2, it follows that

$\displaystyle I(X_{1};X_{2};X_{3}|Y)<0$ (187)

for $\theta_{1}=\theta_{3}$. This has completed the proof from Eqs.(174) and
(187).