$\lambda:\mathcal{M}^{\smash{\scalebox{0.7}[1.4]{\rotatebox{90.0}{{\T1\guilsinglleft}}}}}\otimes\pi^{*}M\xrightarrow{{}_{\sim}}\sigma^{*}\mathcal{M}\otimes\pi^{*}L^{1-2r}.$

Viceversa, suppose that $\mathcal{M}$ is a torsion-free sheaf of rank 1 and
degree $d^{\prime}$ on $X$ with a line bundle $M$ of degree $d$ on $C$ and an
isomorphism
$\lambda:\mathcal{M}^{\smash{\scalebox{0.7}[1.4]{\rotatebox{90.0}{{\T1\guilsinglleft}}}}}\otimes\pi^{*}M\xrightarrow{{}_{\sim}}\sigma^{*}\mathcal{M}\otimes\pi^{*}L^{1-2r}$.
Let $(E,\Phi^{\prime})$ be the Higgs pairs of rank $2r$ and degree $rd$
corresponding to $\mathcal{M}$ by the spectral correspondence. Applying
Relative duality in dimension 0 and the projection formula as in Section 4.3,
we have an isomorphism:

$\displaystyle
E^{\smash{\scalebox{0.7}[1.4]{\rotatebox{90.0}{{\T1\guilsinglleft}}}}}$
$\displaystyle=\pi_{*}(\mathcal{M})^{*}\simeq\pi_{*}(\mathcal{M}^{\smash{\scalebox{0.7}[1.4]{\rotatebox{90.0}{{\T1\guilsinglleft}}}}}\otimes\pi^{*}L^{2r-1})\simeq$
$\displaystyle\simeq\pi_{*}(\sigma^{*}(\mathcal{M})\otimes\pi^{*}M^{-1})\simeq$
$\displaystyle\simeq\pi_{*}(\sigma^{*}(\mathcal{M}))\otimes
M^{-1}=\pi_{*}(\sigma_{*}\sigma^{*}\mathcal{M})\otimes M^{-1}\simeq$
$\displaystyle\simeq\pi_{*}(\sigma_{*}\mathcal{O}_{X}\otimes\mathcal{M})\otimes
M^{-1}=\pi_{*}(\mathcal{O}_{X}\otimes\mathcal{M})\otimes M^{-1}\simeq$
$\displaystyle\simeq\pi_{*}(\mathcal{M})\otimes M^{-1}=E\otimes M^{-1};$

that induces the non-degenerate $M$-valued symplectic structure
$\omega:E\otimes E\rightarrow M$.

To sum up, the datum of $(P,\phi)$ in the fiber
$\mathcal{H}_{\mathrm{GSp},2r,rd}^{-1}({\underline{a}})$ corresponds uniquely
to the datum $(\mathcal{M},M,\lambda)$ of a torsion-free rank-1 sheaf
$\mathcal{M}\in\overline{\mathbb{J}}(X,d^{\prime})$, a line bundle $M\in
J(C,d)$ and an isomorphism
$\lambda:\mathcal{M}^{\smash{\scalebox{0.7}[1.4]{\rotatebox{90.0}{{\T1\guilsinglleft}}}}}\otimes\pi^{*}M\xrightarrow{{}_{\sim}}\sigma^{*}\mathcal{M}\otimes\pi^{*}L^{1-2r}$,
i.e. an element of the equalizer stack $\mathcal{E}_{\underline{a}}$. ∎

###  4.5. $\mathrm{PSp}(2r,\mathbb{C})$-Higgs pairs

The Projective Symplectic group $\mathrm{PSp}(2r,\mathbb{C})$ is defined by
the exact sequence:

(4.9) $\displaystyle
0\rightarrow\mathbb{C}^{*}\xrightarrow{\lambda\mapsto\lambda
I_{2r}}\mathrm{GSp}(2r,\mathbb{C})\rightarrow\mathrm{PSp}(2r,\mathbb{C})\rightarrow
0$

or by the exact sequence:

(4.10) $\displaystyle 0\rightarrow\\{\pm 1\\}\xrightarrow{1\mapsto
I_{2r}}\mathrm{Sp}(2r,\mathbb{C})\rightarrow\mathrm{PSp}(2r,\mathbb{C})\rightarrow
0.$

In particular, the sheaf-theoretic version of sequence 4.9 yields the
cohomology exact sequence:

(4.11) $\displaystyle H^{1}(C,\mathcal{O}_{C}^{*})\rightarrow
H^{1}(C,\mathrm{GSp}(2r,\mathcal{O}_{C}))\ext@arrow
0579\arrowfill@\relbar\relbar\twoheadrightarrow{}{q}H^{1}(C,\mathrm{PSp}(2r,\mathcal{O}_{C}))\rightarrow
0.$

This surjection, read in terms of cocycles, means that
$\mathrm{PSp}(2r,\mathbb{C})$-principal bundles are in one-to-one
correspondence with equivalence classes of
$\mathrm{GSp}(2r,\mathbb{C})$-principal bundles, with respect to the action on
associated bundles given by tensor product of line bundles. If $P$ is a
$\mathrm{PSp}(2r,\mathbb{C})$-principal bundle and $\tilde{P}$ is a
$\mathrm{GSp}(2r,\mathbb{C})$-principal bundle such that $q(\tilde{P)}=P$, we
say that $\tilde{P}$ is a lifting of $P$ to a
$\mathrm{GSp}(2r,\mathbb{C})$-principal bundle.

Moreover, by sequence 4.10, $\mathrm{PSp}(2r,\mathbb{C})$ is the quotient of
$\mathrm{Sp}(2r,\mathbb{C})$ by the action of a finite group, hence the
associated Lie algebras $\mathfrak{psp}(2r,\mathbb{C})$ and
$\mathfrak{sp}(2r,\mathbb{C})$ are equal. If $P$ is any
$\mathrm{PSp}(r2,\mathbb{C})$-principal bundle and $\tilde{P}$ is a lifting of
$P$ to a $\mathrm{GSp}(2r,\mathbb{C})$-principal bundle, then a section $\phi$
of $H^{0}(C,\operatorname{ad}P\otimes L)$ determines uniquely a section
$\tilde{\phi}$ of $H^{0}(C,\operatorname{ad}\tilde{P}\otimes L)$ with trace
equal to 0, and viceversa. We say that $(\tilde{P},\tilde{\phi})$ is a lifting
of $(P,\phi)$ to a $\mathrm{GSp}(2r,\mathbb{C})$-Higgs pair and with a slight
abuse of notation we write $q(\tilde{P},\tilde{\phi})=(P,\phi)$.

To sum up, any $\mathrm{PSp}(2r,\mathbb{C})$-Higgs pair $(P,\phi)$ has a
lifting $(\tilde{P},\tilde{\phi})$ to a $\mathrm{GSp}(2r,\mathbb{C})$-Higgs
pair with trace zero, corresponding via the associated bundle construction to
the datum $(E,\Phi,M,\omega)$ of a Higgs pair $(E,\Phi)$ of rank $2r$, a non-
degenerate symplectic form $\omega:E\otimes E\rightarrow M$ with values in a
line bundle $M$ of degree $d$ on $C$, and a Higgs field $\Phi$ satisfying the
condition

$\omega(\Phi v,w)+\omega(v,\Phi w)=0.$

Then, the datum of $(P,\phi)$ corresponds uniquely to the datum of the
equivalence class $[(E,\Phi,M,\omega)]$ of Higgs pairs of rank $2r$ with a
non-degenerate symplectic form, under the equivalence relation $\sim_{J(C)}$
defined by:

$(E,\Phi,M\omega)\sim_{J(C)}(E\otimes N,\Phi\otimes 1_{N},M\otimes
N^{2},\omega_{N})\hskip 20.00003pt\textrm{for any }N\in J(C)$

where

$\omega_{N}:(E\otimes N)\otimes(E\otimes N)\rightarrow M\otimes N^{2}$

is obtained by extension of scalars.

More precisely, let
$\mathcal{M}_{\mathrm{GSp}}(2r)=\coprod_{d\in\mathbb{Z}}\mathcal{M}_{\mathrm{GSp}}(2r,rd)$
be the moduli stack of $\mathrm{GSp}$-Higgs pairs of rank $2r$ in any degree
and denote with $\mathcal{M}_{\mathrm{GSp}}^{\operatorname{tr}=0}(2r)$ the
closed substack of $\mathcal{M}_{\mathrm{GSp}}(2r)$ given by
$\mathrm{GSp}$-Higgs pairs of rank $2r$ with trace zero. Then,
$[(E,\Phi,M,\omega)]$ is the orbit of $(E,\Phi,M,\omega)$ under the action of
$J(C)$ on $\mathcal{M}_{\mathrm{GSp}}^{\operatorname{tr}=0}(2r)$ defined by:

$\displaystyle\mathcal{M}_{\mathrm{GSp}}^{\operatorname{tr}=0}(2r)\times J(C)$
$\displaystyle\longrightarrow\mathcal{M}_{\mathrm{GSp}}^{\operatorname{tr}=0}(2r)$
$\displaystyle((E,\Phi,M,\omega),N)$ $\displaystyle\longmapsto(E\otimes
N,\Phi\otimes 1_{N},M\otimes N^{2},\omega_{N}).$

Recall that the isomorphism $\det E\simeq M^{r}$ implies that
$\deg(E)=r\deg(M)$; in other words, the degree of the Higgs pair in the datum
$(E,\Phi,M,\omega)$ is determined by the degree of the line bundle $M$ in the
same datum. The action of $J(C)$ on such datum modifies the degree of $M$ by
multiples of $2$, since $\deg(M\otimes N^{2})=\deg(M)+2\deg(N)$. This his fact
implies that the degree of $\mathrm{GSp}$-Higgs pairs associated to a
$\mathrm{PSp}$-Higgs pair is defined only modulo $2r$. Then, we can give the
following definition.

######  Definition 4.5.1.

Let $(P,\phi)$ be a $\mathrm{PSp}(2r,\mathbb{C})$-Higgs pair and let
$(E,\Phi,M,\omega)$ be the datum of a Higgs pair with trace zero and degree
$rd$ endowed with a $M$-valued symplectic form $\omega$, corresponding to a
lifting $(\tilde{P},\tilde{\phi})$ of $(P,\phi)$ to a $\mathrm{GSp}$-Higgs
pair. The degree of $(P,\phi)$ is the congruence class
$\overline{rd}\in\mathbb{Z}/2r\mathbb{Z}$.

######  Remark 4.5.2.

The same definition works for $\mathrm{PSp}(2r,\mathbb{C})$-principal bundles,
without Higgs pair. The degree of a $\mathrm{PSp}$-principal bundle $P$ is the
congruence class (modulo $2r$) of the degree of the vector bundles associated
to any lifting $\tilde{P}$ of $P$ to a $\mathrm{GSp}$-principal bundle.

Clearly, only two cases are possible: the degree of a
$\mathrm{PSp}(2r,\mathbb{C})$-Higgs pair $(P,\phi)$ is congruent either to $0$
or to $r$ modulo $2r$. This reflects the fact that the topological type of $P$
is parametrized by
$\pi_{1}(\mathrm{PSp}(2r,\mathbb{C}))\simeq\mathbb{Z}/2\mathbb{Z}$.

Up to the action on $(E,\Phi,M,\omega)$ of a line bundle of degree $1$ on $C$,
it is straightforward to see that the $J(C)$-orbit $[(E,\Phi,M,\omega)]$
corresponding to a $\mathrm{PGL}$-Higgs pair with degree
$\overline{rd}\in\mathbb{Z}/2r\mathbb{Z}$ is always the orbit of a datum whose
Higgs pair has degree $0$ or $rd$. If we restrict the correspondence to Higgs
pairs with fixed degree $0$ or $rd$, we see that a $\mathrm{PGL}$-Higgs pair
of degree $\overline{rd}$ is identified uniquely by the orbit of a Higgs pair
with trace zero and degree $0$ or $rd$ with respect to the action of line
bundles of degree $0$ on $C$.

A basis for the invariant polynomials of
$\mathfrak{psp}(2r,\mathbb{C})=\mathfrak{sp}(2r,\mathbb{C})$ is given by
$\\{p_{2i}\\}_{i=1,\dots,r}$ where
$p_{2i}(P,\phi):=\operatorname{tr}(\wedge^{2i}\phi)$. The corresponding
$\mathrm{PSp}$-Hitchin morphism takes the form:

$\displaystyle\mathcal{H}_{\mathrm{PSp},2r,\overline{rd}}:\mathcal{M}_{\mathrm{PSp}}(2r,\overline{rd})$
$\displaystyle\longrightarrow\mathcal{A}_{\mathrm{PSp}}(2r)=\bigoplus_{l=1}^{r}H^{0}(C,L^{2l})$
$\displaystyle(P,\phi)$
$\displaystyle\longmapsto(p_{2}(P,\phi),p_{4}(P,\phi),\dots,p_{2r}(P,\phi)).$

If $(\tilde{P},\tilde{\phi})$ is any lifting of $(P,\phi)$ to a
$\mathrm{GSp}(2r,\mathbb{C})$-Higgs pair, then
$\mathcal{H}_{\mathrm{GSp},2r,rd}(\tilde{P},\tilde{\phi})=\mathcal{H}_{\mathrm{PSp},2r,\overline{rd}}(P,\phi)$.
Moreover, for any characteristic
${\underline{a}}\in\mathcal{A}_{\mathrm{PSp}}(r)$, the spectral curve
$\pi:X_{\underline{a}}\rightarrow C$ is defined in the total space
$p:\mathbb{P}(\mathcal{O}_{C}\oplus L^{-1})\rightarrow C$ by the equation

$x^{2r}y+a_{2}x^{2r-2}y^{2}+\dots+a_{2r-2}x^{2}y^{2r-2}+a_{2r}y^{2r}=0.$

Hence, as in the case of $\mathrm{Sp}(2r,\mathbb{C})$-Higgs pairs, the curve
$X_{\underline{a}}$ has an involution defined by $\sigma(x)=-x$.

We can now state the following proposition.

######  Proposition 4.5.3.

(Spectral correspondence for $\mathrm{PSp}(2r,\mathbb{C})$) Let
${\underline{a}}\in\mathcal{A}_{\mathrm{PSp}}(2r)$ be any characteristic, let
$X=X_{\underline{a}}\xrightarrow{\pi}C$ be the associated spectral curve with
involution $\sigma:X\rightarrow X$. Let $d\in\\{0,1\\}$,
$d^{\prime}=rd+r(2r-1)\ell$ and denote with
$\mathcal{P}(d^{\prime},n)=\overline{\mathbb{J}}(X,d^{\prime})\times J^{d}(C)$
the Cartesian product of the compactified Jacobian of degree $d^{\prime}$ on
$X$ and the Jacobian of degree $d$ on $C$, endowed with the projection maps
$p_{X}$ and $p_{C}$ on $\overline{\mathbb{J}}(X,d^{\prime})$ and $J^{d}(C)$
respectively. Let $\mathcal{E}_{{\underline{a}}^{\prime}}$ be the equalizer of
the two maps

$\displaystyle(\mathcal{H}om_{\mathcal{O}_{X}}(\texttt{\char
95\relax},\mathcal{O}_{X})\circ p_{X})\otimes(\pi^{*}\circ
p_{C}):\mathcal{P}(d^{\prime},d)$
$\displaystyle\rightarrow\overline{\mathbb{J}}(X,rd-r(2r-1)\ell)$
$\displaystyle(\sigma^{*}\circ
p_{X})\otimes\pi^{*}L^{1-2r}:\mathcal{P}(d^{\prime},d)$
$\displaystyle\rightarrow\overline{\mathbb{J}}(X,rd-r(2r-1)\ell).$

The group $J^{0}(C)$ of line bundles of degree $0$ on $C$ acts on
$\mathcal{E}_{\underline{a}}$ as follows:

$\displaystyle\mathcal{E}_{\underline{a}}\times J^{0}(C)$
$\displaystyle\longrightarrow\mathcal{E}_{\underline{a}}$
$\displaystyle((\mathcal{M},M,\lambda),N)$
$\displaystyle\longmapsto(\mathcal{M}\otimes\pi^{*}N,M\otimes
N^{2},\lambda_{N})$

where $\lambda_{N}$ is given by the composition of
$\lambda\otimes\operatorname{id}_{\pi^{*}N}$ with the canonical isomorphisms:

$\displaystyle(\mathcal{M}\otimes\pi^{*}N)^{\smash{\scalebox{0.7}[1.4]{\rotatebox{90.0}{{\T1\guilsinglleft}}}}}\otimes\pi^{*}(M\otimes
N^{2})$
$\displaystyle\xrightarrow{{}_{\sim}}\mathcal{M}^{\smash{\scalebox{0.7}[1.4]{\rotatebox{90.0}{{\T1\guilsinglleft}}}}}\otimes(\pi^{*}N)^{-1}\otimes\pi^{*}M\otimes(\pi^{*}N)^{2}\xrightarrow{{}_{\sim}}$
$\displaystyle\xrightarrow{{}_{\sim}}\mathcal{M}^{\smash{\scalebox{0.7}[1.4]{\rotatebox{90.0}{{\T1\guilsinglleft}}}}}\otimes\pi^{*}M\otimes\pi^{*}N\xrightarrow{\lambda\otimes\operatorname{id}_{\pi^{*}N}}$
$\displaystyle\xrightarrow{{}_{\sim}}\sigma^{*}\mathcal{M}\otimes\pi^{*}L^{1-2r}\otimes\pi^{*}N\xrightarrow{{}_{\sim}}$
$\displaystyle\xrightarrow{{}_{\sim}}\sigma^{*}(\mathcal{M}\otimes\pi^{*}N)\otimes\pi^{*}L^{1-2r}.$

Then, the fiber
$\mathcal{H}_{\mathrm{PSp},2r,\overline{rd}}^{-1}({\underline{a}})$ of the
$\mathrm{PSp}$-Hitchin morphism is isomorphic, via the spectral
correspondence, to the quotient $\mathcal{E}_{\underline{a}}/J^{0}(C)$.

###### Proof.

Let $(P,\phi)$ be any $\mathrm{PSp}(2r,\mathbb{C})$-Higgs pair with
characteristic ${\underline{a}}$ and let $(E,\Phi,M,\omega)$ be the datum of a
Higgs pair with a non-degenerate $M$-valued symplectic form such that
$(P\phi)$ corresponds uniquely to the datum of the $J^{0}(C)$-equivalence
class $[(E,\Phi,M,\omega)]$.

By Proposition 4.4.1, the datum of $(E,\Phi,M,\omega)$ corresponds, via the
spectral correspondence, to the datum $(\mathcal{M},M,\lambda)$ of a torsion-
free rank-1 sheaf $\mathcal{M}\in\overline{\mathbb{J}}(X,d^{\prime})$, the
line bundle $M\in J^{d}(C)$ and an isomorphism
$\lambda:\mathcal{M}^{\smash{\scalebox{0.7}[1.4]{\rotatebox{90.0}{{\T1\guilsinglleft}}}}}\otimes\pi^{*}M\xrightarrow{{}_{\sim}}\sigma^{*}\mathcal{M}\otimes\pi^{*}L^{1-2r}$,
i.e. an element of $\mathcal{E}_{\underline{a}}$. If $N$ is any line bundle of
degree $0$ on $C$, then by the projection formula the datum of $(E\otimes
N,\Phi\otimes 1_{N},M\otimes N^{2},\omega_{N})$ corresponds to the datum of
$(\mathcal{M}\otimes\pi^{*}N,M\otimes N^{2},\lambda_{N})$.

We conclude that the datum of the $J^{0}(C)$-orbit $[(E,\Phi,M,\omega)]$
corresponds, via the spectral correspondence, to the datum of the
$J^{0}(C)$-orbit of $(\mathcal{M},M,\lambda)$. ∎

A particular case occurs for $d=0$, i.e. for
$\mathrm{PGL}(2r,\mathbb{C})$-Higgs pairs of degree $0$. Indeed, recall the
sequence 4.9:

$0\rightarrow\\{\pm 1\\}\xrightarrow{1\mapsto
I_{2r}}\mathrm{Sp}(2r,\mathbb{C})\rightarrow\mathrm{PSp}(2r,\mathbb{C})\rightarrow
0.$

Recalla also that $\\{\pm 1\\}=\mu_{2}$ and
$H^{2}(C,\mu_{2})=\mathbb{Z}/2\mathbb{Z}$. Hence this sequence, applied to
structure sheaves, induces the cohomology exact sequence:

$H^{1}(C,\mu_{2})\rightarrow
H^{1}(C,\mathrm{Sp}(2r,\mathcal{O}_{C}))\ext@arrow
0579\arrowfill@\relbar\relbar\twoheadrightarrow{}{q}H^{1}(C,\mathrm{PSp}(2r,\mathcal{O}_{C}))\xrightarrow{\deg/r}\mathbb{Z}/2\mathbb{Z}\rightarrow
0$

where the last map sends a $\mathrm{PSp}(2r,\mathbb{C})$-principal bundle of
degree $\overline{rd}$ to the congruence class $\overline{d}$. This sequence,
read in terms of cocycles, means that a
$\mathrm{PSp}(2r,\mathbb{C})$-principal bundle $P$ can be lifted to
$\mathrm{Sp}(2r,\mathbb{C})$-principal bundle $P_{0}$ if and only $P$ has
degree equal to $0\in\mathbb{Z}/2r\mathbb{Z}$; any other lifting
$P_{0}^{\prime}$ of $P$ differs from $P_{0}$ by the action on the associated
bundles of a $2$-torsion line bundle by tensor product. Moreover, a Higgs
field $\phi$ on $P$ determines uniquely a a Higgs field $\phi_{0}$ on $P$ and
viceversa.

To sum up, any $\mathrm{PSp}(2r,\mathbb{C})$-Higgs pair $(P,\phi)$ of degree
$0$ has a lifting $(\tilde{P},\tilde{\phi})$ to a
$\mathrm{Sp}(r,\mathbb{C})$-Higgs pair, corresponding to the datum
$(E,\Phi,\omega)$ of a Higgs pair of rank $2r$ and degree $0$ and a non-
degenerate symplectic form $\omega$ via the associated bundle construction.
Then, the datum of $(P,\phi)$ corresponds uniquely to the datum of the
equivalence class $[(E,\Phi,\omega)]$ of Higgs pairs of rank $2r$ (and degree
$0$) with a non-degenerate symplectic form, under the equivalence relation
$\sim_{J^{0}(C)[2]}$ defined by:

$(E,\Phi,\omega)\sim_{J^{0}(C)[2]}(E\otimes N,\Phi\otimes
1_{N},\omega_{N,\epsilon})\hskip 20.00003pt\textrm{for any }(N,\epsilon)\in
J^{0}(C)[2].$

Here, $J^{0}(C)[2]$ denotes the group stack parametrizing line bundles with
$2$-torsion $(N,\epsilon)$ on $C$, where $\epsilon$ is the isomorphism
$N^{-1}\xrightarrow{{}_{\sim}}N$, and $\omega_{N,\epsilon}$ is obtained by
extension of scalars, as follows:

$\omega_{N,\epsilon}:(E\otimes N)\otimes(E\otimes N)\rightarrow
N^{2}\xrightarrow[\epsilon^{-1}\otimes 1]{{}_{\sim}}N^{-1}\otimes
N\xrightarrow[\operatorname{ev}]{{}_{\sim}}\mathcal{O}_{C}.$

We can finally state the following proposition.

######  Proposition 4.5.4.

(Spectral correspondence for $\mathrm{PSp}(2r,\mathbb{C})$ of degree $0$) Let
${\underline{a}}\in\mathcal{A}_{\mathrm{PSp}}(2r)$ be any characteristic and
let $X=X_{\underline{a}}\xrightarrow{\pi}C$ be the associated spectral curve
with involution $\sigma:X\rightarrow X$. Let $d^{\prime}=r(2r-1)\ell$ and let
$\mathcal{E}_{\underline{a}}$ be the equalizer stack of the two maps:

$\displaystyle\texttt{\char
95\relax}^{\smash{\scalebox{0.7}[1.4]{\rotatebox{90.0}{{\T1\guilsinglleft}}}}}:=\mathcal{H}om_{\mathcal{O}_{X}}(\texttt{\char
95\relax},\mathcal{O}_{X}):\overline{\mathbb{J}}(X,d^{\prime})$
$\displaystyle\rightarrow\overline{\mathbb{J}}(X,-d^{\prime})$
$\displaystyle\sigma^{*}\texttt{\char
95\relax}\otimes\pi^{*}L^{1-2r}:\overline{\mathbb{J}}(X,d^{\prime})$
$\displaystyle\rightarrow\overline{\mathbb{J}}(X,-d^{\prime})$

The group stack $J^{0}(C)[2]$ of 2-torsion line bundles on $C$ acts on
$\mathcal{E}_{\underline{a}}$ as follows:

$\displaystyle\mathcal{E}_{\underline{a}}\times\pi^{*}J^{0}(C)[2]$
$\displaystyle\longrightarrow\mathcal{E}_{\underline{a}}$
$\displaystyle((\mathcal{M},\lambda),(\pi^{*}N,\pi^{*}\epsilon))$
$\displaystyle\longmapsto(\mathcal{M}\otimes\pi^{*}N,\lambda\otimes\pi^{*}\epsilon).$

Then, the fiber
$\mathcal{H}_{\mathrm{PSp},2r,\overline{0}}^{-1}({\underline{a}})$ of the
$\mathrm{PSp}$-Hitchin morphism is isomorphic, via the spectral
correspondence, to the quotient
$\mathcal{E}_{\underline{a}}/\pi^{*}J^{0}(C)[2]$.

###### Proof.

Let $(P,\phi)$ be any $\mathrm{PSp}(2r,\mathbb{C})$-Higgs pair with degree $0$
and characteristic ${\underline{a}}$ and let $(\tilde{P},\tilde{\phi})$ be any
lifting to a $\mathrm{Sp}(2r,\mathbb{C})$-Higgs pair. By construction,
$(\tilde{P},\tilde{\phi})$ is a $\mathrm{Sp}(2r,\mathbb{C})$-Higgs pair with
characteristic ${\underline{a}}$ and corresponds uniquely, via the associated
bundle construction, to the datum $(E,\Phi,\omega)$ of a Higgs pair of rank
$2r$ and degree $0$ with characteristic ${\underline{a}}$ and a non-degenerate
symplectic form. Then, the datum of $(P,\phi)$ corresponds uniquely to the
datum of the $J^{0}(C)[2]$-equivalence class $[(E,\Phi,\omega)]$.

By Proposition 4.3.1, the datum of $(E,\Phi,\omega)$ corresponds to the datum
$(\mathcal{M},\lambda)$ of a torsion-free rank-1 sheaf
$\mathcal{M}\in\overline{\mathbb{J}}(X,d^{\prime})$ and an isomorphism
$\lambda:\mathcal{M}^{\smash{\scalebox{0.7}[1.4]{\rotatebox{90.0}{{\T1\guilsinglleft}}}}}\xrightarrow{{}_{\sim}}\sigma^{*}\mathcal{M}\otimes\pi^{*}L^{1-2r}$,
i.e. an element of $\mathcal{E}_{\underline{a}}$. If $N$ is any $2$-torsion
line bundle on $C$ with isomorphism $\epsilon:N^{-1}\xrightarrow{{}_{\sim}}N$,
then by the projection formula the datum of $(E\otimes N,\Phi\otimes
1_{N},\omega_{N})$ corresponds to the datum of
$(\mathcal{M}\otimes\pi^{*}N,\lambda\otimes\pi^{*}\epsilon)$.

We conclude that the datum of the $J^{0}(C)[2]$-orbit $[(E,\Phi,\omega)]$
corresponds to the datum of the $\pi^{*}J^{0}(C)[2]$-orbit of
$(\mathcal{M},\lambda)$. ∎

## Bibliography

  * [AIK77] A. B. Altman, A. Iarrobino, and S. L. Kleiman.  Irreducibility of the compactified Jacobian.  Real and complex singularities (Proc. Ninth Nordic Summer School/NAVF Sympos. Math., Oslo, 1976), pages 1–12, 1977\.
  * [BE00] Dave Bayer and David Eisenbud.  Ribbons And Their Canonical Embeddings.  Transactions of the American Mathematical Society, 347, 2000.
  * [Ber12] José Bertin.  The Punctual Hilbert Scheme: an Introduction.  In Michel Brion, editor, Geometric Methods in Representation Theory I, number 24-I in Séminaires et Congrès, pages 1–100. Société Mathématique de France, 2012.
  * [BGS81] J. Briançon, M. Granger, and J.-P. Speder.  Sur le schéma de Hilbert d’une courbe plane.  Annales scientifiques de l’École Normale Supérieure, 4e série, 14(1):1–25, 1981.
  * [BNR89] A. Beauville, M.S. Narasimhan, and S. Ramanan.  Spectral curves and the generalised theta divisor.  Journal für die reine und angewandte Mathematik, 398:169–179, 1989\.
  * [Bot95] Francesco Bottacin.  Symplectic geometry on moduli spaces of stable pairs.  Annales scientifiques de l’École Normale Supérieure, Ser. 4, 28(4):391–433, 1995.
  * [Bou89] N. Bourbaki.  Algebra I. Chapters 1-3.  Springer-Verlag Berlin Heidelberg, 1989.
  * [Bra18] L. C. Branco.  Higgs bundles, Lagrangians and mirror symmetry.  arXiv e-prints, page arXiv:1803.01611, 2018.
  * [BS14] David Baraglia and Laura P. Schaposnik.  Higgs Bundles and (A, B, A)-Branes.  Communications in Mathematical Physics, 331(3):1271–1300, 2014\.
  * [BS16] David Baraglia and Laura P. Schaposnik.  Real structures on moduli spaces of Higgs bundles.  Advances in Theoretical and Mathematical Physics, 20(3):525–551, 2016.
  * [CL10] Pierre-Henri Chaudouard and Gèrard Laumon.  Le lemme fondamental pondèrè. I. Constructions gèomètriques.  Compositio Mathematica, 146(6):1416–1506, 2010.
  * [CL12] Pierre-Henri Chaudouard and Gèrard Laumon.  Le lemme fondamental pondèrè. II. Ènoncès cohomologiques.  Ann. of Math., 176(3):1647–1781, 2012.
  * [CL16] Pierre-Henri Chaudouard and Gérard Laumon.  Un théorème du support pour la fibration de Hitchin.  Annales de l’Institut Fourier, 66(2):711–727, 2016.
  * [CW17] Sebastian Casalaina-Martin and Jonathan Wise.  An introduction to moduli stacks, with a view towards Higgs bundles on algebraic curves.  arXiv e-prints, page arXiv:1708.08124, 2017.
  * [dC17] Mark Andrea de Cataldo.  A support theorem for the Hitchin fibration: the case of $SL_{n}$.  Compositio Mathematica, 153(6):1316–1347, 2017.
  * [dCHM12] Mark de Cataldo, Tamás Hausel, and Luca Migliorini.  Topology of Hitchin systems and Hodge theory of character varieties: the case $A_{1}$.  Annals of Mathematics, 175(3):1329–1407, 2012.
  * [dCMS19] Mark de Cataldo, Davesh Maulik, and Junliang Shen.  Hitchin fibrations, abelian surfaces, and the P=W conjecture.  arXiv e-prints, page arXiv:1909.11885, 2019.
  * [Don93] Ron Donagi.  Decomposition of spectral covers.  In Journées de géométrie algébrique d’Orsay - Juillet 1992, number 218 in Astérisque, pages 145–175. Société mathématique de France, 1993.
  * [DP12] Ron Donagi and Tony Pantev.  Langlands duality for Hitchin systems.  Invent.Math., 3(189):653––735, 2012.
  * [Dre04] Jean-Marc Drezet.  Faisceaux coherents sur les courbes multiples.  arXiv e-prints, page arXiv:math/0402387, 2004.
  * [EG95] David Eisenbud and Mark Green.  Clifford Indices of Ribbons.  Transactions of the American Mathematical Society, 347(3):757–765, 1995.
  * [EH06] D. Eisenbud and J. Harris.  The Geometry of Schemes.  Graduate Texts in Mathematics. Springer New York, 2006.
  * [Eis95] D. Eisenbud.  Commutative Algebra: With a View Toward Algebraic Geometry.  Graduate Texts in Mathematics. Springer, 1995.
  * [Gro61] A. Grothendieck.  Éléments de géométrie algébrique. II: Étude globale élémentaire de quelques classe de morphismes. .  Publ. Math., Inst. Hautes Étud. Sci., 8:5–228, 1961.
  * [Gro67] A. Grothendieck.  Éléments de géométrie algébrique. IV: Étude locale des schémas et des morphismes de schémas (Quatrième partie). Rédigé avec la colloboration de Jean Dieudonné.  Publ. Math., Inst. Hautes Étud. Sci., 32:1–361, 1967.
  * [GWZ17] Michael Groechenig, Dimitri Wyss, and Paul Ziegler.  Mirror symmetry for moduli spaces of Higgs bundles via p-adic integration.  arXiv e-prints, page arXiv:1707.06417, 2017.
  * [Har77] Robin Hartshorne.  Algebraic geometry.  Springer-Verlag, New York, 1977.  Graduate Texts in Mathematics, No. 52.
  * [Har86] Robin Hartshorne.  Generalized divisors on gorenstein curves and a theorem of noether.  Journal of Mathematics of Kyoto University, 26(3):375–386, 1986\.
  * [Har94] R. Hartshorne.  Generalized divisors on Gorenstein schemes.  K-Theory, 8(3):287–339, 1994.
  * [Har07] R. Hartshorne.  Generalized divisors and biliaison.  Illinois J. Math., 51(1):83–98, 01 2007.
  * [Hit87a] N. Hitchin.  Stable bundles and integrable systems.  Duke Mathematical Journal, 54(1):91–114, 1987.
  * [Hit87b] N. Hitchin.  The Self-Duality Equations on a Riemann Surface.  Proceedings of the London Mathematical Society, s3-55(1):59–126, 1987.
  * [Hit07] N. Hitchin.  Langlands Duality and G2 Spectral Curves.  The Quarterly Journal of Mathematics, 58(3):319–344, 2007.
  * [HL10] D. Huybrechts and M. Lehn.  The Geometry of Moduli Spaces of Sheaves.  Cambridge Mathematical Library. Cambridge University Press, 2 edition, 2010.
  * [HP12] T. Hausel and C. Pauly.  Prym varieties of spectral covers.  Geom. Topol., 16(3):1609–1638, 2012.
  * [HS93] Reinhold Hübl and Pramathanath Sastry.  Regular Differential Forms and Relative Duality.  American Journal of Mathematics, 115(4):749–787, 1993.
  * [HS14] Nigel Hitchin and Laura P. Schaposnik.  Nonabelianization of Higgs bundles.  J. Differential Geom., 97(1):79–89, 2014.
  * [HT03] T. Hausel and M. Thaddeus.  Mirror symmetry, Langlands duality, and the Hitchin system.  Inventiones Mathematicae, 153(1):197–229, Jul 2003.
  * [Kas15] Jesse Leo Kass.  Singular curves and their compactified Jacobians.  arXiv e-prints, page arXiv:1508.07644, 2015.
  * [Kle05] S. L. Kleiman.  The Picard scheme.  arXiv e-prints, page arXiv:math/0504020, 2005.
  * [KW07] A. Kapustin and E. Witten.  Electric-magnetic duality and the geometric Langlands program.  Commun. Number Theory Phys., 1(1):1–236, 2007.
  * [Lie06] Max Lieblich.  Remarks on the stack of coherent algebras.  arXiv e-prints, page arXiv:math/0603034, 2006.
  * [Lop04] Ana Cristina Lopez Martin.  Simpson Jacobians of reducible curves.  arXiv e-prints, page arXiv:math/0410393, 2004.
  * [Mar94] Eyal Markman.  Spectral curves and integrable systems.  Compositio Mathematica, 93(3), 1994.
  * [MRV17a] M. Melo, A. Rapagnetta, and F. Viviani.  Fine compactified Jacobians of reduced curves.  Transactions of the American Mathematical Society, 2017.
  * [MRV17b] M. Melo, A. Rapagnetta, and F. Viviani.  Fourier-Mukai and autoduality for compactified Jacobians. I.  Journal für die reine und angewandte Mathematik, 2017.
  * [Ngô06] Bao Châu Ngô.  Fibration de Hitchin et endoscopie.  Inventiones mathematicae, 164(2):399–453, 2006.
  * [Ngô10] Bao Châu Ngô.  Le lemme fondamental pour les algèbres de Lie.  Publications Mathématiques de l’IHÉS, 111:1–169, 2010.
  * [Nit92] N. Nitsure.  Moduli Space of Semistable Pairs on a Curve.  Proceedings of the London Mathematical Society, s3-62(2):275–300, 1992.
  * [Sav19] Michele Savarese.  Coherent Sheaves on Ribbons and their Moduli.  arXiv e-prints, page arxiv:1902.08510, 2019.
  * [Sch98] Daniel Schaub.  Courbes spectrales et compactifications de jacobiennes.  Mathematische Zeitschrift, 227(2):295–312, 1998.
  * [Sch13] L. P. Schaposnik.  Spectral data for G-Higgs bundles.  arXiv e-prints, page arXiv:1301.1981, 2013.
  * [Sch14] L. Schaposnik.  An introduction to spectral data for Higgs bundles.  arXiv e-prints, page arXiv:1408.0333, 2014.
  * [Sim92] Carlos Simpson.  Higgs bundles and local systems.  Publications Mathématiques de l’IHÉS, 75:5–95, 1992.
  * [Sim94a] Carlos Simpson.  Moduli of representations of the fundamental group of a smooth projective variety I.  Publications Mathématiques de l’IHÉS, 79:47–129, 1994.
  * [Sim94b] Carlos Simpson.  Moduli of representations of the fundamental group of a smooth projective variety II.  Publications Mathématiques de l’IHÉS, 80:5–79, 1994.
  * [Sta19] The Stacks project authors.  The Stacks project.  https://stacks.math.columbia.edu, 2019.
  * [SZ18] Junliang Shen and Zili Zhang.  Perverse filtrations, Hilbert schemes, and the P=W conjecture for parabolic Higgs bundles.  arXiv e-prints, page arXiv:1810.05330, 2018.
  * [Vas04] W. Vasconcelos.  Computational Methods in Commutative Algebra and Algebraic Geometry.  Algorithms and Computation in Mathematics. Springer Berlin Heidelberg, 2004.
  * [Wit15] Edward Witten.  More On Gauge Theory And Geometric Langlands.  arXiv e-prints, page arXiv:1506.04293, 2015.


# Information Interactions in Outcome Prediction: Quantification and
Interpretation using Stochastic Block Models

Gaël Poux-Médard  Université de Lyon, Lyon 2, UR 30835 avenue Pierre Mendès
FranceBronFrance<EMAIL_ADDRESS>,  Julien Velcin  Université
de Lyon, Lyon 2, UR 30835 avenue Pierre Mendès FranceBronFrance
<EMAIL_ADDRESS>and  Sabine Loudcher  Université de Lyon, Lyon 2,
UR 30835 avenue Pierre Mendès FranceBronFrance<EMAIL_ADDRESS>
(2021)

###### Abstract.

In most real-world applications, it is seldom the case that a result appears
independently from an environment. In social networks, users’ behavior results
from the people they interact with, news in their feed, or trending topics. In
natural language, the meaning of phrases emerges from the combination of
words. In general medicine, a diagnosis is established on the basis of the
interaction of symptoms. Here, we propose the Interacting Mixed Membership
Stochastic Block Model (IMMSBM), which investigates the role of interactions
between entities (hashtags, words, memes, etc.) and quantifies their
importance within the aforementioned corpora. We find that in inference tasks,
taking them into account leads to average relative changes with respect to
non-interacting models of up to 150% in the probability of an outcome and
greatly improves the predictions performances. Furthermore, their role greatly
improves the predictive power of the model. Our findings suggest that
neglecting interactions when modeling real-world phenomena might lead to
incorrect conclusions being drawn.

Interaction modeling, Block models, Information diffusion, Recommender systems

††journalyear: 2021††copyright: acmcopyright††conference: Fifteenth ACM
Conference on Recommender Systems; September 27-October 1, 2021; Amsterdam,
Netherlands††booktitle: Fifteenth ACM Conference on Recommender Systems
(RecSys ’21), September 27-October 1, 2021, Amsterdam, Netherlands††price:
15.00††doi: 10.1145/3460231.3474254††isbn: 978-1-4503-8458-2/21/09††ccs:
Information systems Social recommendation††ccs: Information systems
Clustering††ccs: Mathematics of computing Probabilistic inference problems

##  1\. Introduction

With the Internet, people have begun to interact with each other as never
before. Nowadays, social networks such as Facebook, Reddit or WhatsApp let us
share and compare ideas. Modeling the dynamics of these exchanges can help us
understanding why and how various pieces of information (e.g., hashtags,
memes, ideas, etc.) will flow through a community. We refer to these pieces of
information as entities. Understanding the underlying dynamics at work
provides powerful means to predict and control entities’ spread. Up to now,
little work has been done on investigating the role of interacting entities in
users’ decisions (retweet, share, comment answer, etc.), or more generally in
the probability of outcomes. A number of previous works on information
diffusion theory only consider a user acting on an isolated piece of
information (Pastor-Satorras et al., 2015; Poux-Médard et al., 2020). On some
occasions, theoretical frameworks have been developed to investigate how the
presence of concurrent pieces of information affect the action a user exerts
on them in a network (Beutel et al., 2012). However, a fundamental question to
be answered is how pieces of information interact in the informational
landscape. Considering a recommender system, a customer that bought a
smartphone might be interested by side accessories such as headphones or
selfies sticks, but a customer that bought a smartphone and a camera lens
extension might be more interested in buying a professional camera. Another
example considering a disease-symptoms corpus, the diagnosis of a disease is
established from the interaction of several symptoms; the interaction between
“running nose” and “headache” symptoms is likely to lead to a “flu” diagnosis.
An approach without interactions would be less successful here, since each of
the products/symptoms can lead to a number of different
recommendations/diagnoses. The same line of reasoning can be applied to the
prediction of retweets (user exposures to A and B affects its retweet of C),
music playlist building (same as before), detection of controversial posts
(what combinations of words trigger what answers), etc. Beyond the state-of-
the-art guess that a single piece of information is enough to trigger another
one (in music recommendation for instance, a user might associate “ACDC”
$\rightarrow$ “Metallica”, or “Chopin” $\rightarrow$ “Mozart”) (Myers and
Leskovec, 2012), we believe that the world is more complex, and that such a
mechanism should not be so simplistic. Our idea in the present work is to take
the interaction of pairs of entities into account in the prediction of an
outcome (“ACDC” + “Metallica” $\rightarrow$ “Pink Floyd”). Therefore,
throughout the present article we develop a novel prediction method with
multiple entity types.

The remainder of this article is organized as follows. In Section 2, we
present an overview of the landscape of information interaction studies for
both information spread and recommender systems. Then in Section 3, we detail
our model and its mathematical derivation and describe an efficient algorithm
to apply it. Next, in Section 4, we assess the model on real-world datasets.
Finally, in Section 5, we quantify the role of interactions in each corpus and
provide insight to analyzing the results from a semantic point of view.

##  2\. Background

###  2.1. Interaction in information diffusion

Much previous research was aimed at understanding the mechanisms of
information propagation. Many studies focus on the importance of the network
structure coupled with the intrinsic contagiousness, or virality, of the
spreading entity (that depends on its nature and content (Nigam et al., 2019;
Romero et al., 2011)) in the modeling of the spreading processes (Kempe et
al., 2015; Rodriguez et al., 2013).

Recent works proposed to model the diffusion of information as the result of
an interaction between information (Beutel et al., 2012; Poux-Medard et al.,
2021). Following a similar idea, Myers and Leskovec (Myers and Leskovec, 2012)
investigated interactions between contagions on Twitter. The authors aim to
find the interaction factors between different tweets in one’s activity feed.
Their findings suggest that interactions between tweets play a determinant
role in their retweet. In their work, they assume that there is an inherent
virality for every tweet (that is an inherent probability to be retweeted)
computed from the frequency of retweets, to which is added a small interaction
term.

While this latter work opens paths for studying interactions, it also presents
various limitations that we address here. Firstly, the method proposed by the
authors makes predictions solely based on tweets that have been observed in
the feed of a given user. It therefore limits the application range of the
model uniquely to systems based on the retweets (or share) concept, where
information has to appear first in order to be spread. This model is hardly
applicable to systems that are based on exogenous reactions (e.g., online
forums, playlist building and recommender systems) where information can
appear as a consequence of different entities (“Capital” + “Netherlands”
$\rightarrow$ “Amsterdam”). We address this problem by developing a more
global framework allowing for outputs that are different from the inputs in
the considered datasets. Secondly, interaction is defined as a correction to
the frequency of retweets of a given tweet in any context. We argue this can
lead to false conclusions about interactions. Imagine that interactions lead
to a shift of $\Delta p$ on the base probability $p$ (virality) of a retweet,
and that this interaction happens in a fraction $f$ of all observations of a
given tweet being retweeted. The virality as defined in (Myers and Leskovec,
2012) then equals $p(1-f)+(p+\Delta p)f=p+f\Delta p$, which is by construct
larger than the actual probability of a retweet in the absence of interaction.
Therefore, defining interaction according to this quantity is wrong (for
instance it results in adding an interaction term $f\Delta p$ when there is in
fact no interaction). Virality needs to be inferred by the model at the same
time as the contribution of interactions to be properly defined, which is part
of what we propose to do here.

###  2.2. A recommender system approach

Research in recommender systems applied to multiple pieces of information is
motivated by numerous descriptive studies on multimodal networks structure
(Sun and Han, 2012; Shi et al., 2016; Rashed et al., 2020; Huan et al., 2017).
Typically in (Sun and Han, 2012), the authors study interaction between
multiple entity types via a heterogeneous network representation and define
clusters of entities based on structural properties of the resulting graph.
However, as pointed out by the authors, this method is heavily influenced by
the structural clustering method used –in this case a meta-path-based
clustering (Sun, 2011). Moreover, defining edge weights in heterogeneous
graphs is subjective and requires additional learning algorithms.

A more direct representation of real-world systems is based on collaborative
filtering techniques, that directly mines clusters from pieces of information
interaction patterns and generates a weighted interaction graph between
entities from unit independent observations. Typically, a widely used method
in commercial applications is based on a Matrix Factorization approach (Koren
et al., 2009). This method considers a large number of user-item pairs and
identifies regularities to model them in a lower dimensional space (e.g., it
groups regularities into clusters). It has been shown that this algorithm is
in fact a particular case of a wider model family: the Mixed Membership
Stochastic Block Model (MMSBM) (E.M. et al., 2008; M. et al., 2006). Another
particular case of this family is the single membership Stochastic Block Model
(SBM) (Wang and Wong, 1987), whose use in the discovery of underlying
interaction dynamics has been suggested in recent years (Guimerà and Sales-
Pardo, 2013).

More recently, in (Godoy-Lorite et al., 2016) the authors develop an extension
of the MMSBM considering bipartite graphs. This formulation generalizes MF
models into a more global framework based on the MMSBM. In this approach,
entities of different type (e.g. users and items in the case of online
shopping recommendation) are grouped into distinct clusters whose interaction
result in an outcome (e.g. buy or not buy). This model is optimized via a
scalable EM algorithm; it outperforms state-of-the-art models such as SBM
(Wang and Wong, 1987; Guimerà and Sales-Pardo, 2013), MF (Koren et al., 2009)
and Mixed-Membership MF (Mackey et al., 2010) both in performing predictions
and in scalability.

However, none of the cited works consider the interaction between pieces of
information of the same nature in the prediction of an outcome. A drug might
interact with another one, but the joint interaction of two drugs on a third
one cannot be investigated (Guimerà and Sales-Pardo, 2013). A user on Netflix
is predicted to like a given movie because she is partially part of the group
that liked the movie A and partially part of the group that disliked movie B,
but all the user groups are independent from one another (Koren et al., 2009;
Godoy-Lorite et al., 2016). Friendship between individuals is determined on
the basis of the independent groups of friends they belong to, but not on the
basis of the joint belonging to various groups (E.M. et al., 2008; M. et al.,
2006; Jamali et al., 2011). Typically in (Godoy-Lorite et al., 2016),
embedding pieces of information of same nature in a bipartite graph seems
irrelevant: taking the example of a online purchase recommendation system, a
product should not interact differently with an other (or belong to different
clusters) because it is on the left or the right side of a bipartite graph.
Instead, we want to enforce a symmetrical interaction and thus a single
clustering for pieces of information that interact in this way.

Relaxing the independence assumption and looking at the probability of an
outcome due to the joint membership to more than one group (while also
allowing for outcomes resulting that do not result from any interaction)
provides a better assumption for modeling the subtle interaction process.
Taking back the previous example of medical diagnosis: a disease can seldom be
diagnosed on the basis of a single symptom (e.g. without interaction between
symptoms), but rather on a symmetric combination of symptoms (e.g. with
interaction).

###  2.3. Contributions

  * •

We develop a scalable model that accounts for interactions between entities:
the Interacting Mixed Membership Stochastic Block Model (IMMSBM).

  * •

We show by comparing our results with non-interacting MMSBM (Godoy-Lorite et
al., 2016) that taking interaction into account leads to a more accurate
recommendations in real-world situations on 4 different datasets.

  * •

We provide a proper way to infer virality, which allows to correctly define
and compute interaction terms.

  * •

We quantify the role of interactions in various corpus and highlight the
necessity of taking them into account when dealing with real-world spreading
or recommendation problems.

##  3\. The IMMSBM model

In this section, we develop the IMMSBM model. We propose an approach based on
standard Mixed Membership Stochastic Block Modeling (E.M. et al., 2008; Godoy-
Lorite et al., 2016), which we modify in order to take interactions into
account. Building our model on the MMSBM allows to assume that each entity
does not have only one membership, which is in line with the real situation.
The IMMSBM requires no prior information on the system and its numerical
implementation is possible via a scalable Expectation-Maximization algorithm
of linear complexity with the size of the dataset. In addition to the state-
of-the-art problems which our method answers, it also offers better predictive
power than non-interacting baselines.

The goal of the model is to predict the most likely result of an interaction
between two entities (i and j in Fig.1). In recommender systems, it would
recommend the song the most likely to fit user’s tastes given previous songs
they listened to, or the product the most likely to be bought given a user’s
purchase history. Another example about assisted medical diagnosis: observing
the words “fatigue” and “cough” in a medical report is more likely to imply
the observation of “flu” than “anemia”, despite “anemia” being often
associated with the “fatigue” symptom. The model will group data into clusters
(membership matrix $\theta$ Fig.1) that interact symmetrically with each other
(interactions tensor p), resulting in a probability over the possible outputs
to appear (histograms Fig.1-top). We have no prior knowledge of the content of
the groups, and we only need to set the number of clusters T.

(a)  (b)

Figure 1. Illustration of the model – (Top) Schema of IMMSBM for a single pair
of entities $(i,j)$ (which could be “fever” and “cough” for instance). Input
entities are grouped into T clusters in different proportions; the proportion
to which they belong to each cluster is quantified by a $\theta$ matrix
(dimension $[I\times T]$ where $I$ is the input space). The clusters then
interact to generate a probability distribution over the output entities
defined by the interactions tensor p (dimension $[T\times T\times O]$ where
$O$ is the output space). (Bottom) Alternative representation of the IMMSBM as
a graphical model. To generate each output, for each observation ($i$, $j$,
$x$) in the set $R^{\circ}$, a cluster ($k$ and $l$) is drawn for each input
entity ($i$, $j$) from a distribution encoded in the matrix $\theta$. The
generated output $x$ is drawn from a multinomial distribution conditioned by
the previously drawn clusters k and l encoded in $\vec{p}$.

###  3.1. Interacting MMSBM

We refer to the interacting entities as input entities $(i,j)\in I^{2}$, and
aim to predict an output entity $x\in O$. $I$ is the input space (the entities
that interact: products, symptoms or songs for instance) and $O$ is the output
space (the entities resulting from the interaction in an answer, diagnosed
diseases for instance). We illustrate in Figure 1-top how the input space and
output space are related according to our model: input entities are clustered,
and the interaction between those clusters give rise to probability
distributions over the output entities. Note that $I$ and $O$ can be identical
or radically different according to what we want to model. In the case of
musical recommendation, $I$ accounts for observed artists in a user’s feed,
and $O$ for recommended artists –that do not necessarily appears in the user’s
feed, unlike (Myers and Leskovec, 2012). In the case of medical diagnosis, $I$
accounts for symptoms (fever, cough, anemia, etc.) and $O$ for diseases
(Alzheimer, hepatitis, etc.). If one wants to predict named entities in the
answer to a Reddit post based on interacting named entities in the original
post, then $I=O$ since the predicted vocabulary can also be used as an input
(e.g., the named entity “Sun” can either be an input that interacts with other
inputs, or an output). As an alternative visualization, we present the
graphical generative model of the IMMSBM in Figure 1-bottom. The observed data
then takes the form of triplets ($i$, $j$, $x$) signifying that the combined
presence of input entities i and j leads to the output entity x. A given
triplet can obviously occur several times in the same dataset.

We assume there are regularities in the studied dataset, so that given inputs
interact with each other in the same way. Their classification into clusters
would therefore be relevant. For the medical dataset example, this means that
symptoms such as “fever” and “pallor” often come in pairs and therefore are
considered as similar regarding the diagnosis; they would belong to the same
cluster. We define the membership matrix $\theta$ associating each input
entity with clusters in different proportions, such that $\theta_{i}$ is a
$[1\times T]$ vector with $\sum_{t}^{T}\theta_{i,t}=1$. Note that unlike in
single membership stochastic block models an entity does not have to belong to
only one group (Funke and Becker, 2019). Given the possible semantic variation
of entities (polysemy of words in natural languages -e.g. “like”, “swallow”-,
symptoms with various causes in medicine -“headache”, “fever”-, etc.), an
approach via a mixed-membership clustering seems more relevant.

We then define the cluster interactions tensor $p_{k,l}(X_{k,l}=x)$ of
dimensions $[T\times T\times O]$ as the probability that the interaction
between clusters $k\in T$ and $l\in T$ gives rise to the output $x\in O$. A
property of this definition is that $\sum_{x}p_{k,l}(X_{k,l}=x)=1\ \forall
k,l$. The role of those two quantities is schematized in Fig.1.

We choose to consider only one membership matrix $\theta$ for all of the
inputs, instead of one per input entry as in (Godoy-Lorite et al., 2016). It
implies the assumption that interactions to be symmetric, which means that an
observation ($i$, $j$, $x$) is equivalent to ($j$, $i$, $x$). This follows the
idea of (Beutel et al., 2012) where it is assumed that the interaction between
two viruses is symmetric, meaning the interaction influences both viruses with
the same magnitude. Therefore, there is no need to consider a different
clustering for inputs i and j, which motivates the use of a single membership
matrix $\theta$.

We now propose to define the entities interactions tensor
$P_{i,j}(X_{i,j}=x)$, representing the probability that the interaction
between inputs i and j implies the output x as:

(1) $P_{i,j}(X_{i,j}=x)=\sum_{k,l}\theta_{i,k}\theta_{j,l}p_{k,l}(X_{k,l}=x)$

For the sake of brevity, from now on we will refer to $p_{k,l}(X_{k,l}=x)$ as
$p_{k,l}(x)$. We define the likelihood of the observations given the
parameters as:

(2) $P(R^{\circ}|\theta,p)=\prod_{(i,j,x)\in
R^{\circ}}\sum_{k,l}\theta_{i,k}\theta_{j,l}p_{k,l}(x)$

where $R^{\circ}$ denotes the set of triplets in the training set (input,
input, output). Note that the remaining triplets $R\setminus R^{\circ}$ are
used as test set.

Importantly, this formulation differs from related work in that the model
presented here considers a symmetric interaction between pieces of information
in the prediction of an outcome instead of their asymmetric combination
(Godoy-Lorite et al., 2016). Considering a musics recommender system for
instance, clustering the same songs using two independent membership matrices
(which is the formulation introduced in (Godoy-Lorite et al., 2016)) would
make few sense and might eventually lead to incorrect results. Indeed, since
the convergence towards a global optimum is not guaranteed, such algorithm
might assign different memberships to a song depending on whether it is on the
left or right side of a triplet. Put differently, we would have different
output probabilities for $(i,j,x)$ and $(j,i,x)$, which is something we avoid
here. In our formulation, only the outputs can be of different nature than the
inputs.

###  3.2. Inference of the parameters

We are now looking at inferring the model parameters, namely the components of
the interactions tensor $P_{i,j}(X_{i,j}=x)$. As stated in the previous
section, we decomposed this tensor into an algebraic combination of the matrix
$\theta$ and the tensor $p$. In this section, we introduce a method to infer
them via an Expectation-Maximization (EM) algorithm. EM is a 2-step iterative
algorithm. The first step consists of computing the expectation of the
likelihood with respect to the latent variables, denoted
$\omega_{i,j,x}(k,l)$, with parameters $\theta$, p set as constant. The second
step consists of maximizing this expectation of the likelihood with respect to
the parameters $\theta$, p. Iterating this process guarantees that the
likelihood converges towards a local maximum.

####  3.2.1. Expectation step

Taking the logarithm of the likelihood as defined in Eq.2, denoted $\ell$, we
have:

(3) $\begin{split}\ell&=\sum_{(i,j,x)\in
R^{\circ}}\ln\sum_{k,l}\theta_{i,k}\theta_{j,l}p_{k,l}(x)\\\
&=\sum_{(i,j,x)\in
R^{\circ}}\ln\sum_{k,l}\omega_{i,j,x}(k,l)\frac{\theta_{i,k}\theta_{j,l}p_{k,l}(x)}{\omega_{i,j,x}(k,l)}\\\
&\geq\sum_{(i,j,x)\in
R^{\circ}}\sum_{k,l}\omega_{i,j,x}(k,l)\ln\frac{\theta_{i,k}\theta_{j,l}p_{k,l}(x)}{\omega_{i,j,x}(k,l)}\end{split}$

We used Jensen’s inequality to go from the 2nd to 3rd line. The inequality in
Eq.3 becomes an equality for:

(4)
$\omega_{i,j,x}(k,l)=\frac{\theta_{i,k}\theta_{j,l}p_{k,l}(x)}{\sum_{k^{\prime},l^{\prime}}\theta_{i,k^{\prime}}\theta_{j,k^{\prime}}p_{k^{\prime},l^{\prime}}(x)}$

where $\omega_{i,j,x}(k,l)$ is interpreted as the probability that the
observation $(i,j,x)$ is due to i belonging to the group $k$ and j to $l$,
that is the expectation of the likelihood of the observation $(i,j,x)$ with
respect to the latent variables $k$ and $l$. Therefore, Eq.4 is the formula
for the expectation step of the EM algorithm. An alternative derivation of the
expectation step formula is discussed in (Bishop, 2006).

####  3.2.2. Maximization step

This step consists in maximizing the likelihood using the parameters of the
model $\theta$ and p, independently of the latent variables. In order to take
into account the normalization constraints, we introduce the Lagrange
multipliers $\phi$ and $\psi$. Following this, the constrained log-likelihood
reads:

(5)
$\ell_{c}=\ell-\sum_{i}(\phi_{i}\sum_{t}\theta_{i,t}-1)-\sum_{k,l}(\psi_{k,l}\sum_{x}p_{k,l}(x)-1)$

We first maximize $\ell_{c}$ with respect to each entry $\theta_{mn}$.

(6)
$\begin{split}\frac{\partial\ell_{c}}{\partial\theta_{mn}}=&\frac{\partial\ell}{\partial\theta_{mn}}-\phi_{m}=0\\\
=&\sum_{\partial
m}\left(\sum_{l}\frac{\omega_{m,j,x}(n,l)}{\theta_{mn}}+\sum_{k}\frac{\omega_{j,m,x}(k,n)}{\theta_{mn}}\right)-\phi_{m}\\\
=&\frac{1}{\theta_{mn}}\sum_{\partial
m}\left(\sum_{t}\omega_{m,j,x}(n,t)+\omega_{j,m,x}(t,n)\right)-\phi_{m}\\\
\Leftrightarrow\ \theta_{mn}=&\frac{1}{\phi_{m}}\sum_{\partial
m}\sum_{t}\omega_{m,j,x}(n,t)+\omega_{j,m,x}(t,n)\end{split}$

where $\partial m$ stands for the set of observations in which the entry m
appears $\\{(j,x)|(m,j,x)\in R^{\circ}\\}$. Note that summing over this set in
line 2 of Eq.6 implies that the relation between two inputs entities is
symmetric –if $(i,j,x)\in R^{\circ}\ \text{ implies }(j,i,x)\in R^{\circ}$.
Summing the last line of Eq.6 over $n\in T$ then multiplying it by $\phi_{m}$,
we get an expression for $\phi_{m}$:

(7) $\phi_{m}\sum_{n}\theta_{mn}=\phi_{m}=\sum_{\partial
m}\sum_{t,n}\omega_{m,j,x}(n,t)+\omega_{i,m,x}(t,n)=\sum_{\partial m}2=2\cdot
n_{m}$

Where $n_{m}$ is the total number of times that m appears as an input in
$R^{\circ}$. Finally, plugging back this result in Eq.6, we get:

(8) $\theta_{mn}=\frac{\sum_{\partial
m}\left(\sum_{t}\omega_{m,j,x}(n,t)+\omega_{i,m,x}(t,n)\right)}{2.n_{m}}$

Following the same line of reasoning for p, we get:

(9) $p_{r,s}(l)=\frac{\sum_{\partial
l}\omega_{i,j,l}(r,s)}{\sum_{(i,j,l^{\prime})\in
R^{\circ}}\omega_{i,j,l^{\prime}}(r,s)}$

The set of Equations 8 and 9 constitutes the maximization step of the EM
algorithm. They hold only if the input entities interactions are symmetric
(e.g. when $\\{(j,x)|(m,j,x)\in R^{\circ}\\}=\\{(i,x)|(i,m,x)\in
R^{\circ}\\}$), which is what we aimed to do. It is worth noting that the
proposed algorithm offers linear complexity with the size of the dataset
$\mathcal{O}(|R^{\circ}|)$ provided the number of clusters is constant, while
guaranteeing convergence to a local maximum.

Figure 2. Performance variations on all the metrics for every dataset
considered. Dashed blue line: upper limit to performances ; blue line: IMMSBM
; yellow dashed line: MMSBM ; red line: naive baseline. Top left: PubMed ; top
right: Spotify ; bottom left: Twitter ; bottom right: Reddit. The vertical
green line shows the selected number of clusters; it is chosen using the AIC
criterion, which matches with the elbow of the various metrics considered.

##  4\. Experiments

###  4.1. Datasets and evaluation protocol

We test the performance of our model on 4 different datasets. The first
dataset (PubMed dataset) is built with 15,809,271 medical reports collected
from the PubMed database as a good approximation for human-disease network
(Zhou et al., 2014). This dataset is not explicitly about recommender systems,
but provides an intuitive way to understand how our recommendation approach
works by suggesting likely diseases given a collection of symptoms. The second
(Twitter dataset) with 139,098 retweets gathered in october 2010 associated
with the 3 last tweets in the feed preceding each retweet (Hodas and Lerman,
2014). The task is to infer tweets a user is the most likely to retweet. A
possible application would be a personnalized recommendation of such tweets in
the “Trends for you” Twitter section. The third (Reddit dataset) with the
entirety of posts in the subreddit r/news in May 2019 (225,485 message-answer
relationships in total). We aim at predicting the content of the answer given
the incoming message. A possible application would be similar as what Gmail
does when suggesting automated answers to an email given keywords present in
the message. Finally, the last (Spotify dataset) is built with 2,000 music
playlists associated with keywords “english” and “rock” of random Spotify
users. We predict the next song a user will add to a playlist given this
user’s history. A RS application would suggest a ranked list of songs to the
user is likely to add to a playlist. Each dataset is formed by associating
every pair of inputs in a message (i.e. a list of symptoms, a user’s feed, a
Reddit post, and a playlist’s last artists) with an answer (i.e. a disease, a
retweet, a Reddit answer, an artist added to a playlist). The building process
of datasets is further described in SI, Section 1, together with direct links
to access them for possible replication studies 111To ease eventual
replication studies, we provide the Python implementation of the IMMSBM and of
the baseline model used in this article in a Git repository at
https://github.com/GaelPouxMedard/IMMSBM..

From the raw datasets, we form the test set by randomly sampling 10% of the
coupled structures (message$\rightarrow$answer). The 90% coupled structures
left are used as a training set. The number of clusters is determined using
the elbow method. For each corpus: 30 for PubMed, 15 for Twitter, 30 for
Reddit, and 15 for Spotify. We perform 100 independent runs, each with
independent random initialization of the parameters $\theta$ and $p$. The EM
loop stops once the relative variation of the likelihood between two
iterations is less than 0.001%.

###  4.2. Baselines

####  4.2.1. Naive baseline

The naive baseline is simply the frequency of the outputs in the test set.
This naive classifier predicts the value of every output independently of the
inputs.

####  4.2.2. MMSBM

We use the classical MMSBM as a second baseline. In this formulation
interactions are not taken into account and is identical to state-of-the-art
work done in recommender systems (E.M. et al., 2008; M. et al., 2006; Godoy-
Lorite et al., 2016). Instead of considering triplets (input, input, output),
we instead here apply the classical MMSBM on pairs (input, output). This
baseline is exactly the same as the model introduced in (E.M. et al., 2008).
We then train our baseline whose log-likelihood is defined as
$\ell_{BL}=\sum_{(i,x)\in R^{\circ}}\ln\sum_{k}\theta_{ik}p_{k}(x)$ on the
same datasets as in the main experiments. We infer the parameters via an
Expectation-Maximization algorithm similar to the one described in Section 1.
The number of clusters being the only parameter to be tuned. It is chosen in
the same way as it is for the IMMSBM –by minimizing the AIC criterion (Akaike,
1973). We see further in the experiments that this criterion matches with the
elbow of the various metrics. We make 100 independent runs with random
initialization and discuss here the results of the highest-likelihood run.
This baseline provides an alternative way to quantify the importance of
interactions by comparison with the case where these are taken into account.
We expect the baseline model to find results that are equivalent to the
diagonal probabilities of the main model (i.e. similar to
$P_{i,i}(x)=\sum_{k,l}\theta_{i,k}\theta_{i,l}p_{k,l}(x)$). This is because
the diagonal $P_{i,i}(x)$ is supposed to account for the apparition of x given
only the presence of i, which is what this baseline computes. Furthermore,
this model will provide insight to the generalization of the assumption made
in Myers and Leskovec (Myers and Leskovec, 2012), stating that the probability
of an output is greatly dominated by a prior quantity unrelated to any
interaction (known as virality, see Section 2).

####  4.2.3. Perfect modeling - Upper limit to prediction

We also compare our results to a mathematical upper limit to our predictions.
While in some systems the predictions could theoretically be perfect, in most
situations the dataset simply does not allow it. Consider as an example a case
where the test set contains twice the triplet (“fever”, “pallor”, “influenza”)
and once the triplet (“fever”, “pallor”, “anemia”): the model cannot make a
prediction better than 66%. In SI, Section 2, we develop a general method to
derive such an upper limit to predictions for a given dataset. This upper
limit is mathematically the best performance any model can do given a dataset
structure.

###  4.3. Results

The metrics we use to assess the performance of our model are the max-F1
score, the area under the receiver operating characteristic curve (ROC) curve
and the precision@10. For all of these quantities, the closer to 1 they are,
the better the performance.

For evaluation, we adopt a guessing process as follows. For every pair of
inputs, we compute the probability vector for the presence of every possible
output. Then we predict all of the probabilities larger than a given threshold
to be “present”, and all the others to be “absent”. Comparing those
predictions with the observations in the test set, we get the confusion matrix
for the given threshold. We then lower the threshold and repeat the process to
compute the various metrics.

We recall that we do not compare our approach to (Myers and Leskovec, 2012)
because its formulation does not allow to make prediction on exogenous output,
that is when the output is not part of the input pair. The more general model
presented here is needed in order to consider the effect of interactions on
the probability of exogenous outputs.

Table 1. Experimental results for the four metrics considered, from each model applied to each corpus. We see that our model outperforms the proposed baselines in every dataset for almost every evaluation metric – the error bars overlap for the AUC (ROC) on the PubMed corpus. The given error corresponds to the standard deviation over the 10 runs. The naive baseline and upper limit results are constant over the runs and therefore have no variance.  |  |  P@10 |  Max-F1 |  AUC (ROC) |
---|---|---|---|---|---
PubMed |  Naive |  0.212 |  0.160 |  0.863 |
MMSBM |  0.627 $\pm$ 0.002 |  0.393 $\pm$ 0.002 |  0.911 $\pm$ 0.000 |
IMMSBM |  0.656 $\pm$ 0.001 |  0.411 $\pm$ 0.001 |  0.911 $\pm$ 0.002 |
Up.lim. |  0.668 |  0.450 |  0.936 |
Twitter |  Naive |  0.462 |  0.147 |  0.554 |
MMSBM |  0.529 $\pm$ 0.005 |  0.254 $\pm$ 0.005 |  0.741 $\pm$ 0.004 |
IMMSBM |  0.610 $\pm$ 0.004 |  0.349 $\pm$ 0.006 |  0.800 $\pm$ 0.001 |
Up.lim. |  0.737 |  0.748 |  0.959 |
Reddit |  Naive |  0.488 |  0.164 |  0.660 |
MMSBM |  0.495 $\pm$ 0.000 |  0.177 $\pm$ 0.000 |  0.686 $\pm$ 0.000 |
IMMSBM |  0.499 $\pm$ 0.000 |  0.181 $\pm$ 0.000 |  0.687 $\pm$ 0.000 |
Up.lim. |  0.558 |  0.582 |  0.933 |
Spotify |  Naive |  0.355 |  0.088 |  0.573 |
MMSBM |  0.426 $\pm$ 0.006 |  0.167 $\pm$ 0.003 |  0.707 $\pm$ 0.002 |
IMMSBM |  0.502 $\pm$ 0.006 |  0.228 $\pm$ 0.005 |  0.723 $\pm$ 0.002 |
Up.lim. |  0.570 |  0.607 |  0.944 |

In Table 1, we show the performances of our model compared with the baselines
introduced in Section 4.2. We see that IMMSBM outperforms all of the baselines
in most cases. A remarkable feature is the good P@10 of our model, primordial
in many applications such as diseases diagnosis or in recommender systems. As
expected, taking interactions between entities into account leads to an
improved accuracy on the prediction of the missing data. This correlates to
the conclusions drawn in Myers and Leskovec, stating the importance of
interactions in real-world phenomenon modeling (Myers and Leskovec, 2012).
Overall, our model performs better in many different applications for most of
the corpora considered. It systematically yields a better P@10 and F1 score,
making it of use in recommender systems applications. The AUC (ROC) score can
be interpreted as the quality of classification of low-probability events.
Overall, little improvement is observed on this aspect compared to the non-
interacting baseline.

We notice however that accounting for interactions did not lead to a
significant improvement in performance over the two baselines on the Reddit
corpus. Given the obvious fact that language is partially formed by
interacting named entities, this result may look surprising. We attribute the
lack of improvement when considering interactions to the dataset provided.
Indeed the dataset contains few observations for every possible pair, due to
the wide range of available vocabulary of natural language (Loreto et al.,
2016). It is likely that we did not train the model with enough data for it to
learn actual regularities in pair interactions. This can also be seen during
the building of the test set: approximately one half of its pairs have never
been observed in the training set. Therefore, we suppose that the model simply
lacks enough data to identify generalities. In future work, it might be
interesting to answer this problem by considering a corpus of pre-clustered
entities instead of independent named entities, hence reducing the vocabulary
range and adding to the regularity of the dataset.

##  5\. Discussion

Figure 3. Importance of interactions \- Contribution of each pair of clusters
$V_{k,l}$ (heatmap) and average impact of the interactions $\bar{V}$ (on the
right) in outcome probabilities for each corpus. Clusters typically interact
with a limited number of others; these interactions still play a significant
role in outcomes probabilities. The cluster have been annotated manually.

###  5.1. Relative importance of interactions

####  5.1.1. Global importance of interactions

We recall the argument exposed in the introduction that interactions have to
be inferred along with the virality of a piece of information, which cannot be
calculated heuristically. Imagine that interactions lead to a shift of $\Delta
p$ on the base probability $p$ (virality) of a retweet, and that this
interaction happens in a fraction $f$ of all observations of a given tweet
being retweeted. The virality as defined in (Myers and Leskovec, 2012) then
equals $p(1-f)+(p+\Delta p)f=p+f\Delta p$, which is by construct larger than
the actual probability of a retweet in the absence of interaction. Therefore,
defining interaction according to this quantity is wrong (for instance it
results in adding an interaction term $f\Delta p$ when there is in fact no
interaction). Virality needs to be inferred by the model at the same time as
the contribution of interactions to be properly defined.

Our model infers virality along with interaction terms and yield better
results than state-of-the-art methods (see Table 1), which provides solid
ground for analyzing the effect of information interaction. Close analysis of
interactions between pieces of informations has been little considered in
literature –what lexical fields, groups of symptoms, musical genres, kind of
tweets interact with each other. We can evaluate the importance of
interactions between entities on the basis of inferred virality. We consider
two quantities within each corpus: the overall relative impact of the
interactions on the probability of an outcome and the contribution of each
pair of groups in the modification of outcome probabilities.

To evaluate the global impact of taking the interactions into account, we
compute the relative change of probability according to the virality for each
triplet and average this quantity over all the triplets in the corpus. We note
this quantity $\bar{V}$:

$\bar{V}=\frac{1}{|R^{\circ}|}\sum_{(i,j,x)\in
R^{\circ}}\frac{|P_{i,i}(x)-P_{i,j}(x)|}{P_{i,i}(x)}$

where $P_{i,j}(x)=\sum_{k,l}\theta_{i,k}\theta_{j,l}p_{k,l}(x)$ denotes the
probability of outcome $x$ given the entities $i$ and $j$; as shown in the
previous section, the diagonal elements $P_{i,i}(x)$ account for the virality
of $i$ on $x$. The results are shown Fig.3. It results that for every corpus,
interaction between entities exerts a non-negligible influence on the
probability of an output. Those results confirm previous work done on
interactions modeling, stating the importance of taking interactions into
account when analyzing real-world datasets (Myers and Leskovec, 2012).
Interactions increase the virality of an output by a factor by 2.58 in the
PubMed corpus, 2.78 in the Twitter corpus, 1.73 in the Spotify corpus and 1.57
in the Reddit corpus. Interactions have a greater effect on output
probabilities for PubMed and Twitter corpora, and a lesser role for the
Spotify and Reddit ones. Besides, our model applied to a dataset where
interactions do not play any role ($\bar{V}=0$) reduces to the non-interacting
MMSBM baseline. This metric therefore allows to assert the importance of the
interactions in a given corpus.

####  5.1.2. Which clusters interact

To evaluate the role of each pair of groups plays in the modification of
output probability, we consider the following quantity:

$V_{k,l}=\frac{\sum_{(i,j,x)\in
R^{\circ}}\theta_{i,k}\theta_{j,l}(p_{k,l}(x)-P_{i,i}(x))}{\sum_{(i,j,x)\in
R^{\circ}}\theta_{i,k}\theta_{j,l}}$

where $\theta$, $p$ and $P_{i,i}(x)$ are defined as before. This quantity is
the weighted average of the change in output probability with respect to
virality due the interaction between every pair of clusters $(k,l)$ to which
belong entities $(i,j)$ in a proportion ($\theta_{ik}$,$\theta_{jl}$). The
results are shown in the heatmaps Fig3. Note that the clusters have been
manually given a name on the basis of their composition; labeling is
subjective.

We see that most of the clusters do not interact with each other; the
interactions essentially take place between a limited number of clusters.
Typically, a cluster interacts significantly with only one or two other
clusters in every corpus (“Vertigo” and “Speech disorder” in PubMed,
“Students” and “Schools” in Reddit, etc.). We also notice that in each corpus,
the model forms some non-interacting cluster with low values of $V_{k,l}$
(“Neuropsychatry” in PubMed, “Recent pop” in Spotify, etc.); for those, the
probability of an output is essentially equal to the virality of this output.
We also notice that the diagonal of the $V_{k,l}$ matrices have low values;
this expresses that the interaction of a group with itself leads to an output
probability close to its virality. To picture how this makes sense, we can
imagine diagnosing a disease on the basis of two ear-related symptoms
(“earache” and “hearing disorders”): the diagnosis is likely to be related to
the ear as we would have guessed with only one symptom (its probability equals
virality). Now imagine two symptoms of different kind (“earache” and “speech
disorder”): the diagnosis is then likely to be related to the brain and less
to the ear, so the interaction lowers the base probability (virality) of the
“ear disease” output and increases the one of the brain disease.

Being able to see in details the extent to which interactions exert an
influence in a corpus and between which categories they take place open new
perspectives in research. Understanding and explaining underlying mechanisms
ruling corpora is a desirable property of AI models in social sciences for
instance (S. et al., 2018; Gutiérres-Roig et al., 2016; Guimera et al., 2012).
As we demonstrated, the present work follows this line by developing an
explainable model for investigating interactions.

###  5.2. Entropy of membership

Another interesting quantity to look at when considering the results is the
membership entropy of the entities. This quantity measures how entities are
spread over all the clusters; when this value is low, it means that the model
finds strong regularities in the corpus, and the clusters are likely to be
readily interpretable. Therefore, we use the normalized Shannon entropy of
memberships of user i, $S_{i}^{(m)}$, whose formula is:

(10)
$S_{i}^{(m)}=\frac{1}{\log_{2}\frac{1}{T}}\sum_{t}^{T}\theta_{i,t}\log_{2}\theta_{i,t}$

Here the lowest entropy reachable is 0, which corresponds to an entity
belonging to only one group ; the largest is 1 corresponding to belonging to
every cluster evenly (with probability $\frac{1}{T}$).

Overall, the entropy of memberships is low. The average entropy values per
corpus are: 0.320 for PubMed (equivalent to belonging on average to 2-3
clusters), 0.324 for Twitter (2 clusters), 0.561 for Reddit (6-7 clusters) and
0.364 for Spotify (2-3 clusters). The low number of entities spread among
clusters means that the clustering done by our model is easy to interpret
–which eased the manual annotation of the clusters presented in the previous
section.

##  6\. Conclusion

In most previous approaches to information spreading, the effect of
interactions between diffusing entities has been neglected. Here, we proposed
a modified MMSBM that allows for investigating the role of interactions and
quantifying them. By design, it also allows assessment of the importance of
interactions compared with the virality of single pieces of information (i.e.
their intrinsic ability to spread on their own). On this basis, we show that
the effect of interactions on information spread is not trivial and that
taking them into account increases predictive performance in several real-
world applications. Following this observation, we proposed an implementation
via a scalable EM algorithm allowing for its application on large datasets.

However, a major limitation to our model is that it only accounts for pair
interactions. While our conclusions state their importance in some real-world
systems, it might not be sufficient for a number of others. For instance, a
disease is seldom diagnosed on the basis of a single pair of symptoms; a model
accounting for the interaction between n entities might be more relevant in
this case. The same line of reasoning can be applied to gene and protein
interactions (Zhou et al., 2014), bacteria symbiosis or even species co-
evolution networks. We are currently working on a generalization of our model
that will hopefully provide a better description of interacting processes at
work everywhere in nature.

## References

  * (1)
  * Akaike (1973) Hirotogu Akaike. 1973\.  Information Theory and an Extension of the Maximum Likelihood Principle.  _2nd International Symposium on Information Theory_ (1973), 267–281.
  * Beutel et al. (2012) Alex Beutel, Aditya B. Prakash, Roni Rosenfeld, and Christos Faloutsos. 2012.  Interacting viruses in networks: Can both survive?  _Proceedings of the ACM SIGKDD_ (2012).
  * Bishop (2006) Christopher Bishop. 2006\.  _Pattern Recognition and Machine Learning_. Vol. 16.  450–455.  https://doi.org/10.1117/1.2819119
  * E.M. et al. (2008) Airoldi E.M., Blei D.M., Fienberg S.E., and Xing E.P. 2008\.  Mixed Membership Stochastic Blockmodels.  _Journal of Machine Learning Research_ 9 (2008), 1991–1992.
  * Funke and Becker (2019) Thorben Funke and Till Becker. 2019.  Stochastic block models: A comparison of variants and inference methods.  _PloS one_ (2019).
  * Godoy-Lorite et al. (2016) Antonia Godoy-Lorite, Roger Guimerà, Cristopher Moore, and Marta Sales-Pardo. 2016.  Accurate and scalable social recommendation using mixed-membership stochastic block models.  _PNAS_ 113, 50 (2016), 14207–14212.
  * Guimera et al. (2012) Roger Guimera, Alejandro Llorente, and Marta Sales-Pardo. 2012\.  Predicting Human Preferences Using the Block Structure of Complex Social Networks.  _PloS One_ 7, 9 (2012).
  * Guimerà and Sales-Pardo (2013) Roger Guimerà and Marta Sales-Pardo. 2013.  A Network Inference Method for Large-Scale Unsupervised Identification of Novel Drug-Drug Interactions.  _PLoS Comput Biol_ (2013).
  * Gutiérres-Roig et al. (2016) Mario Gutiérres-Roig, Carlota Segura, Jordi Dutch, and Josep Perello. 2016.  Market Imitation and Win-Stay Lose-Shift Strategies Emerge as Unintended Patterns in Market Direction Guesses.  _PLOS One_ (2016).
  * Hodas and Lerman (2014) Nathan O. Hodas and Kristina Lerman. 2014.  The Simple Rules of Social Contagion.  _Scientific Reports_ 4, 4343 (2014).
  * Huan et al. (2017) Zhao Huan, Quanming Yao, Jianda Li, Yangqiu Song, and Dik Lun Lee. 2017.  Meta-graph based recommendation fusion over heterogeneous information networks.  _SIGKDD_ (2017).
  * Jamali et al. (2011) Mohsen Jamali, Tianle Huang, and Martin Ester. 2011\.  A Generalized Stochastic Block Model for Recommendation in Social Rating Networks _(RecSys ’11)_. Association for Computing Machinery, New York, NY, USA, 53–60.  https://doi.org/10.1145/2043932.2043946
  * Kempe et al. (2015) David Kempe, Jon Kleinberg, and Eva Tardos. 2015\.  Maximizing the Spread of Influence through a Social Network.  _Theory of computing_ 11 (2015).
  * Koren et al. (2009) Y. Koren, R. Bell, and C. Volinsky. 2009.  Matrix Factorization Techniques for Recommender Systems.  _Computer_ 42, 8 (2009), 30–37.
  * Loreto et al. (2016) Vittorio Loreto, Pietro Gravino, Vito D. P. Servedio, and Francesca Tria. 2016.  On the Emergence of Syntactic Structures: Quantifying and Modeling Duality of Patterning.  _Topics in Cognitive Science_ 8, 2 (2016), 469–480.
  * M. et al. (2006) Airoldi E. M., Blei D. M., Fienberg S. E., E. P. Xing, and Jaakkola T. 2006.  Mixed membership stochastic block models for relational data with application to protein-protein interactions.  _Proceedings of the international biometrics society annual meeting_ 15 (2006).
  * Mackey et al. (2010) LW Mackey, DJ Weiss, and MI Jordan. 2010.  Mixed Membership Matrix Factorization.  _International Conference on Machine Learning (ICML)_ (2010).
  * Myers and Leskovec (2012) Seth Myers and Jure Leskovec. 2012.  Clash of the Contagions: Cooperation and Competition in Information Diffusion.  _IEEE International Conference on Data Mining (ICDM)_ (2012), 539–548.
  * Nigam et al. (2019) Aastha Nigam, Kijung Shin, Ashwin Bahulkar, Bryan Hooi, David Hachen, Boleslaw Szymanski, Christos Faloutsos, and Nitesh Chawla. 2019\.  ONE-M: Modeling the Co-evolution of Opinions and Network Connections.  _European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD)_ (2019), 122–140.
  * Pastor-Satorras et al. (2015) Romualdo Pastor-Satorras, Claudio Castellano, Piet Van Mieghem, and Alessandro Vespignani. 2015\.  Epidemic processes in complex networks.  _Reviews of Modern Physics_ 87, 3 (2015), 925–979.
  * Poux-Médard et al. (2020) Gaël Poux-Médard, Romualdo Pastor-Satorras, and Claudio Castellano. 2020.  Influential spreaders for recurrent epidemics on networks.  _Physical Review Research_ 2, 2 (2020), 023332\.
  * Poux-Medard et al. (2021) Gael Poux-Medard, Julien Velcin, and Sabine Loudcher. 2021\.  Information Interaction Profile of Choice Adoption.  _ECML-PKDD’21 (TBP)_ (2021).
  * Rashed et al. (2020) Ahmed Rashed, Shayan Jawed, Lars Schmidt-Thieme, and Andre Hintsches. 2020.  MultiRec: A Multi-Relational Approach for Unique Item Recommendation in Auction Systems. In _Fourteenth ACM Conference on Recommender Systems_ (Virtual Event, Brazil) _(RecSys ’20)_. Association for Computing Machinery, New York, NY, USA, 230–239.  https://doi.org/10.1145/3383313.3412242
  * Rodriguez et al. (2013) Manuel Gomez Rodriguez, Jure Leskovec, and Bernhard Schoelkopf. 2013.  Structure and Dynamics of Information Pathways in Online Media.  _ACM International Conference on Web Search and Data Mining (WSDM)_ (2013).
  * Romero et al. (2011) Daniel Romero, Brendan Meeder, and Jon Kleinberg. 2011\.  Differences in the Mechanics of Information Diffusion Across Topics: Idioms, Political Hashtags, and Complex Contagion on Twitter.  _WWW_ 695-704 (2011), 695–704.
  * S. et al. (2018) Cobo-López S., Godoy-Lorite A., and Duch J. 2018\.  Optimal prediction of decisions and model selection in social dilemmas using block models.  _EPJ Data Sci_ 7(48) (2018).
  * Shi et al. (2016) Chuan Shi, Yitong Li, Jiawei Zhang, Yizhou Sun, and Philip S. Yu. 2016.  A survey of heterogeneous information network analysis.  _IEEE Transactions on Knowledge and Data Engineering_ (2016), 17–37.
  * Sun (2011) Yizhou Sun. 2011\.  PathSim: meta path-based top-K similarity search in heterogeneous information networks.  _VLDB_ (2011).
  * Sun and Han (2012) Yizhou Sun and Jiawei Han. 2012.  Mining Heterogeneous Information Networks: A Structural Analysis Approach.  _SIGKDD Explorations_ 14 (2012).
  * Wang and Wong (1987) Yuchung J. Wang and George Y. Wong. 1987.  Stochastic Blockmodels for Directed Graphs.  _J. Amer. Statist. Assoc._ 82, 397 (1987), 8–19.
  * Zhou et al. (2014) Xuezhong Zhou, Jörg Menche, Albert-Laszlo Barabasi, and Amitabh Sharma. 2014.  Human symptoms–disease network.  _Nature communications_ 5 (2014), 4212.