# Model-based Clustering for Network Data via a Latent Shrinkage Position
Cluster Model

Xian Yao Gwee Isobel Claire Gormley Michael Fop
School of Mathematics and Statistics xian-
yao.gwee@ucdconnect.ieclaire.gormley<EMAIL_ADDRESS>
Isobel Claire Gormley and Michael Fop have contributed equally to this work.
University College Dublin
Dublin  Ireland

###### Abstract

Low-dimensional representation and clustering of network data are tasks of
great interest across various fields. Latent position models are routinely
used for this purpose by assuming that each node has a location in a low-
dimensional latent space, and enabling node clustering. However, these models
fall short in simultaneously determining the optimal latent space dimension
and the number of clusters. Here we introduce the latent shrinkage position
cluster model (LSPCM), which addresses this limitation. The LSPCM posits a
Bayesian nonparametric shrinkage prior on the latent positions’ variance
parameters resulting in higher dimensions having increasingly smaller
variances, aiding in the identification of dimensions with non-negligible
variance. Further, the LSPCM assumes the latent positions follow a sparse
finite Gaussian mixture model, allowing for automatic inference on the number
of clusters related to non-empty mixture components. As a result, the LSPCM
simultaneously infers the latent space dimensionality and the number of
clusters, eliminating the need to fit and compare multiple models. The
performance of the LSPCM is assessed via simulation studies and demonstrated
through application to two real Twitter network datasets from sporting and
political contexts. Open source software is available to promote widespread
use of the LSPCM.

Keywords: Latent position model, Network analysis, Mixture models, Bayesian
nonparametric priors, Multiplicative gamma process

##  1 Introduction

Network data are an important class of structured data where objects are
represented as nodes and the relationships between these objects are
represented as edges. Network data arise in various fields, including
epidemiology (Jo et al.,, 2021), neuroscience (Yang et al.,, 2020), brain
connectivity (Aliverti and Durante,, 2019) and sociology (D’Angelo et al.,,
2019). A key interest in the analysis of network data is the task of
clustering the nodes in the network, where the aim is to group together nodes
that share similar characteristics or behaviours.

While a variety of network models exist, many originate from the influential
Erdös Rényi random graph model (Erdős and Rényi,, 1959; Gilbert,, 1959),
including the widely utilised stochastic block model (Holland et al.,, 1983;
Snijders and Nowicki,, 1997) and the latent position model (LPM, Hoff et al.,,
2002). Here, we focus on the LPM which assumes each node in a network has a
position in a latent $p$-dimensional space. Under the LPM, the probability of
an edge between two nodes is determined by their proximity in the latent
space. The latent position cluster model (LPCM, Handcock et al.,, 2007)
extended the LPM to allow for model-based clustering of nodes in a network, by
assuming the latent positions are generated from a finite mixture of Gaussian
distributions. Various extensions to the LPCM have been proposed, including
approaches that account for random effects (Krivitsky et al.,, 2009), that
incorporate covariates through a mixture of experts framework (Gormley and
Murphy,, 2010), that employ a variational approach to Bayesian inference
(Salter-Townshend and Murphy,, 2013), that model longitudinal networks (Sewell
and Chen,, 2017), that allow for edge clustering (Sewell,, 2020), and that
model multiplex networks (D’Angelo et al.,, 2023); see Kaur et al., (2023) for
a comprehensive review.

Although the LPCM is widely used, inferring the optimal number of clusters and
the optimal dimensionality of the latent space remains a challenging task. In
practice, the numbers of clusters and dimensions are typically treated as
user-defined parameters, with the latter set as $2$ in the majority of cases
to allow for simple visualisation and interpretation (D’Angelo et al.,, 2023;
Liu and Chen,, 2023). Often, a set of LPCMs with different numbers of clusters
and dimensions are fitted to the network data and a model selection criterion
is used to select the optimal model. Many model selection criteria have been
used for this purpose e.g., Handcock et al., (2007) use a variant of the
Bayesian information criterion (BIC) to select optimal numbers of clusters and
dimensions, but highlight its lack of robustness. Other model selection
criteria such as the Watanabe–Akaike information criterion (WAIC) (Ng et al.,,
2021; Sosa and Betancourt,, 2022), the Akaike information criterion (AIC) and
the integrated completed likelihood (ICL) (Sewell,, 2020) have also been used.
While these criteria have found some success, they can give conflicting
inference on the same data and their use requires fitting a large set of
models, each with a different combination of number of clusters and number of
dimensions, which becomes computationally prohibitive as the set of possible
models grows.

Alternative, automated strategies for inferring the numbers of clusters and
dimensions from the network data have emerged. In the context of stochastic
block models, Yang et al., (2020) utilize a frequentist framework, while
Passino and Heard, (2020) employ a Bayesian framework for this purpose. In the
LPCM setting, automatic inference on the number of clusters has been
considered in D’Angelo et al., (2023) via an infinite mixture model, while
Durante and Dunson, (2014) and Gwee et al., (2023) employed a nonparametric
shrinkage prior to infer the latent space dimension. However, automated,
simultaneous inference of both the number of clusters and the number of
dimensions in the LPCM setting has not yet been considered.

Here the latent shrinkage position cluster model (LSPCM) is introduced, which
simultaneously infers the node clustering structure and facilitates automated
inference of the latent space dimension. To achieve clustering of the nodes, a
sparse finite mixture model (Malsiner-Walli et al.,, 2014, 2017) is employed
that overfits the number of components in the mixture model. The adoption of a
sparse prior on the mixture weights encourages emptying of redundant
components, thereby allowing inference on the number of clusters. Within each
cluster, a Bayesian nonparametric truncated gamma process shrinkage prior
(Bhattacharya and Dunson,, 2011; Gwee et al.,, 2023) is placed on the variance
of the nodes’ positions in the latent space. While the latent space is assumed
to have infinitely many dimensions, the shrinkage prior implies that higher
dimensions have negligible variance and therefore are non-informative. The
LSPCM eliminates the need for choosing a model selection criterion and only
requires the fitting of a single model to simultaneously infer both the
optimal number of clusters and the optimal latent space dimension, reducing
the required computation time. Additionally, the Bayesian framework naturally
provides uncertainty quantification for the number of clusters and the number
of dimensions through their posterior distributions.

The remainder of this article is structured as follows: Section 2 describes
the proposed LSPCM while Section 3 outlines the inferential process along with
practical implementation details. Section 4 describes simulation studies
conducted to explore the performance of the LSPCM on clustering networks in a
variety of settings where the numbers of nodes, of dimensions and of clusters
vary. Section 5 applies the proposed LSPCM to two real Twitter network data
sets: one concerning football players in the English premier league and one
concerning the Irish political context. Section 6 concludes and discusses
potential extensions. R (R Core Team,, 2023) code with which all results
presented herein were produced is freely available from the lspm GitLab
repository.

##  2 The latent shrinkage position cluster model

To cluster nodes in a network, we introduce the LSPCM which draws together
ideas underpinning both the LSPM (Gwee et al.,, 2023) and the LPCM (Handcock
et al.,, 2007).

###  2.1 The latent shrinkage position model

Network data typically take the form of an $n\times n$ adjacency matrix,
$\mathbf{Y}$, where $n$ is the number of nodes and entry $y_{i,j}$ denotes the
relationship or edge between nodes $i$ and $j$. Self-loops are not permitted
and thus the diagonal elements of $\mathbf{Y}$ are zero. Here we consider
binary edges but a variety of edge types can be considered. Under the LSPM,
edges are assumed independent, conditional on the latent positions of the
nodes. The sampling distribution is

$\mathbb{P}(\mathbf{Y}\mid\alpha,\mathbf{Z})=\prod_{i\neq
j}\mathbb{P}(y_{i,j}\mid\alpha,\mathbf{z}_{i},\mathbf{z}_{j})$

where $\mathbf{Z}$ is the matrix of latent positions, with $\mathbf{z}_{i}$
denoting the latent position of node $i$; $\alpha$ is a global parameter that
captures the overall connectivity level in the network. Denoting by $q_{i,j}$
the probability of an edge between nodes $i$ and $j$, i.e.
$\mathbb{P}(y_{i,j}=1\mid\alpha,\mathbf{z}_{i},\mathbf{z}_{j})$, a logistic
regression model formulation is used where the log odds of an edge between
nodes $i$ and $j$ depends on the Euclidean distance between their respective
positions $\mathbf{z}_{i}$ and $\mathbf{z}_{j}$ in the latent space,

$\log\frac{q_{i,j}}{1-q_{i,j}}=\alpha-\|\mathbf{z}_{i}-\mathbf{z}_{j}\|^{2}_{2}.$
(1)

As in Gollini and Murphy, (2016) and D’Angelo et al., (2019), in (1) the
distance is taken to be the squared Euclidean distance.

The LSPM is a Bayesian nonparametric extension of the LPM which allows
automatic inference on $p$, the number of effective latent space dimensions
i.e., the number of dimensions necessary to fully describe the network. Under
the LSPM, the latent positions are assumed to have a zero-centred Gaussian
distribution with diagonal precision matrix $\mathbf{\Omega}$, whose entries
$\omega_{\ell}$ denote the precision of the latent positions in dimension
$\ell$, for $\ell=1,\ldots,\infty$. The LSPM employs a multiplicative
truncated gamma process (MTGP) prior on the precision parameters: the latent
dimension $h$ has an associated shrinkage strength parameter $\delta_{h}$,
where the cumulative product of $\delta_{1}$ to $\delta_{\ell}$ gives the
precision $\omega_{\ell}$. An unconstrained gamma prior is assumed for
$\delta_{1}$, while a truncated gamma distribution is assumed for the
remaining dimensions to ensure shrinkage. Specifically, for $i=1,\ldots,n$

$\displaystyle\mathbf{z}_{i}\sim\text{MVN}(\mathbf{0},\mathbf{\Omega}^{-1})\qquad\mathbf{\Omega}=\begin{bmatrix}\omega_{1}^{-1}&\dots&0\\\
\vdots&\ddots&\\\
0&&\omega_{\infty}^{-1}\end{bmatrix}\qquad\omega_{\ell}=\prod_{h=1}^{\ell}\delta_{h}\mbox{
for }\ell=1,\ldots,\infty$ (2)
$\displaystyle\delta_{1}\sim\text{Gam}(a_{1},b_{1}=1)\qquad\delta_{h}\sim\text{Gam}^{\text{T}}(a_{2},b_{2}=1,t_{2}=1)\mbox{
for }h>1.$

Here $a_{1}$ and $b_{1}$ are the gamma prior’s shape and rate parameters on
the first dimension’s shrinkage parameter, while $a_{2}$ is the shape
parameter, $b_{2}$ is the rate parameter, and $t_{2}$ is the left truncation
point (here set to 1) of the truncated gamma prior for dimensions $h>1$. This
MTGP prior results in an increasing precision and therefore a shrinking
variance of the positions in the higher dimensions of the latent space. Under
this MTGP prior, the LSPM is nonparametric with infinitely many dimensions,
where unnecessary higher dimensions’ variances are increasingly shrunk towards
zero. Dimensions that have variance very close to zero will then have little
meaningful information encoded in them as the distances between nodes will be
close to zero. Thus, the effective dimensions are those in which the variance
is non-negligible.

###  2.2 The latent shrinkage position cluster model

To infer the unknown number of clusters of nodes, we fuse the LSPM with the
latent position cluster model (LPCM) of Handcock et al., (2007) and the sparse
finite mixture model framework of Malsiner-Walli et al., (2014, 2017) to give
the latent shrinkage position cluster model (LSPCM). The LSPCM assumes that
the latent positions arise from a finite mixture of $G$ spherical multivariate
normal distributions, with an MTGP shrinkage prior assumed for the precision
parameters i.e., for $i=1,\ldots,n$

$\displaystyle\mathbf{z}_{i}\sim\sum_{g=1}^{G}\tau_{g}\text{MVN}(\bm{\mu}_{g},\mathbf{\Omega}^{-1})\qquad\mathbf{\Omega}=\begin{bmatrix}\omega_{1}^{-1}&\dots&0\\\
\vdots&\ddots&\\\
0&&\omega_{\infty}^{-1}\end{bmatrix}\qquad\omega_{\ell}=\prod_{h=1}^{\ell}\delta_{h}\mbox{
for }\ell=1,\ldots,\infty$ (3)
$\displaystyle\quad\delta_{1}\sim\text{Gam}(a_{1},b_{1}=1)\quad\delta_{h}\sim\text{Gam}^{\text{T}}(a_{2},b_{2}=1,t_{2}=1)\mbox{
for }h>1$ (4)
$\displaystyle\qquad\bm{\mu}_{g}|\mathbf{\Omega}\sim\text{MVN}(\mathbf{0},\xi\mathbf{\Omega}^{-1})\qquad\bm{\tau}=(\tau_{1},\ldots,\tau_{G})\sim\text{Dir}(\nu,\ldots,\nu).$
(5)

Here $\tau_{g}$ denotes the probability that a node belongs to the $g$-th
component, so that $\tau_{g}\geq 0$ $(g=1,\ldots,G)$ and
$\sum_{g=1}^{G}\tau_{g}=1$. While Handcock et al., (2007) and Ryan et al.,
(2017) consider a component specific precision matrix in the Gaussian mixture,
here, to facilitate simultaneous shrinkage of the variance across higher
dimensions for all components, a single precision matrix is assumed.
Importantly, for the mean latent position $\bm{\mu}_{g}$ of the $g$-th
component, a multivariate Gaussian prior with zero mean and precision matrix
having a MTGP prior is assumed. Consequently, the component means are
increasingly shrunk towards zero, resulting in higher dimensions characterized
by increasingly overlapping mixture components, and, as a result, becoming
less informative for cluster separation. Further, as in Ryan et al., (2017),
the prior covariance on $\bm{\mu}_{g}$ is inflated by a scaling factor,
$\xi=9$, so that component means are modelled as more dispersed than the
component members.

While the LPCM assumes a finite mixture of spherical multivariate normal
distributions and casts the problem of inferring the number of clusters as one
of model selection, the LSPCM considers an overfitted (or sparse) finite
mixture model. Here, following Malsiner-Walli et al., (2014); Frühwirth-
Schnatter and Malsiner-Walli, (2018), a distinction between the number of
mixture components $G$, the number of non-empty mixture components $G_{+}$,
and the number of clusters in the data $G^{*}$ is made. In addition, a
symmetric Dirichlet prior is placed on the mixture weights $\bm{\tau}$ with
the Dirichlet’s hyperparameter $\nu$ playing an influential role in inducing
sparsity. To define an overfitting mixture distribution, a large number $G$ of
initial components is specified and small values of $\nu$ are considered,
ensuring that unnecessary components are emptied out during the inferential
process. The number of non-empty components $G_{+}$ serves then as an estimate
of the number of clusters $G^{*}$. Thus, the LSPCM allows for automatic,
simultaneous inference of the number of clusters and of the latent space
dimension, and it requires only a single model to be fit to the network data.

##  3 Inference

To facilitate inference, we introduce $\bm{C}=(\bm{c}_{1},\ldots,\bm{c}_{n})$
which contains the latent component membership vectors
$\bm{c}_{i}=(c_{i1},\ldots,c_{iG})$, for $i=1,\ldots,n$, where $c_{ig}=1$ if
node $i$ belongs to component $g$ and $c_{ig}=0$ otherwise. Thus,
$\bm{c}_{i}\sim\text{MultiNom}(1,\bm{\tau})$. Denoting by $\bm{\Theta}$ the
component means $\bm{\mu}_{1},\ldots,\bm{\mu}_{G}$, and the precision matrix
$\bm{\Omega}$, the joint posterior distribution of the LSPCM is then

$\displaystyle\mathbb{P}(\alpha,\mathbf{Z},\mathbf{C},\bm{\tau},\bm{\Theta}\mid\mathbf{Y})$
$\displaystyle\propto\mathbb{P}(\mathbf{Y}\mid\alpha,\mathbf{Z})\mathbb{P}(\alpha)\mathbb{P}(\mathbf{Z}\mid\bm{C},\bm{\tau},\bm{\Theta})\mathbb{P}(\bm{C}\mid\bm{\tau})\mathbb{P}(\bm{\tau})\mathbb{P}(\bm{\Theta})$

where $\mathbb{P}(\alpha)$ is the non-informative
$N(\mu_{\alpha}=0,\sigma^{2}_{\alpha}=4)$ prior for the $\alpha$ parameter,
and $\mathbb{P}(\bm{\tau})$ and $\mathbb{P}(\bm{\Theta})$ denote the prior
distributions outlined in Section 2.2. For the MTGP priors, similar to Gwee et
al., (2023) and Durante, (2017), the hyperparameters $a_{1}=2$ and $a_{2}=3$.
Throughout, we consider sensitivity of inference to the Dirichlet
hyperparameter $\nu$.

###  3.1 An adaptive Metropolis-within-Gibbs sampler

Markov chain Monte Carlo (MCMC) is employed to draw samples from the joint
posterior distribution. After defining necessary notation in Appendix A,
derivations of the full conditional distributions of the latent positions,
cluster membership vectors and model parameters are given in Appendix B. As
the MTGP prior is nonparametric and assumes an infinite number of latent
dimensions, in practice setting a finite truncation level, $p_{0}$, on the
number of dimensions fitted is required. Thus, an adaptive Metropolis-within-
Gibbs sampler is employed where $p$ is dynamically shrunk or augmented as the
sampler proceeds. Denoting the current iteration by $s$ (but for clarity $s$
is omitted where it is unnecessary), a single pass of the sampler proceeds as
follows:

  1. 1.

Sample the component mean $\bm{\mu}_{g}$ for $g=1,\ldots,G$ from
$\text{MVN}_{p}\left(\frac{\sum_{i=1}^{n}c_{ig}\mathbf{z}_{i}}{\sum_{i=1}^{n}c_{ig}+\xi^{-1}},\left[\bm{\Omega}\left(\sum_{i=1}^{n}c_{ig}+\xi^{-1}\right)\right]^{-1}\right).$

  2. 2.

Sample the mixing weights $\bm{\tau}$ from
Dir$(\sum_{i=1}^{n}c_{i1}+\nu,\ldots,\sum_{i=1}^{n}c_{iG}+\nu)$.

  3. 3.

Sample the latent memberships $\bm{c}_{i}$ for each node $i=1,\ldots,n$ from a
Multinomial with $G$ categories by drawing the $g$th category with probability
$\frac{\tau_{g}\phi_{p}(\mathbf{z}_{i};\bm{\mu}_{g},\bm{\Omega}^{-1})}{\sum_{r=1}^{G}\tau_{r}\phi_{p}(\mathbf{z}_{i};\bm{\mu}_{r},\bm{\Omega}^{-1})}$
where $\phi_{p}(\mathbf{z}_{i};\bm{\mu},\bm{\Omega}^{-1})$ is the
$p$-dimensional multivariate normal density.

  4. 4.

Sample $\check{\mathbf{Z}}$ from a
MVN${}_{p}(\mathbf{Z}^{(s)},k\mathbf{\Omega}^{-1(s)})$ proposal distribution
where $k$ is a step size factor. Accept $\check{\mathbf{Z}}$ as
$\mathbf{Z}^{(s+1)}$ with probability
$\frac{\mathbb{P}(\mathbf{Y}\mid\alpha^{(s)},\check{\mathbf{Z}})}{\mathbb{P}(\mathbf{Y}\mid\
\alpha^{(s)},\mathbf{Z}^{(s)})}\frac{\phi_{p}(\mathbf{z}_{i};\check{\mathbf{Z}},k\mathbf{\Omega}^{-1(s)})}{\phi_{p}(\mathbf{z}_{i};\mathbf{Z}^{(s)},k\mathbf{\Omega}^{-1(s)})}$,
otherwise set $\mathbf{Z}^{(s+1)}=\mathbf{Z}^{(s)}$.

  5. 5.

Sample $\check{\alpha}$ from an informed Gaussian proposal distribution
(Gormley and Murphy,, 2010) and accept following the Metropolis-Hastings
acceptance ratio.

  6. 6.

Sample $\delta_{1}$ from
$\text{Gam}\left(\frac{\left(n+G\right)p}{2}+a_{1},\right.\\\
\left.\qquad\qquad\frac{1}{2}\sum_{i=1}^{n}\sum_{g=1}^{G}(\mathbf{z}_{i}-\bm{\mu}_{g})^{T}\left(\prod_{m=2}^{\ell}\delta_{m}\right)\mathbf{I}_{p}(\mathbf{z}_{i}-\bm{\mu}_{g})c_{ig}+\right.\\\
\left.\qquad\qquad\qquad\frac{1}{2}\sum_{g=1}^{G}\bm{\mu}_{g}^{T}(\xi^{-1})\left(\prod_{m=2}^{\ell}\delta_{m}\right)\mathbf{I}_{p}\bm{\mu}_{g}+b_{1}\right).$

  7. 7.

Sample $\delta_{h}^{(s+1)}$ for $h=2,\ldots,p$ from
$\text{Gam}^{\text{T}}\left(\frac{\left(n+G\right)(p-h+1)}{2}+a_{2},\right.\\\
\left.\qquad\qquad\frac{1}{2}\sum_{i=1}^{n}(\mathbf{z}_{i}-\bm{\mu}_{g})^{T}\left(\prod_{m=1,m\neq
h}^{\ell}\delta_{m}^{(s^{*})}\right)\mathbf{I}_{p}(\mathbf{z}_{i}-\bm{\mu}_{g})c_{ig}+\right.\\\
\left.\qquad\qquad\qquad\frac{1}{2}\sum_{g=1}^{G}\bm{\mu}_{g}^{T}(\xi^{-1})\left(\prod_{m=1,m\neq
h}^{\ell}\delta_{m}^{(s^{*})}\right)\mathbf{I}_{p}\bm{\mu}_{g}+b_{2},\quad
1\right)$
where $s^{*}=s+1$ for $m<h$ and $s^{*}=s$ for $m>h$.

  8. 8.

Calculate $\omega_{\ell}$ by taking the cumulative product of $\delta_{1}$ to
$\delta_{\ell}$.

###  3.2 Practical implementation details

When implementing the adaptive Metropolis-within-Gibbs sampler, practical
details such as initialisation of latent variables and parameters, adapting
the latent dimension and post-processing of the MCMC chain require attention.

####  3.2.1 Initialisation

While the LSPCM allows for an infinite number of latent dimensions and a
potentially large number of components in the overfitted mixture, this is
computationally impractical. Setting initial values for both the truncation
level of the latent dimensions, $p_{0}$, and for the number of components in
the mixture, $G$, prevents the sampler from exploring computationally
unfeasible models. Here, $p_{0}=5$ and $G=20$ are used throughout as these
settings were empirically observed to strike a balance between exploring
complex models and computational feasibility for the type of networks
analysed.

The LSPCM’s parameters, cluster allocations and latent positions also require
initialisation, achieved here as follows, where $s=0$:

  1. 1.

Calculate the geodesic distances (Kolaczyk and Csárdi,, 2020) between the
nodes in the network. Apply classical multidimensional scaling (Cox and Cox,,
2001) to the geodesic distances and set $\mathbf{Z}^{(s)}$ to be the resulting
$n\times p_{0}$ positions.

  2. 2.

Fit a standard regression model, with regression coefficients $\alpha$ and
$\beta$, to the vectorised adjacency matrix where $\mbox{log
odds}(q_{i,j}=1)=\alpha-\beta\|\mathbf{z}_{i}^{(s)}-\mathbf{z}_{j}^{(s)}\|^{2}_{2}$
to obtain estimates $\hat{\alpha}$ and $\hat{\beta}$. Set
$\alpha^{(s)}=\hat{\alpha}$.

  3. 3.

As the LSPCM model (1) constrains $\beta=1$, centre and rescale the latent
positions by setting
$\mathbf{Z}^{(s)}=\sqrt{|\hat{\beta}|}\tilde{\mathbf{Z}}^{(s)}$, where
$\tilde{\mathbf{Z}}^{(s)}$ are the mean-centred initial latent positions.

  4. 4.

Apply model-based clustering to $\mathbf{Z}^{(s)}$ via mclust (Scrucca et
al.,, 2016) with up to $G$ clusters and the EEI covariance structure to obtain
initial cluster allocations.

  5. 5.

Obtain $\omega_{\ell}^{(s)}$ for $\ell=1,\ldots,p_{0}$ by calculating the
empirical precision of each column of $\mathbf{Z}^{(s)}$.

  6. 6.

Set $\delta_{1}^{(s)}=\omega_{1}^{(s)}$ and calculate
$\delta_{h}^{(s)}=\frac{\omega_{h}^{(s)}}{\omega_{h-1}^{(s)}}$ where
$h=2,\ldots,p_{0}$.

####  3.2.2 Adapting the latent dimension

As in Gwee et al., (2023), the LSPCM adapts the latent space dimension $p$ as
the MCMC sampler runs, meaning there is a finite number of effective
dimensions at each iteration. Similar to Bhattacharya and Dunson, (2011), the
probability of adapting the number of latent dimensions at iteration $s$ is
taken as $\mathbb{P}(s)=\exp(-\kappa_{0}-\kappa_{1}s)$, which decreases
exponentially as the MCMC chain evolves. Here, setting $\kappa_{0}=4$ and
$\kappa_{1}=5\times 10^{-4}$ demonstrated good performance in empirical
experiments.

After the burn-in period, at an adaptation step, $p$ is reduced by 1 if the
cumulative proportion of the latent position variance that the dimensions
$\ell=1,\ldots,p-1$ contain is greater than $\epsilon_{1}$. In general,
$\epsilon_{1}=0.8$ was found to work well but higher $\epsilon_{1}$ showed
better performance in networks with large $n$. If the criterion for reducing
$p$ is not met, an increase in $p$ is then considered by examining
$\delta_{p}^{-1}$ and a threshold $\epsilon_{2}$: if
$\delta_{p}^{-1}>\epsilon_{2}$, $p$ is increased by 1, with the additional
associated parameters drawn from their respective priors. We consider
$\epsilon_{2}=0.9$ which was found to work well in practice. The case where
$p=1$ at an adaptation step requires a different criterion: if the proportion
of latent positions that have absolute deviation from their empirical mean is
greater than $\epsilon_{3}$ times the 95% critical value of a standard Normal
distribution, then $p$ is increased to 2; reducing $p$ is not considered if
$p=1$. Setting $\epsilon_{3}=5$ was found to work well in practice.

Crucially, the adaptive MCMC sampler provides the posterior distribution of
the number of active dimensions $p$. Here, the posterior mode $p_{m}$ is used
as the estimate of the effective dimension, denoted by $p^{*}$, with credible
intervals quantifying the associated uncertainty.

####  3.2.3 Post-processing of the MCMC chain

The likelihood function of the LSPCM depends on the Euclidean distances
between the latent positions, hence it remains unaffected by rotations,
reflections, or translations of these positions, giving rise to
identifiability issues. To ensure valid posterior inference, similar to
Gormley and Murphy, (2010), a Procrustean transformation of the sampled latent
positions $\mathbf{Z}^{(1)},\ldots,\mathbf{Z}^{(S)}$ is considered. The
transformation aligns the sampled positions with a reference configuration
$\tilde{\mathbf{Z}}$, which is selected based on the configuration that yields
the highest log-likelihood during the burn-in phase of the MCMC chain.
Although this choice is arbitrary, it has little effect as the reference
configuration solely serves the purpose of addressing identifiability.

Since the MCMC samples of the component allocations will have varying numbers
of non-empty components, and because of the label switching problem caused by
invariance of the mixture with respect to permutation of the component labels,
care must be taken when deriving posterior summaries of cluster labels and
parameters. While Frühwirth-Schnatter et al., (2019) give an overview of many
potential approaches, here, as it demonstrated robust and accurate performance
for the networks examined, we adopt the procedure of Fritsch and Ickstadt,
(2009) to estimate the cluster labels of the nodes and subsequently the number
of clusters. The method, implemented in the R package mcclust (Fritsch,,
2022), first estimates the posterior similarity matrix containing the
proportion of times a pair of nodes are placed in the same cluster, and then
maximizes the posterior expected adjusted Rand index (PEAR) to obtain the
optimal cluster labels. In addition, as in Malsiner-Walli et al., (2014), the
posterior distribution of the number of filled components $G_{+}$ and the
corresponding posterior mode $G_{m}$ are examined to inspect the uncertainty
in the number of clusters. In some situations (see Section 5.1 for an
example), the posterior mode may differ from the number of clusters estimated
by the PEAR method as it tends to aggregate together small clusters. Finally,
for summarizing the posterior distributions of the cluster parameters, as in
Gormley and Murphy, (2010), cluster parameters are obtained after permuting
cluster labels to minimise a loss function based on the cluster means.

##  4 Simulation studies

The performance of the LSPCM is assessed on simulated data scenarios by
evaluating its ability to recover the dimension of the latent space and to
correctly infer the latent positions, the number of clusters and the cluster
allocations. The latent positions are simulated according to (5) where the
mixture weights $\bm{\tau}$ are generated from a symmetric $\text{Dir}(10)$
which gives rise to clusters with similar numbers of nodes. Given the latent
positions, a network is then generated as in (1). Two scenarios are
considered: Sections 4.1 and 4.2, respectively, assess the performance of
LSPCM on networks with small and moderate numbers of nodes, dimensions, and
clusters. A total of 10 networks are simulated in each scenario. In Section
4.1, the MCMC chains are run for 500,000 iterations with a burn-in of 50,000
iterations, thinned every 1,000th. In Section 4.2, 1,000,000 iterations are
considered, with a burn-in of 100,000 iterations, thinned every 2,000th
iteration. The hyperparameter $\nu$ of the Dirichlet mixing prior has been set
at different orders of magnitude from $10^{-1}$ to $10^{-5}$ to assess
sensitivity when inferring the number of clusters. Also, different values from
2 to 3.5 are used for the step size $k$ in the proposal distributions to
ensure acceptance rates are in the $20\%-40\%$ range.

###  4.1 Scenario 1: small networks

Networks with $n=50$ are generated with effective latent dimension $p^{*}=2$
and true number of clusters $G^{*}=3$, with shrinkage strength
$\bm{\delta}=(1,1.05)$ and cluster mean positions
$\bm{\mu}=\\{(0,0),(-4,0),(-4,0)\\}$ giving well separated clusters. Across
the 10 simulated networks, the smallest cluster contained 9 nodes while the
largest cluster contained 31, and fixing $\alpha=6$ gave rise to network
densities between 27% and 38%.

Table 1 indicates accurate inference across the various settings of $\nu$,
with the posterior modal number of dimensions $p_{m}=2$ and the posterior
modal number of clusters $G_{m}=3$, except when $\nu=0.1$. When $\nu=0.1$,
while the number of clusters is overestimated with $G_{m}=6$, nodes are
predominantly allocated to 3 clusters while the remaining clusters typically
contain only one node. Thus, the relatively large value of $\nu=0.1$ induces
the algorithm to struggle to empty superfluous components in the small network
scenario. Overall, the adjusted Rand index (ARI, Hubert and Arabie,, 1985),
which measures agreement between the inferred cluster memberships and the true
cluster labels, shows good correspondence, with values typically around 0.9
across different $\nu$. However, the lower bound of the 95% credible interval
for the ARI drops when $\nu=0.00001$ as the relatively small value of $\nu$ at
times overshrinks the mixture components, collapsing all the nodes into one
cluster. Procrustes correlations (PC, Peres-Neto and Jackson,, 2001) between
inferred and true latent positions also show that the latent positions are
accurately estimated, with average values $\geq 0.92$ and lower and upper
bounds of 95% credible intervals of 0.84 and 1.0, respectively. Additional
visualization of the posterior distributions on the number of dimensions,
clusters, variances, shrinkage strengths, and $\alpha$ are available in
Appendix C.

Table 1: Sensitivity analysis across various settings of $\nu$ on 10 simulated networks with $n=50,p^{*}=2,G^{*}=3$ assessed via $p_{m}$, $G_{m}$, ARI, and PC. The 95% credible intervals are given in parentheses. $\nu$ | $p_{m}$ | $G_{m}$ | ARI | PC
---|---|---|---|---
0.1 | 2 (1, 3) | 6 (3, 10) | 0.88 (0.81, 0.99) | 0.92 (0.85, 0.99)
0.01 | 2 (2, 3) | 3 (2, 5) | 0.93 (0.83, 1.00) | 0.96 (0.92, 0.99)
0.001 | 2 (2, 3) | 3 (1, 4) | 0.92 (0.80, 1.00) | 0.96 (0.86, 0.99)
0.0001 | 2 (1, 3) | 3 (1, 3) | 0.92 (0.83, 1.00) | 0.94 (0.84, 1.00)
0.00001 | 2 (1, 3) | 3 (1, 3) | 0.79 (0.19, 1.00) | 0.97 (0.88, 1.00)

For comparison, the LPCM is fitted on the same simulated data using the R
package latentnet (Krivitsky et al.,, 2022; Krivitsky and Handcock,, 2008).
Upon fitting 25 LPCMs with different combinations of $p=\\{1,\ldots,5\\}$ and
$G=\\{1,\ldots,5\\}$ to each of the 10 simulated networks, the BIC suggested 2
dimensions and 3 clusters for all of them. In terms of computational cost,
fitting the LSPCM on a computer with an i7-10510U CPU and 16GB RAM took on
average 33 minutes. The LPCM with the correct $p$ and $G$ took 4.5 minutes on
average to run for the same number of iterations. Across the various
replicates, the total time taken to fit the 25 LPCMs with different
combinations of $p$ and $G$ was comparable to or longer than the time taken to
fit the LSPCM. However, notably, when fitting the LPCMs no quantification of
the uncertainty in $p$ and $G$ is provided.

###  4.2 Scenario 2: moderately sized networks

Networks with $n=200$ were generated with $p^{*}=4$ effective latent
dimensions and $\bm{\delta}=(1,1.1,1.05,1.02)$. The true number of clusters
$G^{*}=7$ with $\bm{\mu}=\\{(-5,0,0,0),(-5,5,0,0),\\\
(0,-5,5,0),(0,0,-5,5),(2,0,2,-5),(-2,2,-2,0),(0,-2,0,0)\\}$. The 7 clusters
have different degrees of separation across the various dimensions. Among the
10 simulated networks, the smallest cluster size had 11 nodes while the
largest cluster had 55. The parameter $\alpha=20$, which resulted in networks
with density varying between 16% and 22%.

Table 2 shows that the posterior modal number of dimensions $p_{m}$ tended to
underestimate the true number of dimensions, although it was contained within
the 95% credible interval in all settings of $\nu$. Inspection of the true
latent positions empirical variance reveals that the variance across
dimensions are similar but a relatively large decrease is seen in the 4th
dimension. Using a higher dimension threshold parameter of $\epsilon_{1}=0.9$
resulted in $p_{m}=4$. The posterior modal number of clusters $G_{m}$
corresponded to the true generating number of clusters, except again in the
case where $\nu=0.1$ due to insufficient shrinkage. The ARI values indicate
very accurate and robust inference of cluster membership with all 95% credible
interval lower bound values $\geq 0.96$. The average PC values were $\geq
0.86$ but lower bounds of the 95% credible intervals were only $\geq 0.71$.
The PC values are calculated using only dimensions up to and including the
posterior modal number of dimensions; in cases where $p_{m}<p^{*}$, the higher
dimensions are not included in the comparison resulting in lower PC values.
Additional visualization of the posterior distributions on the number of
dimensions, clusters, variances, shrinkage strengths, and $\alpha$ are
available in Appendix C.

Compared to the small networks scenario, intuitively, the credible intervals
tend to be narrower when $n=200$. The LSPCM again struggled to eliminate
unnecessary components when the mixing prior was not sparse enough i.e., when
$\nu=0.1$. However, unlike the small networks scenario, here the sparse prior
when $\nu=0.00001$ did not cause the components to collapse into a reduced
number of large clusters, indicating robust inference under the larger size
network scenario.

Upon fitting the 25 LPCMs from all combinations of $p=\\{1,\ldots,5\\}$ and
$G=\\{5,\ldots,10\\}$ to the 10 simulated networks, the BIC suggested 3
dimensions and 7 clusters $90\%$ of the time. The average time for the
completion of one MCMC chain for the LSPCM was 230 minutes. However, fitting
the LPCM for multiple combinations of $p$ and $G$ incurred a considerably
larger computational cost. In fact, solely for the correct pair $p=4$ and
$G=7$ it took 200 minutes to run the MCMC via latentnet for the same number of
iterations, and again no uncertainty quantification is provided.

Table 2: Sensitivity analysis across various settings of $\nu$ on 10 simulated networks with $n=200,p^{*}=4,G^{*}=7$ assessed via $p_{m}$, $G_{m}$, ARI, and PC. The 95% credible intervals are given in parentheses. $\nu$ | $p_{m}$ | $G_{m}$ | ARI | PC
---|---|---|---|---
0.1 | 2 (2, 4) | 9 (7, 13) | 0.97 (0.96, 0.99) | 0.86 (0.71, 0.97)
0.01 | 3 (2, 4) | 7 (7, 9) | 0.99 (0.97, 1.00) | 0.93 (0.76, 0.98)
0.001 | 3 (2, 4) | 7 (7, 8) | 0.99 (0.97, 1.00) | 0.90 (0.72, 0.98)
0.0001 | 3 (3, 4) | 7 (7, 7) | 0.99 (0.97, 1.00) | 0.95 (0.93, 0.98)
0.00001 | 4 (2, 5) | 7 (7, 7) | 0.98 (0.96, 1.00) | 0.91 (0.74, 0.98)

##  5 Application to Twitter network data

The LSPCM is fitted to two binary Twitter networks with different
characteristics: a football players network with a small number of nodes, in
which each player is known to belong to one of three football clubs, and a
network among Irish politicians with a moderate number of nodes, where each
politician is affiliated to one of seven Irish political parties. The data are
publicly available at this link (Greene and Cunningham,, 2013). In the
analyses, hyperparameters, initial values, and step sizes are set as in
Section 4.

###  5.1 Football players network

This binary network consists of directed edges indicating the presence of
Twitter mentions from one English Premier League football player’s Twitter
account to another. The data were adapted from those provided in Greene and
Cunningham, (2013): 55 players playing for 3 different Premier League clubs
are considered with 15 players from Stoke City football club (Stoke), 23
players from Tottenham Hotspur (Spurs), and 17 players from West Bromwich
Albion (West Brom). In total, there were 497 mentions between players, giving
a network density of 16.73%. To fit the LSPCM to these data, 10 MCMC chains
are run with $\nu=0.01$, each for 500,000 iterations with a burn-in period of
50,000 and thinned every 1,000th. The average time for the completion of one
MCMC chain was 35 minutes.

Figure 1 illustrates that under the LSPCM the posterior modal number of
clusters $G_{m}=$ 4 (1, 6) and the posterior modal number of dimensions is 2
(1, 2), with 95% credible intervals in brackets. There is uncertainty in
$G_{+}$ as demonstrated by the wide posterior interval and notable support for
$G_{+}=3$, but there is strong certainty on the number of dimensions $p$.

(a)

(b)

Figure 1: Football players network, (a) the posterior distribution of the
number of non-empty components $G_{+}$, and (b) the posterior distribution of
the latent space dimension $p$.

Despite $G_{m}=4$, further inspection showed that, across all the 10 chains, a
small number of nodes was typically allocated to the 4th cluster. Maximising
PEAR resulted in an estimate of the number of clusters of 3, with an ARI of
0.94 between the final clustering and the players’ clubs. Figures 2(a) and
2(b) show the Fruchterman-Reingold layouts (Kolaczyk and Csárdi,, 2020) of the
network with the players’ clubs and inferred clusters detailed respectively.
Only one player was clustered differently to the other players in his club:
the West Brom player Romelu Lukaku who was originally from Chelsea football
club and was on a season-long loan deal with West Brom. Lukaku mentioned and
was mentioned by only one player, the Spurs player Jan Vertonghen and
intuitively the LSPCM has clustered them together. Figure 2(c) shows the
posterior mean latent positions coloured by the cluster labels that maximises
the PEAR across the 10 chains. Cluster 1 (which contains all of the Stoke
players only) and cluster 2 (all players from Spurs, and Romelu Lukaku) are
separate from each other on the first dimension, while in the second dimension
cluster 3 (which captures West Brom players except Romelu Lukaku) is separated
from clusters 1 and 2, indicating both dimensions are necessary for
representation of the nodes in the three clusters.

(a)

(b)

(c)

Figure 2: Football players network, (a) the Fruchterman-Reingold layout with
players coloured by club, (b) the Fruchterman-Reingold layout with players
coloured by inferred cluster label and (c) posterior mean latent positions on
the $p_{m}=2$ dimensions coloured by inferred cluster label.

Through the posterior similarity matrix, Figure 3 illustrates the uncertainty
in the football players’ cluster labels where the colour intensity indicates
the probability of a player being clustered together with another player.
Cluster 3 (predominantly players from West Brom) has the strongest certainty
in its cluster labels, while there is some uncertainty in the membership of
players in clusters 1 and 2.

Figure 3: Heat map of the posterior similarity matrix of the cluster labels
inferred from the football network, ordered by the cluster labels.

Upon fitting 16 LPCMs from all combinations of $p=\\{1,\ldots,4\\}$ and
$G=\\{1,\ldots,4\\}$, the BIC suggests 3 clusters and 2 dimensions as optimal.
The Procrustes correlation between the posterior mean latent positions under
the optimal LPCM and the LSPCM was 0.94 (standard deviation of 0.05 across the
10 LSPCM chains). Additional results regarding the posterior distributions of
variance and shrinkage strength parameters can be found in the supplementary
material, Appendix D.

###  5.2 Irish politicians network

The LSPCM is used to analyse a Twitter network between 348 Irish politicians
from the year 2012. The network consists of binary directed edges indicating
if one politician follows another (Greene and Cunningham,, 2013). Each of the
politicians is affiliated with one of the seven Irish political parties: 49
are affiliated with Fianna Fáil (FF), 143 with Fine Gael (FG), 7 with the
Green Party (Green), 79 with the Labour Party (Lab), 31 with Sinn Féin (SF), 8
with the United Left Alliance (ULA) and 31 are Independent (Ind). There are
16,856 directed relationships between the politicians, giving a network
density of 13.96%. For inference, 10 MCMC chains are run with $\nu=0.00001$,
each for 2,000,000 iterations with a burn-in period of 100,000 and thinned
every 4,500th sample. The average time for the completion of one MCMC chain
was 340 minutes.

Figure 4 shows that, under the LSCPM, the posterior modal number of clusters
is 5 (4, 6) and the posterior modal number of dimensions is 4 (3, 5), with the
95% credible intervals reported in brackets. Upon fitting multiple LPCMs
across all combinations of $p=\\{2,\ldots,8\\}$ and $G=\\{2,\ldots,8\\}$, the
BIC suggests 6 clusters and 6 dimensions. The median Procrustes correlation
between the posterior mean positions under the LPCM and LSPCM was 0.71
(standard deviation of 0.03 across the 10 LSPCM chains). Additional results
regarding the posterior distributions of variance and shrinkage strength
parameters can be found in the supplementary material, Appendix D.

(a)

(b)

Figure 4: For the Irish politicians network: (a) the posterior distribution of
the number of non-empty components $G_{+}$, and (b) the posterior distribution
of the latent space dimension.

After maximising the PEAR across the 10 chains, the estimated number of
clusters is 5 and the ARI between the LSPCM inferred cluster labels and the
politicians’ political affiliation is 0.92. Table 3 presents the cross-
tabulation of the political party membership and the LSPCM cluster labels.
Additionally, Figure 5 shows the Fruchterman-Reingold layout of the network
with nodes coloured by political party affiliation and LSPCM inferred cluster
label. The inferred clustering structure captures the composition and nature
of the 2012 Irish political landscape: a coalition government of Fine Gael and
the Labour Party were in power with Fianna Fáil in opposition. Cluster 1
captures the entirety of the Labour Party along with one politician from Fine
Gael (Catherine Yore), one politician from the Green Party (Ciarán Cuffe), and
five Independent politicians (John Crown, David Norris, Marian Harkin,
Veronica Cawley, Katherine Zappone). Cluster 2 captures the entirety of Sinn
Féin with no other politician present. Cluster 3 captures the entirety of Fine
Gael, bar one politician, along with one Green Party politician (Daniel Boyle)
and 3 other Independents (Michael Lowry, Seán Gallagher, Jillian van
Turnhout). Notably Michael Lowry was previously a member of Fine Gael. Cluster
4 is a combination of the Green Party, Independents, and the United Left
Alliance, with the former two parties having the majority of their politicians
in this cluster and the latter having its entire set of politicians in this
cluster. Cluster 5 captures the entirety of Fianna Fáil but has one Green
Party politician (Eamon Ryan) and two Independents (Rónán Mullen, Mattie
McGrath). There is a lack of overlap in the clustering structure of members of
the three main parties at the time of Fine Gael, Labour and Fianna Fáil.

Through the posterior similarity matrix, Figure 6 shows that politicians in
cluster 2 (Sinn Féin) have the strongest certainty among their labels while
other politicians have some uncertainty.

Table 3: Cross-tabulation of political party membership and the LSPCM representative cluster labels.  |  | Cluster
---|---|---
|  | 1 | 2 | 3 | 4 | 5
Political Parties | Fianna Fáil |  |  |  |  | 49
Fine Gael | 1 |  | 142 |  |
Green Party | 1 |  | 1 | 4 | 1
Independent | 5 |  | 3 | 21 | 2
Labour Party | 79 |  |  |  |
Sinn Féin |  | 31 |  |  |
United Left Alliance |  |  |  | 8 |

(a)

(b)

Figure 5: Fruchterman-Reingold layout of the Irish politicians network with
nodes coloured by (a) political party affiliation and (b) LSPCM inferred
cluster labels. Figure 6: Heat map of the posterior similarity matrix of the
Irish politicians ordered by the cluster labels.

Figure 7 shows the politicians’ posterior mean latent positions on the
$p_{m}=4$ dimensions with nodes coloured by the maximised PEAR estimated
cluster allocation across the 10 chains. On each dimension, one cluster tends
to be located separately to the others, indicating that all the dimensions
provide relevant information to distinguish the clusters. For example, on
dimension 1 the politicians in cluster 3 are located separately from the other
politicians. Similar patterns are apparent for politicians in cluster 5 on
dimension 2, politicians in cluster 2 on dimension 3 and politicians in
cluster 4 on dimension 4. Visually, the combination of all four dimensions
allows for separation of the five clusters.

Figure 7: LSPCM inferred posterior mean latent positions of the Irish
politicians on the $p_{m}=4$ dimensions with nodes coloured by cluster
membership.

##  6 Discussion

The latent shrinkage position cluster model (LSPCM) enables simultaneous
inference on the number of clusters of nodes in a network and the dimension of
the latent space required to represent it. This is achieved, respectively,
through employing a sparse prior on the weights of a mixture model and a
Bayesian nonparametric shrinkage prior on the variance of the nodes’ positions
in the latent space. The LSPCM eliminates the need for the computationally
expensive procedure of fitting multiple latent position cluster models (LPCMs)
with different numbers of clusters and dimensions, and then choosing the best
model using a range of model selection criteria. The performance of the LSPCM
was assessed through simulation studies and its application to sporting and
political social networks uncovered interesting and intuitive clustering
structures.

While the LSPCM demonstrated strong performance, it is important to consider
its sensitivity to specification of priors’ parameters, especially in small
networks. As in Malsiner-Walli et al., (2014), the sensitivity analyses
conducted here highlighted the importance of careful specification of the
Dirichlet hyperparameter $\nu$, with the model inherently supporting many
mixture weights close to zero when small values of $\nu$ are employed.
Although this can avoid overfitting, careful consideration and robust
sensitivity analyses with regard to choosing $\nu$ are imperative when
employing the LSPCM. Placing a gamma hyperprior on $\nu$ (Frühwirth-Schnatter
and Malsiner-Walli,, 2018; Murphy et al.,, 2020) could allow for inference on
this influential Dirichlet hyperparameter. Although our focus in this paper is
on the clustering solution, the sensitivity of the latent position dimension
threshold parameters are also important. In practice, the value of the
threshold parameter required to recover the optimal dimension may change
according to the size and density of the network. Using a prior may enable
more robust inference on this parameter.

Fitting the LSPCM is computationally feasible on networks of the scale
considered here, however, it would be computationally burdensome to apply it
to larger networks. Further, while adapting the LSPCM to facilitate modelling
of networks with more complex edge types is a natural extension, such advances
would come with additional computational cost. Addressing these issues could
be possible by employing case-control approaches for the likelihood function
(Raftery et al.,, 2012) and/or avoiding MCMC through the use of variational
inference methods (Salter-Townshend and Murphy,, 2013).

Finally, while here a MGP shrinkage prior was employed to infer the number of
dimensions and an overfitted mixture was used to infer the number of clusters,
the LSPCM could be viewed as a member of a broader family of such models given
the variety of alternative shrinkage priors and clustering approaches
available. Grushanina, (2023) provides a broad review of approaches to
infinite factorisations. For example, the Indian buffet process (IBP) has been
employed to penalise increasing dimensionality in latent factor models
(Ročková and George,, 2016; Knowles and Ghahramani,, 2011), however the
sparsity the IBP enforces could be too restrictive here. From the clustering
point of view, an infinite mixture model could be used to infer the number of
clusters. Frühwirth-Schnatter and Malsiner-Walli, (2018) discuss linkages
between infinite and overfitted mixtures, where they highlight overfitted and
infinite mixtures yield comparable clustering performance when these
hyperprior are matched.

As network data continue to emerge in a broad range of application areas, and
the detection of clusters of nodes in such networks is often of interest, the
open access provision of R code at the lspm GitLab repository will facilitate
the use of the LSPCM by practitioners.

## Acknowledgements

This publication has emanated from research conducted with the financial
support of Science Foundation Ireland under Grant number 18/CRT/6049. For the
purpose of Open Access, the author has applied a CC BY public copyright
licence to any Author Accepted Manuscript version arising from this
submission.

The authors are grateful for discussions with members of the Working Group in
Model-based Clustering which greatly contributed to this work.

## Appendix

##  Appendix A Notation and terminology

$n$: Number of nodes.
$p$: Truncation level i.e. the number of dimensions in the fitted LSPM.
$p^{*}$: The true effective dimension of the latent space.
$p_{0}$: The initial truncation level of the number of dimensions.
$p_{m}$: The posterior modal number of dimensions.
$G$: The number of mixture components.
$G_{+}$: The number of non-empty mixture components.
$G^{*}$: The true number of clusters.
$G_{m}$: The posterior modal number of non-empty components.
$\mathbf{Y}$: $n\times n$ network adjacency matrix.
$y_{i,j}$: Edge between node $i$ and node $j$.
$\mathbf{Z}$: $n\times p$ matrix of latent positions.
$z_{i\ell}$: The latent position of node $i$ in dimension $\ell$.
$q_{i,j}$: Probability of forming an edge between node $i$ and node $j$.
$\alpha$: Global parameter that captures the overall connectivity level in the
network.
$\tau_{g}$: The mixing weight of component $g$.
$c_{ig}$: A binary indicator variable of membership of node $i$ in component
$g$.
$\bm{\mu}$: $G\times p$ matrix of the mean latent positions.
$\bm{\mu}_{g}$: The mean latent position parameter for component $g$.
$\mathbf{\Omega}$: $p\times p$ precision matrix of the latent positions.
$\omega_{\ell}$: Precision/global shrinkage parameter for dimension $\ell$.
$\delta_{h}$: Shrinkage strength from dimension $h$.
$\xi$: Scaling factor in the prior covariance of the component means.
$\bm{\Theta}$: A collective term for $\bm{\tau},\bm{\mu},\bm{\Omega}$.
$a_{h}$: Shape parameter of gamma distribution for $\delta_{h}$ in dimension
$h$.
$b_{h}$: Rate parameter of the gamma distribution for $\delta_{h}$ in
dimension $h$.
$t_{h}$: Left truncation point of the gamma distribution for $\delta_{h}$.
$\kappa_{0},\kappa_{1}$: The dimension adaptation probability parameters.
$\epsilon_{1}$: The threshold for decreasing dimensions in the dimension
adaptation step.
$\epsilon_{2}$: The threshold for adding a dimension in the dimension
adaptation step.
$\epsilon_{3}$: The threshold for adding a dimension in the dimension
adaptation step when the number of active dimensions is 1.

##  Appendix B Derivation of full conditional distributions

Links between nodes $i$ and $j$ are assumed to form probabilistically from a
Bernoulli distribution:

$\displaystyle y_{i,j}$ $\displaystyle\sim\text{Bin}(q_{i,j})$

For a binary network, the probability $q_{i,j}$ is expressed in terms of a
logistic model i.e.

$\log\frac{q_{i,j}}{1-q_{i,j}}=\alpha-\|\mathbf{z}_{i}-\mathbf{z}_{j}\|^{2}_{2}.$

Denoting $\eta_{i,j}=\alpha-\|\mathbf{z}_{i}-\mathbf{z}_{j}\|^{2}_{2}$, then
$q_{i,j}=\frac{\exp(\eta_{i,j})}{1+\exp(\eta_{i,j})}.\\\ $ The likelihood
function of the LSPCM is

$\displaystyle L(\mathbf{Y}|\mathbf{Z},\alpha)$ $\displaystyle=\prod_{i\neq
j}P(y_{i,j}|\mathbf{z}_{i},\mathbf{z}_{j},\alpha)$ $\displaystyle=\prod_{i\neq
j}\left[\frac{\exp(\eta_{i,j})}{1+\exp(\eta_{i,j})}\right]^{y_{i,j}}\left[1-\frac{\exp(\eta_{i,j})}{1+\exp(\eta_{i,j})}\right]^{1-y_{i,j}}$
$\displaystyle=\prod_{i\neq
j}\frac{\exp(\eta_{i,j}y_{i,j})}{1+\exp(\eta_{i,j})},$

The joint posterior distribution of the LSPM is

$\displaystyle\mathbb{P}(\alpha,\mathbf{Z},\mathbf{C},\bm{\Theta}\mid\mathbf{Y})$
$\displaystyle\propto\mathbb{P}(\mathbf{Y}\mid\alpha,\mathbf{Z})P(\alpha)\mathbb{P}(\mathbf{Z}\mid\bm{C},\bm{\Theta})\mathbb{P}(\bm{C}\mid\bm{\tau})\mathbb{P}(\bm{\Theta})$
$\displaystyle\mathbb{P}(\alpha,\mathbf{Z},\mathbf{C},\bm{\tau},\bm{\mu},\bm{\delta}\mid\mathbf{Y})$
$\displaystyle\propto\mathbb{P}(\mathbf{Y}\mid\alpha,\mathbf{Z})\mathbb{P}(\alpha)\mathbb{P}(\mathbf{Z}\mid\bm{C},\bm{\mu},\bm{\delta})\mathbb{P}(\bm{C}\mid\bm{\tau})\mathbb{P}(\bm{\tau})\mathbb{P}(\bm{\mu}\mid\bm{\delta})\mathbb{P}(\bm{\delta})$

$\displaystyle\mathbb{P}(\alpha,\mathbf{Z},\mathbf{C},\bm{\tau},\bm{\mu},\bm{\delta}\mid\mathbf{Y})$
$\displaystyle\propto\left\\{\prod_{i\neq
j}\left[\frac{\exp(\eta_{i,j}y_{i,j})}{1+\exp(\eta_{i,j})}\right]\right\\}$
$\displaystyle\qquad\times\left\\{\frac{1}{\sqrt{2\pi\sigma_{\alpha}^{2}}}\exp\left[-\frac{1}{2\sigma_{\alpha}^{2}}(\alpha-\mu_{\alpha})^{2}\right]\right\\}$
$\displaystyle\qquad\times\prod_{i=1}^{n}\prod_{g=1}^{G}\left\\{\left(\frac{1}{2\pi}\right)^{\frac{p}{2}}(\text{det
}\mathbf{\Omega})^{\frac{1}{2}}\exp\left[-\frac{1}{2}(\mathbf{z}_{i}-\bm{\mu}_{g})^{T}\mathbf{\Omega}(\mathbf{z}_{i}-\bm{\mu}_{g})\right]\right\\}^{c_{ig}}$
$\displaystyle\qquad\times\prod_{i=1}^{n}\left[\frac{n!}{c_{i1}!\dotsm
c_{iG}!}\tau^{c_{i1}}_{1}\dotsm\tau^{c_{iG}}_{G}\right]$
$\displaystyle\qquad\times\left[\frac{1}{\beta(\nu)}\prod_{g=1}^{G}\tau_{g}^{\nu-1}\right]$
$\displaystyle\qquad\times\left\\{\prod_{g=1}^{G}\left(\frac{1}{2\pi}\right)^{\frac{p}{2}}[\text{det
}(\xi^{-1}\mathbf{\Omega})]^{-\frac{1}{2}}\exp\left[-\frac{1}{2}(\bm{\mu}_{g}-\bm{0})^{T}(\xi^{-1})\mathbf{\Omega}(\bm{\mu}_{g}-\bm{0})\right]\right\\}$
$\displaystyle\qquad\times\left\\{\frac{b_{1}^{a_{1}}}{\Gamma(a_{1})}(\delta_{1})^{a_{1}-1}\exp\left[-b_{1}(\delta_{1})\right]\right\\}$
$\displaystyle\qquad\times\left\\{\prod_{h=2}^{p}\frac{b_{2}^{a_{2}}}{\Gamma(a_{2})}(\delta_{h})^{a_{2}-1}\exp\left[-b_{2}(\delta_{h})\right]\right\\}$

Indicating with $-$ the conditioning on all the remaining variables, the full
conditional distribution for $\alpha$ is:

$\displaystyle\mathbb{P}(\alpha\mid-)$
$\displaystyle\propto\left\\{\prod_{i\neq
j}\left[\frac{\exp(\eta_{i,j}y_{i,j})}{1+\exp(\eta_{i,j})}\right]\right\\}\times\frac{1}{\sqrt{2\pi\sigma_{\alpha}^{2}}}\exp\left[-\frac{1}{2}\frac{(\alpha-\mu_{\alpha})^{2}}{\sigma_{\alpha}^{2}}\right]$
$\displaystyle\propto\left\\{\prod_{i\neq
j}\left[\frac{\exp(\eta_{i,j}y_{i,j})}{1+\exp(\eta_{i,j})}\right]\right\\}\times\exp\left[-\frac{1}{2}\frac{(\alpha-\mu_{\alpha})^{2}}{\sigma_{\alpha}^{2}}\right]$
$\displaystyle\log\mathbb{P}(\alpha\mid-)$ $\displaystyle\propto\sum_{i\neq
j}\hskip
2.0pt\left\\{\eta_{i,j}y_{i,j}-\log\left[1+\exp(\eta_{i,j})\right]\right\\}-\frac{1}{2}\frac{(\alpha-\mu_{\alpha})^{2}}{\sigma_{\alpha}^{2}}$
$\displaystyle\propto\sum_{i\neq j}\hskip
2.0pt\left\\{\eta_{i,j}y_{i,j}-\log\left[1+\exp(\eta_{i,j})\right]\right\\}-(\alpha-\mu_{\alpha})^{2}$
$\displaystyle\propto\sum_{i\neq j}\hskip
2.0pt\left\\{(\alpha-\|\mathbf{z}_{i}-\mathbf{z}_{j}\|^{2}_{2})y_{i,j}-\log\left[1+\exp(\alpha-\|\mathbf{z}_{i}-\mathbf{z}_{j}\|^{2}_{2})\right]\right\\}-(\alpha-\mu_{\alpha})^{2}$
$\displaystyle\log\mathbb{P}(\alpha\mid-)$ $\displaystyle\propto\sum_{i\neq
j}\hskip 2.0pt\left\\{\alpha
y_{i,j}-\log\left[1+\exp(\alpha-\|\mathbf{z}_{i}-\mathbf{z}_{j}\|^{2}_{2})\right]\right\\}-(\alpha-\mu_{\alpha})^{2}$

As this is not a recognisable distribution, the Metropolis-Hasting algorithm
is employed. The full conditional distribution for $\mathbf{Z}$ is:

$\displaystyle\mathbb{P}(\mathbf{Z}\mid-)$
$\displaystyle\propto\left\\{\prod_{i\neq
j}\left[\frac{\exp(\eta_{i,j}y_{i,j})}{1+\exp(\eta_{i,j})}\right]\right\\}$
$\displaystyle\qquad\times\prod_{i=1}^{n}\prod_{g=1}^{G}\left\\{\left(\frac{1}{2\pi}\right)^{\frac{p}{2}}(\text{det
}\mathbf{\Omega})^{\frac{1}{2}}\exp\left[-\frac{1}{2}(\mathbf{z}_{i}-\bm{\mu}_{g})^{T}\mathbf{\Omega}(\mathbf{z}_{i}-\bm{\mu}_{g})\right]\right\\}^{c_{ig}}$
$\displaystyle\log\mathbb{P}(\mathbf{Z}\mid-)$
$\displaystyle\propto\sum_{i\neq j}\hskip
2.0pt\left\\{\eta_{i,j}y_{i,j}-\log\left[1+\exp(\eta_{i,j})\right]\right\\}-\sum_{i=1}^{n}\sum_{g=1}^{G}\left[\frac{1}{2}(\mathbf{z}_{i}-\bm{\mu}_{g})^{T}\mathbf{\Omega}(\mathbf{z}_{i}-\bm{\mu}_{g})\right]^{c_{ig}}$
$\displaystyle\propto\sum_{i\neq j}\hskip
2.0pt\left\\{\|\mathbf{z}_{i}-\mathbf{z}_{j}\|^{2}_{2}y_{i,j}-\log\left[1+\exp(\alpha-\|\mathbf{z}_{i}-\mathbf{z}_{j}\|^{2}_{2}\right)]\right\\}$
$\displaystyle\qquad-\sum_{i=1}^{n}\sum_{g=1}^{G}\left[\frac{1}{2}(\mathbf{z}_{i}-\bm{\mu}_{g})^{T}\mathbf{\Omega}(\mathbf{z}_{i}-\bm{\mu}_{g})\right]^{c_{ig}}$

As this is not a recognisable distribution, the Metropolis-Hasting algorithm
is employed. The full conditional distribution for $\bm{c}_{ig}=1$ is:

$\displaystyle\mathbb{P}(\bm{c}_{ig}=1\mid-)$
$\displaystyle=\frac{\mathbb{P}(-\mid\bm{c}_{ig}=1)\mathbb{P}(\bm{c}_{ig}=1)}{\mathbb{P}(-)}=\frac{\mathbb{P}(\mathbf{z}_{i}\mid\bm{c}_{ig}=1)\mathbb{P}(\bm{c}_{ig}=1)}{\sum_{r=1}^{G}\mathbb{P}(\mathbf{z}_{ir}\mid\bm{c}_{ir}=1)\mathbb{P}(\bm{c}_{ir}=1)}$
$\displaystyle=\frac{\text{MVN}_{p}(\mathbf{z}_{i};\bm{\mu}_{g},\bm{\Omega}^{-1})\tau_{g}}{\sum_{r=1}^{G}\text{MVN}_{p}(\mathbf{z}_{ir};\bm{\mu}_{r},\bm{\Omega}^{-1})\tau_{r}}=\frac{\tau_{g}\text{MVN}_{p}(\mathbf{z}_{i};\bm{\mu}_{g},\bm{\Omega}^{-1})}{\sum_{r=1}^{G}\tau_{r}\text{MVN}_{p}(\mathbf{z}_{ir};\bm{\mu}_{r},\bm{\Omega}^{-1})}$

The full conditional distribution for $\tau_{g}$ is:

$\displaystyle\mathbb{P}(\tau_{g}\mid-)$
$\displaystyle\propto\left[\frac{1}{\beta(\nu)}\tau_{g}^{\nu-1}\right]\left[\prod_{i=1}^{n}\tau_{g}^{\bm{c}_{ig}}\right]$
$\displaystyle\propto\tau_{g}^{\nu-1}\tau_{g}^{\sum_{i=1}^{n}\bm{c}_{ig}}$
$\displaystyle\propto\tau_{g}^{\sum_{i=1}^{n}\bm{c}_{ig}+\nu-1}$
$\displaystyle\sim\text{Dir}(\sum_{i=1}^{n}\bm{c}_{ig}+\nu)$

The full conditional distribution for $\bm{\mu}_{g}$ is:

$\displaystyle\mathbb{P}(\bm{\mu}_{g}\mid-)$
$\displaystyle\propto\prod_{i=1}^{n}\left\\{\left(\frac{1}{2\pi}\right)^{\frac{p}{2}}(\text{det
}\mathbf{\Omega})^{\frac{1}{2}}\exp\left[-\frac{1}{2}(\mathbf{z}_{i}-\bm{\mu}_{g})^{T}\mathbf{\Omega}(\mathbf{z}_{i}-\bm{\mu}_{g})\right]\right\\}^{c_{ig}}$
$\displaystyle\qquad\times\left\\{\left(\frac{1}{2\pi}\right)^{\frac{p}{2}}[\text{det
}(\xi^{-1}\mathbf{\Omega})]^{\frac{1}{2}}\exp\left[-\frac{1}{2}(\bm{\mu}_{g}-\bm{0})^{T}(\xi^{-1})\bm{\Omega}(\bm{\mu}_{g}-\bm{0})\right]\right\\}$
$\displaystyle\propto\exp\left\\{\sum_{i=1}^{n}\left[-\frac{1}{2}(\mathbf{z}_{i}-\bm{\mu}_{g})^{T}\mathbf{\Omega}(\mathbf{z}_{i}-\bm{\mu}_{g})(c_{ig})\right]-\frac{\bm{\mu}_{g}^{T}(\xi^{-1})\bm{\Omega}\bm{\mu}_{g}}{2}\right\\}$
$\displaystyle\propto\exp\left\\{-\frac{1}{2}\left[\sum_{i=1}^{n}\left[(\mathbf{z}_{i}^{T}\bm{\Omega}\mathbf{z}_{i}c_{ig}-2\mathbf{z}_{i}^{T}\bm{\Omega}\bm{\mu}_{g}c_{ig}+\bm{\mu}_{g}^{T}\bm{\Omega}\bm{\mu}_{g}c_{ig})\right]+\bm{\mu}_{g}^{T}(\xi^{-1})\bm{\Omega}\bm{\mu}_{g}\right]\right\\}$
$\displaystyle\propto\exp\left\\{-\frac{1}{2}\left[-2\sum_{i=1}^{n}c_{ig}\mathbf{z}_{i}^{T}\bm{\Omega}\bm{\mu}_{g}+\sum_{i=1}^{n}c_{ig}\bm{\mu}_{g}^{T}\bm{\Omega}\bm{\mu}_{g}+\bm{\mu}_{g}^{T}(\xi^{-1})\bm{\Omega}\bm{\mu}_{g}\right]\right\\}$
$\displaystyle\propto\exp\left\\{-\frac{1}{2}\left[\bm{\mu}_{g}^{T}\bm{\Omega}\left(\sum_{i=1}^{n}c_{ig}+\xi^{-1}\right)\bm{\mu}_{g}-2\bm{\mu}_{g}^{T}\left(\bm{\Omega}\sum_{i=1}^{n}(c_{ig}\mathbf{z}_{i})\right)\right]\right\\}$
$\displaystyle\propto\exp\left\\{-\frac{1}{2}\left[\bm{\mu}_{g}^{T}-\left\\{\bm{\Omega}\sum_{i=1}^{n}(c_{ig}\mathbf{z}_{i})\right\\}^{T}\left\\{\bm{\Omega}\left(\sum_{i=1}^{n}c_{ig}+\xi^{-1}\right)\right\\}^{-1}\right]\left[\bm{\Omega}\left(\sum_{i=1}^{n}c_{ig}+\xi^{-1}\right)\right]\right.$
$\displaystyle\qquad\times\left.\left[\bm{\mu}_{g}-\left\\{\bm{\Omega}^{T}\left(\sum_{i=1}^{n}c_{ig}+\xi^{-1}\right)\right\\}^{-1}\left\\{\bm{\Omega}\sum_{i=1}^{n}(c_{ig}\mathbf{z}_{i})\right\\}\right]\right\\}$
since $\displaystyle\bm{\Omega}\text{ is a diagonal matrix,
}\bm{\Omega}=\bm{\Omega}^{T}\text{ and
}\bm{\Omega}\bm{\Omega}^{-1}=\mathbf{I}\text{, thus,}$
$\displaystyle\propto\exp\left\\{-\frac{1}{2}\left[\bm{\mu}_{g}-\frac{\sum_{i=1}^{n}c_{ig}\mathbf{z}_{i}}{\sum_{i=1}^{n}c_{ig}+\xi^{-1}}\right]^{T}\left[\bm{\Omega}\sum_{i=1}^{n}c_{ig}+\xi^{-1}\right]\right.$
$\displaystyle\qquad\times\left.\left[\bm{\mu}_{g}-\frac{\sum_{i=1}^{n}c_{ig}\mathbf{z}_{i}}{\sum_{i=1}^{n}c_{ig}+\xi^{-1}}\right]\right\\}$
$\displaystyle\sim\text{MVN}_{p}\left(\frac{\sum_{i=1}^{n}c_{ig}\mathbf{z}_{i}}{\sum_{i=1}^{n}c_{ig}+\xi^{-1}}\quad,\quad\left[\bm{\Omega}\left(\sum_{i=1}^{n}c_{ig}+\xi^{-1}\right)\right]^{-1}\right)$

The full conditional distribution for $\delta_{1}$ is:

$\displaystyle\mathbb{P}(\delta_{1}\mid-)$
$\displaystyle\propto\prod_{i=1}^{n}\prod_{g=1}^{G}\left\\{\left(\frac{1}{2\pi}\right)^{\frac{p}{2}}(\text{det
}\mathbf{\Omega})^{\frac{1}{2}}\exp\left[-\frac{1}{2}(\mathbf{z}_{i}-\bm{\mu}_{g})^{T}\mathbf{\Omega}(\mathbf{z}_{i}-\bm{\mu}_{g})\right]\right\\}^{c_{ig}}$
$\displaystyle\qquad\times\prod_{g=1}^{G}\left\\{\left(\frac{1}{2\pi}\right)^{\frac{p}{2}}[\text{det
}(\xi^{-1}\mathbf{\Omega})]^{\frac{1}{2}}\exp\left[-\frac{1}{2}(\bm{\mu}_{g}-\bm{0})^{T}(\xi^{-1})\bm{\Omega}(\bm{\mu}_{g}-\bm{0})\right]\right\\}$
$\displaystyle\qquad\times\left[\frac{b_{1}^{a_{1}}}{\Gamma(a_{1})}(\delta_{1})^{a_{1}-1}\exp\left(-b_{1}\delta_{1}\right)\right]$
$\displaystyle\propto\left\\{\bm{\omega}^{\frac{p\sum_{i=1}^{n}\sum_{i=g}^{G}c_{ig}}{2}}\mathbf{I}_{p}\exp\left[-\frac{1}{2}\sum_{i=1}^{n}\sum_{i=g}^{G}(\mathbf{z}_{i}-\bm{\mu}_{g})^{T}\bm{\omega}\mathbf{I}_{p}(\mathbf{z}_{i}-\bm{\mu}_{g})c_{ig}\right]\right\\}$
$\displaystyle\qquad\times\left\\{\bm{\omega}^{\frac{Gp}{2}}\mathbf{I}_{p}\exp\left[-\frac{1}{2}\sum_{g=1}^{G}\bm{\mu}_{g}^{T}(\xi^{-1}\bm{\omega})\mathbf{I}_{p}\bm{\mu}_{g}\right]\right\\}\times\left[\delta_{1}^{a_{1}-1}\exp(-b_{1}\delta_{1})\right]$
$\displaystyle\propto\left\\{\delta_{1}^{\frac{np}{2}}\exp\left[-\frac{1}{2}\sum_{i=1}^{n}\sum_{i=g}^{G}(\mathbf{z}_{i}-\bm{\mu}_{g})^{T}\left(\prod_{m=1}^{\ell}\delta_{m}\right)\mathbf{I}_{p}(\mathbf{z}_{i}-\bm{\mu}_{g})c_{ig}\right]\right\\}$
$\displaystyle\qquad\times\left\\{\delta_{1}^{\frac{Gp}{2}}\exp\left[-\frac{1}{2}\sum_{g=1}^{G}\bm{\mu}_{g}^{T}(\xi^{-1})\left(\prod_{m=1}^{\ell}\delta_{m}\right)\mathbf{I}_{p}\bm{\mu}_{g}\right]\right\\}\times\left[\delta_{1}^{a_{1}-1}\exp(-b_{1}\delta_{1})\right]$
$\displaystyle\propto\delta_{1}^{\frac{np}{2}}\delta_{1}^{\frac{Gp}{2}}\delta_{1}^{a_{1}-1}\exp\left[-\frac{1}{2}\sum_{i=1}^{n}\sum_{i=g}^{G}(\mathbf{z}_{i}-\bm{\mu}_{g})^{T}\left(\delta_{1}\prod_{m=2}^{\ell}\delta_{m}\right)\mathbf{I}_{p}(\mathbf{z}_{i}-\bm{\mu}_{g})c_{ig}\right.$
$\displaystyle\qquad\left.-\frac{1}{2}\sum_{g=1}^{G}\bm{\mu}_{g}^{T}(\xi^{-1})\left(\delta_{1}\prod_{m=2}^{\ell}\delta_{m}\right)\mathbf{I}_{p}\bm{\mu}_{g}-b_{1}\delta_{1}\right]$
$\displaystyle\propto\delta_{1}^{\frac{\left(n+G\right)p}{2}+a_{1}-1}\exp\left\\{-\left[\frac{1}{2}\sum_{i=1}^{n}\sum_{g=1}^{G}(\mathbf{z}_{i}-\bm{\mu}_{g})^{T}\left(\prod_{m=2}^{\ell}\delta_{m}\right)\mathbf{I}_{p}(\mathbf{z}_{i}-\bm{\mu}_{g})c_{ig}\right.\right.$
$\displaystyle\qquad\left.\left.+\frac{1}{2}\sum_{g=1}^{G}\bm{\mu}_{g}^{T}(\xi^{-1})\left(\prod_{m=2}^{\ell}\delta_{m}\right)\mathbf{I}_{p}\bm{\mu}_{g}+b_{1}\right]\delta_{1}\right\\}$
$\displaystyle\sim\text{Gam}\left(\frac{\left(n+G\right)p}{2}+a_{1}\quad,\right.$
$\displaystyle\qquad\left.\quad\frac{1}{2}\sum_{i=1}^{n}\sum_{g=1}^{G}(\mathbf{z}_{i}-\bm{\mu}_{g})^{T}\left(\prod_{m=2}^{\ell}\delta_{m}\right)\mathbf{I}_{p}(\mathbf{z}_{i}-\bm{\mu}_{g})c_{ig}\quad+\right.$
$\displaystyle\left.\qquad\qquad\frac{1}{2}\sum_{g=1}^{G}\bm{\mu}_{g}^{T}(\xi^{-1})\left(\prod_{m=2}^{\ell}\delta_{m}\right)\mathbf{I}_{p}\bm{\mu}_{g}+b_{1}\right)$

The full conditional distribution for $\delta_{h}$, where $h\geq 2$ is:

$\displaystyle\mathbb{P}(\delta_{h}\mid-)$
$\displaystyle\propto\prod_{i=1}^{n}\prod_{g=1}^{G}\left\\{\left(\frac{1}{2\pi}\right)^{\frac{p}{2}}(\text{det
}\mathbf{\Omega})^{\frac{1}{2}}\exp\left[-\frac{1}{2}(\mathbf{z}_{i}-\bm{\mu}_{g})^{T}\mathbf{\Omega}(\mathbf{z}_{i}-\bm{\mu}_{g})\right]\right\\}^{c_{ig}}$
$\displaystyle\qquad\times\prod_{g=1}^{G}\left\\{\left(\frac{1}{2\pi}\right)^{\frac{p}{2}}[\text{det
}(\xi^{-1}\mathbf{\Omega})]^{\frac{1}{2}}\exp\left[-\frac{1}{2}(\bm{\mu}_{g}-\bm{0})^{T}(\xi^{-1})\bm{\Omega}(\bm{\mu}_{g}-\bm{0})\right]\right\\}$
$\displaystyle\qquad\times\left[\frac{b_{2}^{a_{2}}}{\Gamma(a_{2})}(\delta_{h})^{a_{2}-1}\exp(-b_{2}\delta_{h})\right]$
$\displaystyle\propto\left\\{\bm{\omega}^{\frac{(p-h+1)\sum_{i=1}^{n}\sum_{g=1}^{G}c_{ig}}{2}}\mathbf{I}_{p}\exp\left[-\frac{1}{2}\sum_{i=1}^{n}\sum_{g=1}^{G}(\mathbf{z}_{i}-\bm{\mu}_{g})^{T}\bm{\omega}\mathbf{I}_{p}(\mathbf{z}_{i}-\bm{\mu}_{g})c_{ig}\right]\right\\}$
$\displaystyle\qquad\times\left\\{\bm{\omega}^{\frac{Gp}{2}}\mathbf{I}_{p}\exp\left[-\frac{1}{2}\sum_{g=1}^{G}\bm{\mu}_{g}^{T}(\xi^{-1}\bm{\omega})\mathbf{I}_{p}\bm{\mu}_{g}\right]\right\\}$
$\displaystyle\qquad\times\left[\delta_{h}^{a_{2}-1}\exp(-b_{2}\delta_{h})\right]$
$\displaystyle\propto\left\\{\delta_{h}^{\frac{(p-h+1)n}{2}}\exp\left[-\frac{1}{2}\sum_{i=1}^{n}\sum_{g=1}^{G}(\mathbf{z}_{i}-\bm{\mu}_{g})^{T}\left(\prod_{m=1}^{\ell}\delta_{m}\right)\mathbf{I}_{p}(\mathbf{z}_{i}-\bm{\mu}_{g})c_{ig}\right]\right\\}$
$\displaystyle\qquad\times\left\\{\delta_{h}^{\frac{G(p-h+1)}{2}}\exp\left[-\frac{1}{2}\sum_{g=1}^{G}\bm{\mu}_{g}^{T}(\xi^{-1})\left(\prod_{m=1}^{\ell}\delta_{m}\right)\mathbf{I}_{p}\bm{\mu}_{g}\right]\right\\}$
$\displaystyle\qquad\times\left[\delta_{h}^{a_{2}-1}\exp(-b_{2}\delta_{h})\right]$
$\displaystyle\propto\delta_{h}^{\frac{n(p-h+1)}{2}}\delta_{h}^{\frac{G(p-h+1)}{2}}\delta_{h}^{a_{2}-1}\exp\left[-\frac{1}{2}\sum_{i=1}^{n}\sum_{g=1}^{G}(\mathbf{z}_{i}-\bm{\mu}_{g})^{T}\left(\delta_{h}\prod_{m=1,m\neq
h}^{\ell}\delta_{m}\right)\mathbf{I}_{p}(\mathbf{z}_{i}-\bm{\mu}_{g})c_{ig}\right.$
$\displaystyle\qquad\left.-\frac{1}{2}\sum_{g=1}^{G}\bm{\mu}_{g}^{T}(\xi^{-1})\left(\delta_{h}\prod_{m=1,m\neq
h}^{\ell}\delta_{m}\right)\mathbf{I}_{p}\bm{\mu}_{g}-b_{2}\delta_{h}\right]$
$\displaystyle\propto\delta_{h}^{\frac{\left(n+G\right)(p-h+1)}{2}+a_{2}-1}\exp\left\\{-\left[\frac{1}{2}\sum_{i=1}^{n}\sum_{g=1}^{G}(\mathbf{z}_{i}-\bm{\mu}_{g})^{T}\left(\prod_{m=1,m\neq
h}^{\ell}\delta_{m}\right)\mathbf{I}_{p}(\mathbf{z}_{i}-\bm{\mu}_{g})c_{ig}\right.\right.$
$\displaystyle\qquad\left.\left.+\frac{1}{2}\sum_{g=1}^{G}\bm{\mu}_{g}^{T}(\xi^{-1})\left(\prod_{m=1,m\neq
h}^{\ell}\delta_{m}\right)\mathbf{I}_{p}\bm{\mu}_{g}+b_{2}\right]\delta_{h}\right\\}$

since $\delta_{h}$ is bounded between $[1,\infty)$,

$\displaystyle\sim\text{Gam}^{\text{T}}$
$\displaystyle\left(\frac{\left(n+G\right)(p-h+1)}{2}+a_{2}\quad,\right.$
$\displaystyle\left.\frac{1}{2}\sum_{i=1}^{n}(\mathbf{z}_{i}-\bm{\mu}_{g})^{T}\left(\prod_{m=1,m\neq
h}^{\ell}\delta_{m}\right)\mathbf{I}_{p}(\mathbf{z}_{i}-\bm{\mu}_{g})c_{ig}\quad+\right.$
$\displaystyle\left.\qquad\qquad\frac{1}{2}\sum_{g=1}^{G}\bm{\mu}_{g}^{T}(\xi^{-1})\left(\prod_{m=1,m\neq
h}^{\ell}\delta_{m}\right)\mathbf{I}_{p}\bm{\mu}_{g}+b_{2}\quad,\quad
1\right)$

##  Appendix C Simulation studies’ additional posterior distribution plots

The LSPCM performance is assessed via the posterior distributions of the
number of non-empty components (estimate of the number of clusters), number of
dimensions, latent positions’ variance, shrinkage strength, $\alpha$
parameters. Included here are supplementary plots to assist in assessing the
performance of the LSPCM.

Figure 8 shows an increasing probability that $G_{+}=3$ when $\nu$ decreases
from 0.1 to 0.001, but $G_{+}=1$ receives increasing probability when $\nu$
decreases from 0.001 to 0.00001, indicating the nodes are collapsing into one
cluster. Figure 9 indicates that, irrespective of the value of $\nu$, the
number of dimensions is well estimated with $p_{m}=2$. Figures 10, 11 and 12
demonstrate accurate parameter estimation.

On the other hand, Figure 13 shows increasingly posterior probability for the
correct $G_{+}=7$ when $\nu$ decreases from 0.1 to 0.00001. However, Figure 14
indicates more uncertain inference of the number of dimensions with notable
posterior probability for $p=2$ to $p=4$. Figures 15, 16 and 17 also show some
bias in the estimation of the parameters associated with the dimensions of the
latent space.

Figure 8: Posterior distributions on the number of non-empty components for
the simulated network with $p^{*}=2$ and $G^{*}=3$ for different values of
$\nu$. Figure 9: Posterior distributions on the number of dimensions for the
simulated network with $p^{*}=2$ and $G^{*}=3$ across different values of
$\nu$. Figure 10: For the simulated network with $p^{*}=2$ and $G^{*}=3$ the
posterior distribution of the shrinkage strength parameter across different
values of $\nu$. Figure 11: For the simulated network with $p^{*}=2$ and
$G^{*}=3$, the posterior distribution of the variance parameters across
different values of $\nu$. Figure 12: For the simulated network with $p^{*}=2$
and $G^{*}=3$, the posterior distribution of $\alpha$ across different values
of $\nu$. Figure 13: For the simulated network with $p^{*}=4$ and $G^{*}=7$,
the posterior distribution on the number of clusters across different values
of $\nu$. Figure 14: For the simulated network with $p^{*}=4$ and $G^{*}=7$,
the posterior distribution of the the number of dimensions across different
values of $\nu$. Figure 15: For the simulated network with $p^{*}=4$ and
$G^{*}=7$, the posterior distributions of the shrinkage strength parameters
across dimensions for different values of $\nu$. Figure 16: For the simulated
network with $p^{*}=4$ and $G^{*}=7$, the posterior distributions of the
variance parameters across dimensions for different values of $\nu$. Figure
17: For the simulated network with $p^{*}=4$ and $G^{*}=7$, the posterior
distribution of $\alpha$ for different values of $\nu$.

##  Appendix D Additional results from analyses of Twitter networks

Supplementary results from applying the LSPCM to the Twitter networks are
provided here, including the posterior distributions of the variance,
shrinkage strength, and $\alpha$ parameters. As noted in Gwee et al., (2023),
different dimensions solution will gives bias estimate with the $\alpha$
parameter being very obvious with its own set of violin plots separated from
another.

(a)

(b)

(c)

Figure 18: For the football Twitter network, (a) the posterior distributions
of the shrinkage strength parameters across dimensions, (b) the posterior
distributions of the variance parameters across dimensions, and (c) the
posterior distribution of $\alpha$.

(a)

(b)

(c)

Figure 19: For the Irish politicians Twitter network, (a) the posterior
distribution of the shrinkage strength parameter across dimensions, (b) the
posterior distributions of the variance parameters across dimensions, and (c)
the posterior distribution of $\alpha$.

## References

  * Aliverti and Durante, (2019) Aliverti, E. and Durante, D. (2019).  Spatial modeling of brain connectivity data via latent distance models with nodes clustering.  Statistical Analysis and Data Mining: The ASA Data Science Journal, 12:185–196.
  * Bhattacharya and Dunson, (2011) Bhattacharya, A. and Dunson, D. B. (2011).  Sparse Bayesian infinite factor models.  Biometrika, 98:291–306.
  * Cox and Cox, (2001) Cox, T. F. and Cox, M. A. A. (2001).  Multidimensional scaling.  Chapman & Hall/CRC, 2nd ed edition.
  * Durante, (2017) Durante, D. (2017).  A note on the multiplicative gamma process.  Statistics & Probability Letters, 122:198–204.
  * Durante and Dunson, (2014) Durante, D. and Dunson, D. B. (2014).  Nonparametric Bayes dynamic modelling of relational data.  Biometrika, 101:883–898.
  * D’Angelo et al., (2023) D’Angelo, S., Alfò, M., and Fop, M. (2023).  Model-based clustering for multidimensional social networks.  Journal of the Royal Statistical Society Series A: Statistics in Society, 186:481–507.
  * D’Angelo et al., (2019) D’Angelo, S., Murphy, T. B., and Alfò, M. (2019).  Latent space modelling of multidimensional networks with application to the exchange of votes in Eurovision song contest.  The Annals of Applied Statistics, 13.
  * Erdős and Rényi, (1959) Erdős, P. and Rényi, A. (1959).  On random graphs. i.  Publicationes Mathematicae, 6:290–297.
  * Fritsch, (2022) Fritsch, A. (2022).  mcclust: Process an MCMC Sample of Clusterings.  R package version 1.0.1.
  * Fritsch and Ickstadt, (2009) Fritsch, A. and Ickstadt, K. (2009).  Improved criteria for clustering based on the posterior similarity matrix.  Bayesian Analysis, 4:367–391.
  * Frühwirth-Schnatter et al., (2019) Frühwirth-Schnatter, S., Celeux, G., and Robert, C. P. (2019).  Handbook of mixture analysis.  CRC Press, Taylor &; Francis Group.
  * Frühwirth-Schnatter and Malsiner-Walli, (2018) Frühwirth-Schnatter, S. and Malsiner-Walli, G. (2018).  From here to infinity: sparse finite versus Dirichlet process mixtures in model-based clustering.  Advances in Data Analysis and Classification, 13:33–64.
  * Gilbert, (1959) Gilbert, E. N. (1959).  Random graphs.  The Annals of Mathematical Statistics, 30:1141–1144.
  * Gollini and Murphy, (2016) Gollini, I. and Murphy, T. B. (2016).  Joint modeling of multiple network views.  Journal of Computational and Graphical Statistics, 25:246–265.
  * Gormley and Murphy, (2010) Gormley, I. C. and Murphy, T. B. (2010).  A mixture of experts latent position cluster model for social network data.  Statistical Methodology, 7:385–405.
  * Greene and Cunningham, (2013) Greene, D. and Cunningham, P. (2013).  Producing a unified graph representation from multiple social network views.  WebSci ’13: Proceedings of the 5th Annual ACM Web Science Conference, page 118–121.
  * Grushanina, (2023) Grushanina, M. (2023).  A review of bayesian methods for infinite factorisations.  arXiv preprint arXiv:2309.12990.
  * Gwee et al., (2023) Gwee, X. Y., Gormley, I. C., and Fop, M. (2023).  A latent shrinkage position model for binary and count network data.  Bayesian Analysis, accepted.
  * Handcock et al., (2007) Handcock, M. S., Raftery, A. E., and Tantrum, J. M. (2007).  Model-based clustering for social networks.  Journal of the Royal Statistical Society: Series A (Statistics in Society), 170:301–354.
  * Hoff et al., (2002) Hoff, P. D., Raftery, A. E., and Handcock, M. S. (2002).  Latent space approaches to social network analysis.  Journal of the American Statistical Association, 97:1090–1098.
  * Holland et al., (1983) Holland, P. W., Laskey, K. B., and Leinhardt, S. (1983).  Stochastic blockmodels: First steps.  Social Networks, 5:109–137.
  * Hubert and Arabie, (1985) Hubert, L. and Arabie, P. (1985).  Comparing partitions.  Journal of Classification, 2:193–218.
  * Jo et al., (2021) Jo, W., Chang, D., You, M., and Ghim, G.-H. (2021).  A social network analysis of the spread of COVID-19 in South Korea and policy implications.  Scientific Reports, 11.
  * Kaur et al., (2023) Kaur, H., Rastelli, R., Friel, N., and Raftery, A. E. (2023).  Latent position network models.  arXiv 2304.02979.
  * Knowles and Ghahramani, (2011) Knowles, D. and Ghahramani, Z. (2011).  Nonparametric bayesian sparse factor models with application to gene expression modeling.  The Annals of Applied Statistics, 5(2B):1534 – 1552.
  * Kolaczyk and Csárdi, (2020) Kolaczyk, E. D. and Csárdi, G. (2020).  Statistical analysis of network data with R.  Springer, 2nd ed edition.
  * Krivitsky and Handcock, (2008) Krivitsky, P. N. and Handcock, M. S. (2008).  Fitting position latent cluster models for social networks with latentnet.  Journal of Statistical Software, 24.
  * Krivitsky et al., (2009) Krivitsky, P. N., Handcock, M. S., Raftery, A. E., and Hoff, P. D. (2009).  Representing degree distributions, clustering, and homophily in social networks with latent cluster random effects models.  Social Networks, 31:204–213.
  * Krivitsky et al., (2022) Krivitsky, P. N., Handcock, M. S., Shortreed, S. M., Tantrum, J., Hoff, P. D., Wang, L., Li, K., Fisher, J., and Bates, J. T. (2022).  latentnet: Latent position and cluster models for statistical networks.
  * Liu and Chen, (2023) Liu, Y. and Chen, Y. (2023).  Variational inference for latent space models for dynamic networks.  Statistica Sinica, 32.
  * Malsiner-Walli et al., (2014) Malsiner-Walli, G., Frühwirth-Schnatter, S., and Grün, B. (2014).  Model-based clustering based on sparse finite Gaussian mixtures.  Statistics and Computing, 26:303–324.
  * Malsiner-Walli et al., (2017) Malsiner-Walli, G., Frühwirth-Schnatter, S., and Grün, B. (2017).  Identifying mixtures of mixtures using Bayesian estimation.  Journal of Computational and Graphical Statistics, 26:285–295.
  * Murphy et al., (2020) Murphy, K., Viroli, C., and Gormley, I. C. (2020).  Infinite mixtures of infinite factor analysers.  Bayesian Analysis, 15.
  * Ng et al., (2021) Ng, J., Murphy, T. E., Westling, T., McCormick, T. H., and Fosdick, B. K. (2021).  Modeling the social media relationships of Irish politicians using a generalized latent space stochastic blockmodel.  The Annals of Applied Statistics, 15.
  * Passino and Heard, (2020) Passino, F. S. and Heard, N. A. (2020).  Bayesian estimation of the latent dimension and communities in stochastic blockmodels.  Statistics and Computing, 30:1291–1307.
  * Peres-Neto and Jackson, (2001) Peres-Neto, P. R. and Jackson, D. A. (2001).  How well do multivariate data sets match? The advantages of a procrustean superimposition approach over the Mantel test.  Oecologia, 129:169–178.
  * R Core Team, (2023) R Core Team (2023).  R: A language and environment for statistical computing.  CRAN.
  * Raftery et al., (2012) Raftery, A. E., Niu, X., Hoff, P. D., and Yeung, K. Y. (2012).  Fast inference for the latent space network model using a case-control approximate likelihood.  Journal of Computational and Graphical Statistics, 21:901–919.
  * Ročková and George, (2016) Ročková, V. and George, E. I. (2016).  Fast bayesian factor analysis via automatic rotations to sparsity.  Journal of the American Statistical Association, 111(516):1608–1622.
  * Ryan et al., (2017) Ryan, C., Wyse, J., and Friel, N. (2017).  Bayesian model selection for the latent position cluster model for social networks.  Network Science, 5(1):70–91.
  * Salter-Townshend and Murphy, (2013) Salter-Townshend, M. and Murphy, T. B. (2013).  Variational Bayesian inference for the latent position cluster model for network data.  Computational Statistics & Data Analysis, 57:661–671.
  * Scrucca et al., (2016) Scrucca, L., Fop, M., Murphy, T. B., and Raftery, A. E. (2016).  mclust 5: clustering, classification and density estimation using Gaussian finite mixture models.  The R Journal, 8(1):289–317.
  * Sewell, (2020) Sewell, D. K. (2020).  Model-based edge clustering.  Journal of Computational and Graphical Statistics, 30:390–405.
  * Sewell and Chen, (2017) Sewell, D. K. and Chen, Y. (2017).  Latent space approaches to community detection in dynamic networks.  Bayesian Analysis, 12:351–377.
  * Snijders and Nowicki, (1997) Snijders, T. A. and Nowicki, K. (1997).  Estimation and prediction for stochastic blockmodels for graphs with latent block structure.  Journal of Classification, 14:75–100.
  * Sosa and Betancourt, (2022) Sosa, J. and Betancourt, B. (2022).  A latent space model for multilayer network data.  Computational Statistics & Data Analysis, 169:107432.
  * Yang et al., (2020) Yang, C., Priebe, C. E., Park, Y., and Marchette, D. J. (2020).  Simultaneous dimensionality and complexity model selection for spectral graph clustering.  Journal of Computational and Graphical Statistics, 30:422–441.