# Dynamic Prototype Adaptation with Distillation for
Few-shot Point Cloud Segmentation

Jie Liu1 , Wenzhe Yin1, Haochen Wang1 , Yunlu Chen2, Jan-Jakob Sonke3,
Efstratios Gavves1
1{j.liu5, w.yin, h.wang3<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
1University of Amsterdam, Netherlands 2Carnegie Mellon University, USA 3The
Netherlands Cancer Institute, Netherlands

###### Abstract

Few-shot point cloud segmentation seeks to generate per-point masks for
previously unseen categories, using only a minimal set of annotated point
clouds as reference. Existing prototype-based methods rely on support
prototypes to guide the segmentation of query point clouds, but they encounter
challenges when significant object variations exist between the support
prototypes and query features. In this work, we present dynamic prototype
adaptation (DPA), which explicitly learns task-specific prototypes for each
query point cloud to tackle the object variation problem. DPA achieves the
adaptation through prototype rectification, aligning vanilla prototypes from
support with the query feature distribution, and prototype-to-query attention,
extracting task-specific context from query point clouds. Furthermore, we
introduce a prototype distillation regularization term, enabling knowledge
transfer between early-stage prototypes and their deeper counterparts during
adaption. By iteratively applying these adaptations, we generate task-specific
prototypes for accurate mask predictions on query point clouds. Extensive
experiments on two popular benchmarks show that DPA surpasses state-of-the-art
methods by a significant margin, e.g., 7.43% and 6.39% under the 2-way 1-shot
setting on S3DIS and ScanNet, respectively. Code is available at
https://github.com/jliu4ai/DPA.

##  1 Introduction

Point cloud semantic segmentation is a critical computer vision task with
wide-ranging applications, such as autonomous driving, robotics, and augmented
reality. It involves assigning semantic labels to individual points in a 3D
point cloud, providing valuable insights into the scene’s geometry and
semantics. Although existing point cloud segmentation methods [17, 18, 8, 27,
14, 13, 9, 30] have achieved remarkable performance within the supervised
learning paradigm, they heavily rely on large annotated training datasets,
which are both time-consuming and labor-intensive to create. Furthermore,
these methods often encounter challenges when confronted with novel classes
not present in the training data.

Figure 1: Illustration of the proposed method. Support and query point clouds
often exhibit significant object variations, which can result in prototypes
obtained from the support data being unsuitable for segmenting the query point
cloud accurately. In this work, we propose to generate task-specific
prototypes for the query point cloud by dynamically adapting prototypes with
prototype distillation into query feature distribution.

By leveraging a small set of annotated support point clouds as a reference,
few-shot point cloud segmentation methods [32, 16, 28, 6] provides a promising
solution to address the aforementioned limitations. These methods follow a
prototype-based paradigm, where prototypes are extracted from the support
point clouds and utilized to segment the query point cloud. For instance, MPTI
[32] utilizes a transductive label propagation to exploit the affinity between
multiple support prototypes and query features. BFG [16] introduces a
bidirectional feature globalization strategy between labeled prototypes and
query features to enhance the generalization ability of point features.
Despite significant progress, existing methods in few-shot point cloud
segmentation still encounter challenges due to notable object variations
between support and query features [6]. For example, objects from the same
class but different scenes often exhibit different scales, appearances, and
feature variations. Consequently, the vanilla prototypes generated from
support features may not be well-suited for accurately segmenting query point
clouds.

To tackle the issue of object variations, we propose to generate task-specific
prototypes for the query point cloud. This is achieved by dynamically adapting
vanilla prototypes through prototype distillation from the support to the
query feature distribution. As depicted in Figure 1, there is typically a
significant gap between the support and query feature distributions, directly
applying vanilla prototypes to segment query point cloud would cause inferior
performance. Our motivation is to map the vanilla prototypes from the support
feature distribution to the query feature distribution, enabling the model to
generate task-specific prototypes well-suited for accurately segmenting the
query point cloud. The prototype distillation process between the prototypes
is introduced to enhance the prototype adaptation, enabling information
exchange between prototypes from different adaptation stages.

The prototype adaptation process consists of three key components: prototype
rectification, prototype-to-query attention, and prototype distillation.
Initially, vanilla prototypes are initialized using support features, which
exhibit a significant feature gap with the query point cloud. To bridge this
gap, prototype rectification is employed to map the vanilla prototypes from
the support feature distribution to the query feature distribution, aligning
the prototypes with the characteristics of the query point cloud.
Additionally, the prototype-to-query attention mechanism aggregates context
information from the query features into the prototypes, enabling the
generation of task-specific prototypes for each query point cloud and
enhancing the adaptation to specific segmentation tasks. To further improve
prototype adaptation, a prototype distillation regularization term is
proposed, enabling early-stage prototypes to learn from their deeper
counterparts. This facilitates knowledge transfer and refines the prototype
representation during adaptation. By iteratively applying the above adaptation
process, we generate task-specific prototypes that act as optimal classifiers
for the segmentation of query point clouds.

In a nutshell, the main contributions of our work are:

$\bullet$ We propose DPA, an end-to-end Dynamic Prototype Adaptation framework
that addresses object variation issues between support and query features by
adapting vanilla prototypes to task-specific prototypes.

$\bullet$ We introduce prototype rectification and prototype-to-query
attention mechanisms to facilitate the adaptation process. Additionally, we
design a prototype distillation regularization to further enhance the
adaptation.

$\bullet$ We achieve state-of-the-art performance on two few-shot point cloud
segmentation benchmarks, i.e., S3DIS and ScanNet. Notably, our method
outperforms previous SOTA by 7.43% and 6.39% under the 2-way 1-shot setting on
S3DIS and ScanNet, respectively.

##  2 Related Work

###  2.1 Point Cloud Semantic Segmentation

Point cloud semantic segmentation [8, 27, 14, 13, 9, 30], which seeks to
allocate specific labels to each individual point within a given point cloud,
has been extensively studied in the computer vision community. PointNet [17]
is the pioneering work that adopts an end-to-end symmetric MLP network to
segment raw point clouds. Later on point-based methods have sprung up due to
its efficiency and simplicity. Point-wise MLP methods employ shared MLP layers
as basic blocks of the network to extract features, such as PointNet++ [18],
ShellNet [31], and PointSIFT [10]. Inspired by the 2D convolution on image,
point convolution methods design point-cloud-based convolution operations,
including KPConv [20], Deformable-Filter [25], DPC [5], and PointConv [24]. To
extract the spatial geometric features of points, graph-based methods [23, 22,
11] construct graphs inside point sets and design novel graph convolutions.
While these methods have demonstrated promising results in point cloud
segmentation, they often cannot generalize well to novel categories not
previously seen in the data. In this work, we tackle the point cloud semantic
segmentation task of unseen classes with just a few labeled samples.

###  2.2 Few-shot Point Cloud Semantic Segmentation

Few-shot point cloud semantic segmentation extends the general point cloud
semantic segmentation task to the few-shot scenario, where the model is
endowed with the ability to segment novel classes with limited annotated
support data. AttMPTI [32] represents pioneering work in the realm of few-shot
point cloud segmentation. It introduces a graph network to facilitate
transductive inference for query point clouds. BFG [16] proposes a
bidirectional feature globalization method to aggregate global information
from both support and query data. SCAT [28] introduces a class-specific
attention based transformer network to improve both performance and efficiency
for few-shot point cloud segmentation. QGPA [6] proposes a prototype
adaptation and projection mechanism to achieve prototype refinement, which
leads to significant performance improvement. [33] introduces depth
information to achieve cross-modal few-shot point cloud segmentation. Despite
recent progress, the object variation problem, i.e., the objects with the same
label but from different point clouds exhibit scale, appearance, and feature
variations, is still challenging for current methods. In this work, we explore
adopting dynamic prototype adaptation to address the object variation problem
for few-shot point cloud segmentation.

###  2.3 Knowledge Distillation

Knowledge distillation (KD) [2, 7] aims to transfer knowledge, such as logits
or intermediate features, from a high-capacity teacher model to a lightweight
student network. It has been applied in various domains, including computer
vision and natural language processing, and has shown to be effective for
model compression, regularization, and few-shot learning scenarios. Despite
its competitive generalization improvement, pretraining great teacher model
requires extra training time and computation cost [19]. To enhance efficiency
in knowledge transferring and eliminate the requirement for extra teacher
networks, self-distillation is proposed to distill knowledge from the model
itself. There are three popular ways to achieve self knowledge distillation,
i.e., 1) data-distortion based self-distillation [12, 26], 2) regard history
information as the teacher model, 3) distilling across auxiliary head [15,
29]. In this work, we propose a prototype distillation scheme that regularizes
the model to learn task-specific prototypes for the few-shot point cloud
segmentation task.

##  3 Method

Figure 2: Diagram of our proposed method. (a) Encoder: Given the support and
query point clouds, the feature encoder $\mathcal{E}$ extracts per-point
feature $f_{s}$ and $f_{q}$, respectively. The prototype encoder transforms
support features into the set of vanilla prototypes $\mathbb{P}$. (b)
Prototype Decoder: The prototype decoder adapts vanilla prototypes to task-
specific prototypes $\tilde{\mathbb{P}}$ through $L$ transformer-based decoder
blocks, which is composed of prototype rectification, prototype-to-query
attention, and prototype distillation. The prototype distillation is
introduced to enable early-stage prototypes $\hat{\mathbb{P}}$ to glean
insights from their deeper counterparts $\tilde{\mathbb{P}}$. (c) Mask
Decoder: The mask prediction module generates per-class mask logits and then
produces a final prediction using a softmax.

###  3.1 Preliminary

Following previous few-shot point cloud segmentation methods [32, 6, 16], we
adopt the episodic paradigm. Specifically, all classes in the dataset are
divided into seen class set $C_{\text{seen}}$ and unseen class set
$C_{\text{unseen}}$, and $C_{\text{seen}}\cap C_{\text{unseen}}=\varnothing$.
Each few-shot task (a.k.a. an episode)
$\mathcal{T}=\\{\mathcal{S},\mathcal{Q}\\}$, where $\mathcal{S}$ and
$\mathcal{Q}$ are the support and query set, respectively, is instantiated as
an $N$-way $K$-shot point cloud segmentation task. The support set
$\mathcal{S}=\\{(\mathbf{I}_{s}^{1,k},\mathbf{M}_{s}^{1,k}),\ldots,(\mathbf{I}_{s}^{N,k},\mathbf{M}_{s}^{N,k})\\}_{k=1}^{K}$
consists of $K$ annotated support point clouds $\mathbf{I}_{s}^{n,k}$ and its
binary mask $\mathbf{M}_{s}^{n,k}\in\mathbb{R}^{T\times 1}$ for each of the
$N$ unique classes. The query set
$\mathcal{Q}=\\{(\mathbf{I}_{q}^{i},\mathbf{M}_{q}^{i})\\}_{i=1}^{H}$ contains
$H$ (usually $H=N$) pairs of query point cloud $\mathbf{I}_{q}^{i}$ and its
corresponding ground-truth mask $\mathbf{M}_{q}^{i}\in\mathbb{R}^{T\times 1}$,
which is only available at training. Normally, each point cloud
$\mathbf{I}\in\mathbb{R}^{T\times(3+f_{0})}$ contains $T$ points associated
with the coordinate information $\in\mathbb{R}^{3}$ and an additional feature
$\in\mathbb{R}^{f_{0}}$, e.g., color. Given an $N$-way $K$-shot task, few-shot
point cloud segmentation aims to predict the label
$\mathbf{\hat{M}}_{q}\in\mathbb{R}^{T\times(N+1)}$ for any query point cloud
$\mathbf{I}_{q}$ based on the support set $\mathcal{S}$.

###  3.2 Overview

The diagram of our proposed method is illustrated in Fig. 2, which consists of
three important modules, encoder, prototype decoder, and mask prediction.
Given support and query point clouds, the feature encoder $\mathcal{E}$
extracts per-point features $\mathbf{f}_{s}$ and $\mathbf{f}_{q}$ from the
support and query point clouds, respectively. The prototype encoder generates
initial prototypes $\mathbf{p}$ from the support feature. Then, the prototype
decoder, which is composed of prototype rectification, prototype-to-query
attention, and FFN, is introduced to refine initial prototypes and adapt them
into task-specific prototypes $\mathbf{p}^{*}$. To achieve effective prototype
adaptation, we introduce prototype distillation regularization to enable
early-stage prototypes to glean insights from their deeper counterparts during
adaptation. Finally, in the mask prediction module, the cosine similarity is
employed between the query feature $\mathbf{f}_{q}$ and the task-specific
prototype $\mathbf{p}^{*}$ to produce mask logits for generating the final
mask prediction $\mathbf{M}_{q}^{*}$. The cross-entropy loss between
prediction $\mathbf{M}_{q}^{*}$ and ground truth $\mathbf{M}_{q}$ is used to
supervise the model training.

###  3.3 Encoder

Feature Encoder. We adopt DGCNN [23], a dynamic graph CNN architecture, as the
backbone of the feature encoder. Following Zhao et al. [32], we apply a self-
attention network (SAN) on the generated semantic feature to mine semantic
correlation between points in the global context. Given a task, we have the
query point cloud $\mathbf{I}_{q}$, the support point cloud
$\mathbf{I}_{s}^{n,k}$ and its binary mask $\mathbf{M}_{s}^{n,k}$, where
$n\in\\{1,\ldots,N\\}$ and $k\in\\{1,\ldots,K\\}$ are the $N$-way $K$-shot
indexes, respectively. Formally, the extracted features are given by:

$\displaystyle\mathbf{f}_{q}$
$\displaystyle=\mathcal{E}(\mathbf{f}_{q})\in\mathbb{R}^{T\times d}$ (1)
$\displaystyle\mathbf{f}_{s}^{n,k}$
$\displaystyle=\mathcal{E}(\mathbf{I}_{s}^{n,k}\odot\mathbf{M}_{s}^{n,k})\in\mathbb{R}^{T\times
d},$

where $\mathcal{E}$ represents the feature encoder, $T$ and $d$ denote the
number and channel of point features, respectively.

Prototype Encoder. In the prototype encoder, we generate a prototype for each
target category by conducting masked average pooling over support features
with corresponding masks. The prototype for category $n$ is obtained by:

$\textbf{p}^{n}=\frac{1}{K}\sum_{k}\frac{\sum_{x}\mathbf{f}_{s,x}^{n,k}\mathbb{I}(\mathbf{M}_{s,x}^{n,k}=n)}{\sum_{x}\mathbb{I}(\mathbf{M}_{s,x}^{n,k}=n)},$
(2)

where $x\in{1,\ldots,N}$ denotes the coordinate position and $\mathbb{I}(*)$
is the indicator that outputs 1 when * is true. Besides, we compute a
prototype $\mathbf{p}^{0}$ to represent the points that belong to background
category:

$\textbf{p}^{0}=\frac{1}{NK}\sum_{n,k}\frac{\sum_{x}\mathbf{f}_{s,x}^{n,k}\mathbb{I}(\mathbf{M}_{s,x}^{n,k}\notin\\{1,\ldots,N\\})}{\sum_{x}\mathbb{I}(\mathbf{M}_{s,x}^{n,k}\notin\\{1,\ldots,N\\})}.$
(3)

Considering the $N$-way $K$-shot few-shot point cloud segmentation task, the
prototype encoder generates the vanilla prototype set
$\mathbb{P}=\\{\mathbf{p}^{0},\mathbf{p}^{1},\ldots,\mathbf{p}^{N}\\}$.

###  3.4 Prototype Decoder

Support and query point clouds often exhibit large object variations, which
can result in the vanilla prototypes obtained from support objects being
unsuitable for the segmentation of the query point cloud. To address this
critical issue, we propose a transformer-based prototype decoder that adapts
the vanilla prototypes to task-specific prototypes, ensuring their suitability
for query point cloud segmentation. Specifically, the prototype decoder
comprises of $L$ decoder blocks, with each block consisting of three key
modules: prototype rectification, prototype-to-query attention, and prototype
distillation. We take one block as an example and provide detailed
explanations of these modules in the following text.

Prototype Rectification. Due to the significant feature variations between the
vanilla prototype set $\mathbb{P}$ and the query feature $\mathbf{f}_{q}$, we
introduce the prototype rectification technique to adjust the vanilla
prototypes and align them with the distribution of the query feature.
Specifically, the rectified prototype
$\mathbf{\hat{p}}^{i}\in\mathbb{R}^{1\times d}$, where
$i\in\\{0,1,\ldots,N\\}$, is defined as:

$\mathbf{\hat{p}}^{i}=\mathbf{p}^{i}+\mathbf{A}^{i}(\mathbf{p}^{i})^{\top},$
(4)

where $\mathbf{A}^{i}\in\mathbb{R}^{d\times d}$ is the transformation matrix,
and the original prototype is added as a residual connection for the stable
training. The transformation matrix $\mathbf{A}^{i}$ plays a crucial role in
achieving accurate prototype rectification. It needs to fulfill two essential
conditions: 1) bridging the distribution gap between support and query
features, and 2) introducing more diverse query information. To this end, we
formulate the transformation matrix $\mathbf{A}^{i}$ as:

$\mathbf{A}^{i}=\text{softmax}((\mathbf{f}_{q}^{i})^{\top}\mathbf{f}_{s}^{*}),$
(5)

where $\mathbf{f}_{s}^{*}\in\mathbb{R}^{T\times d}$ is the support set
representation that captures the statistics of all support features, and we
simply define it as
$\mathbf{f}_{s}^{*}=\frac{1}{NK}\sum_{n,k}\mathbf{f}_{s}^{n,k}$. The query
feature $\mathbf{f}_{q}^{i}\in\mathbb{R}^{T\times d}$,
$i\in\\{0,1,\ldots,N\\}$ is from the query set that contains $N$ query point
clouds, where $\mathbf{f}_{q}^{0}=\frac{1}{N}\sum_{i}\mathbf{f}_{q}^{i}$. The
transformation matrix $\mathbf{A}^{i}$ establishes channel-wise
correspondences between query and support features, thus bridging their
distribution gap. Since the transformation matrix $\mathbf{A}^{i}$ varies with
$\mathbf{f}_{q}^{i}$, we incorporate more diverse query information when
computing the transformation matrix $\mathbf{A}^{i}$ for each prototype
$\mathbf{p}^{i}$. Following Eq.(4) to Eq.(5), $N+1$ initial prototypes
$\mathbb{P}=\\{\mathbf{p}^{0},\mathbf{p}^{1},\ldots,\mathbf{p}^{N}\\}$ are
rectified as
$\mathbb{\hat{P}}=\\{\mathbf{\hat{p}}^{0},\mathbf{\hat{p}}^{1},\ldots,\mathbf{\hat{p}}^{N}\\}$.
The prototype rectification bridges the distribution gap between support and
query features, enhancing compatibility between prototypes and the query point
cloud.

Prototype-to-Query Attention. While prototype rectification aligns prototypes
with query features, rectified prototypes still lack query-specific context
information. To generate task-specific prototypes, i.e., customized prototypes
for a given query point cloud, we introduce prototype-to-query attention,
which aggregates context information from the query feature into the
prototypes.

Given the rectified prototype $\mathbf{\hat{p}}^{i}$ and a specific query
feature $\mathbf{f}_{q}$, the cross-attention between the prototype and query
feature is represented as:

$\mathbf{\dot{p}}^{i}=\text{softmax}(\mathbf{Q}\mathbf{K}^{\top})\textbf{V}+\mathbf{\hat{p}}^{i}.$
(6)

Here, $\mathbf{Q}=f_{Q}(\mathbf{\hat{p}}^{i})\in\mathbb{R}^{1\times d}$, where
$f_{Q}(\cdot)$ is the transformation function.
$\mathbf{K},\mathbf{V}\in\mathbb{R}^{T\times d}$ are the query feature under
transformation $f_{K}(\cdot)$ and $f_{V}(\cdot)$, respectively. We implement
$f_{Q},f_{K},f_{V}$ as linear transformations. We further feed
$\mathbf{\dot{p}}^{i}$ into a feed-forward network (FFN) to get the final
task-specific prototype:

$\mathbf{\tilde{p}}^{i}=\text{FFN}(\mathbf{\dot{p}}^{i})+\mathbf{\dot{p}}^{i},$
(7)

where $\mathbf{\tilde{p}}^{i}$ is the task-specific prototype tailored for the
query point cloud, and the FFN is implemented as multiple 1D convolution
layers. Following Eq.(6) to Eq.(7), we could get the final task-specific
prototype set
$\mathbb{\tilde{P}}=\\{\mathbf{\tilde{p}}^{0},\mathbf{\tilde{p}}^{1},\ldots,\mathbf{\tilde{p}}^{N}\\}$,
which is more accurate than vanilla prototype set $\mathbb{P}$ for the
segmentation of query point cloud.

Prototype Distillation. The proposed prototype-decoder adapts initial
prototypes into task-specific prototypes. To enhance the prototype adaptation,
we introduce a prototype distillation regularization term, which enables
early-stage prototypes to glean insights from their deeper counterparts.
Specifically, the prototype distillation regularization is formulated as a
self-distillation paradigm :

$\mathcal{L}_{kd}=\text{KL}(\mathbb{\hat{P}}||\mathbb{\tilde{P}}),$ (8)

where KL refers to the Kullback-Leibler divergence. In our case, we utilize
$\mathbb{\hat{P}}$ as the student prototypes and $\mathbb{\tilde{P}}$ as the
teacher prototypes. By distilling knowledge through the approximation of
$\mathbb{\tilde{P}}$ by $\mathbb{\hat{P}}$, the self-distillation process
enables effective knowledge transfer between prototypes from different
adaptation stages and enhances the adaptation of vanilla prototypes to task-
specific prototypes.

###  3.5 Mask Prediction

After $L$ steps of prototype adaptation in the prototype decoder, we generate
task-specific prototypes for query mask prediction. We adopt a non-parametric
metric learning paradigm to perform segmentation. Since segmentation can be
seen as classification at each point location, we calculate the distance
between the query feature vector at each point location with each prototype.
Subsequently, we apply a softmax function to the distances, generating mask
logits that represent the different semantic classes, including the
background. Concretely, given a distance function $d$, let
$\mathbb{\tilde{P}}=\\{\mathbf{\tilde{p}}^{0},\mathbf{\tilde{p}}^{1},\ldots,\mathbf{\tilde{p}}^{N}\\}$
be the task-specific prototype set for the query feature $\mathbf{f}_{q}$. For
each task-specific prototype $\tilde{\mathbf{p}}^{i}$, mask logits are defined
as:

$\mathbf{\hat{M}}_{q}^{i}=\frac{\text{exp}(-\alpha
d(\mathbf{f}_{q},\tilde{\mathbf{p}}^{i}))}{\sum_{\tilde{\mathbf{p}}^{i}\in\mathbb{R}}\text{exp}(-\alpha
d(\mathbf{f}_{q},\tilde{\mathbf{p}}^{i}))},$ (9)

where $\alpha$ is an amplification factor and we set it as 1.0 in our
experiments. The predicted segmentation mask is then given by:

$\mathbf{\hat{M}}_{q}=\mathop{\text{arg
max}}\limits_{i}\mathbf{\hat{M}}_{q}^{i}.$ (10)

###  3.6 Objective

At the training stage, the proposed model is supervised by two loss functions,
i.e., the cross-entropy loss CE between the prediction $\mathbf{\hat{M}}_{q}$
and ground-truth $\mathbf{M}_{q}$, and the self-distillation regularization
term. Formally, the overall loss is defined as:

$\mathcal{L}=\text{CE}(\mathbf{\hat{M}}_{q},\mathbf{M}_{q})+\gamma\mathcal{L}_{kd},$
(11)

Where $\gamma$ is the balancing loss weight and we set it as 0.1 in our
experiments. The cross-entropy loss works as the main optimization objective,
which aims to learn task-specific prototypes. The prototype distillation loss
acts as a crucial regularization term, facilitating knowledge transfer from
early-stage prototypes to their deeper counterparts. This process enhances the
adaptation of vanilla prototypes into task-specific prototypes, resulting in
improved results.

##  4 Experiments

Table 1: Performance on S3DIS dataset using mean-IoU metric (%). Si represents the split $i$ is used for model testing. The best results are masked in bold. Our method achieves consistent better performance than state-of-the-art methods across different $N$-way $K$-shot settings. Method | 2-way | 3-way
---|---|---
1-shot | 5-shot | 1-shot | 5-shot
S0 |  S1 | mean |  S0 |  S1 | mean |  S0 |  S1 | mean |  S0 |  S1 | mean
FT [32] | 36.34 | 38.79 | 37.57 | 56.49 | 56.99 | 56.74 | 30.05 | 32.19 | 31.12 | 46.88 | 47.57 | 47.23
ProtoNet [4] | 48.39 | 49.98 | 49.19 | 57.34 | 63.22 | 60.28 | 40.81 | 45.07 | 42.94 | 49.05 | 53.42 | 51.24
MPTI [32] | 52.27 | 51.48 | 51.88 | 58.93 | 60.56 | 59.75 | 44.27 | 46.92 | 45.60 | 51.74 | 48.57 | 50.16
AttProtoNet [4] | 50.98 | 51.90 | 51.44 | 61.02 | 65.25 | 63.14 | 42.16 | 46.76 | 44.46 | 52.20 | 56.20 | 54.20
AttMPTI [32] | 53.77 | 55.94 | 54.86 | 61.67 | 67.02 | 64.35 | 45.18 | 49.27 | 47.23 | 54.92 | 56.79 | 55.86
BFG [16] | 55.60 | 55.98 | 55.79 | 63.71 | 66.62 | 65.17 | 46.18 | 48.36 | 47.27 | 55.05 | 57.80 | 56.43
QGPA [6] | 59.45 | 66.08 | 62.76 | 65.40 | 70.30 | 67.85 | 48.99 | 56.57 | 52.78 | 61.27 | 60.81 | 61.04
Ours | 66.08 | 74.30 | 70.19 | 71.10 | 77.03 | 74.07 | 50.67 | 59.53 | 55.10 | 64.52 | 63.34 | 63.93
Table 2: Performance on ScanNet dataset using mean-IoU metric (%). Si represents the split $i$ is used for model testing. The best results are masked in bold. Our method achieves consistent better performance than state-of-the-art methods across different $N$-way $K$-shot settings. Method | 2-way | 3-way
---|---|---
1-shot | 5-shot | 1-shot | 5-shot
S0 |  S1 | mean |  S0 |  S1 | mean |  S0 |  S1 | mean |  S0 |  S1 | mean
FT [32] | 31.55 | 28.94 | 30.25 | 42.71 | 37.24 | 39.98 | 23.99 | 19.10 | 21.55 | 34.93 | 28.10 | 31.52
ProtoNet [4] | 33.92 | 30.95 | 32.44 | 45.34 | 42.01 | 43.68 | 28.47 | 26.13 | 27.30 | 37.36 | 34.98 | 36.17
MPTI [32] | 39.27 | 36.14 | 37.71 | 46.90 | 43.59 | 45.25 | 29.96 | 27.26 | 28.61 | 38.14 | 34.36 | 36.25
AttProtoNet [4] | 37.99 | 34.67 | 36.33 | 52.18 | 46.89 | 49.54 | 32.08 | 28.96 | 30.52 | 44.49 | 39.45 | 41.97
AttMPTI [32] | 42.55 | 40.83 | 41.69 | 54.00 | 50.32 | 52.16 | 35.23 | 30.72 | 32.98 | 46.74 | 40.80 | 43.77
BFG [16] | 42.15 | 40.52 | 41.34 | 51.23 | 49.39 | 50.31 | 34.12 | 31.98 | 33.05 | 46.25 | 41.38 | 43.82
QGPA [6] | 57.08 | 55.94 | 56.51 | 64.55 | 59.64 | 62.10 | 55.27 | 55.60 | 55.44 | 59.02 | 53.16 | 56.09
Ours | 62.75 | 63.04 | 62.90 | 67.19 | 64.62 | 65.91 | 61.97 | 61.72 | 61.85 | 66.13 | 64.67 | 65.40
Table 3: Ablation study of key components in the proposed method on S3DIS dataset under 2-way 1-shot setting. ’PR’ denotes the prototype rectification, ’P2QA’ denotes the prototype-to-query attention, and ’PD’ denotes self-distillation regularization. PR | P2QA | PD | $S^{0}$ | $S^{1}$ | mean
---|---|---|---|---|---
|  |  | 50.45 | 51.12 | 50.79
✓ |  |  | 60.42 | 68.55 | 64.49
✓ | ✓ |  | 64.35 | 70.72 | 67.54
✓ | ✓ | ✓ | 66.08 | 74.30 | 70.19

###  4.1 Datasets and Evaluation Metrics

Datasets. The proposed method is evaluated on two point cloud segmentation
benchmarks, i.e., Stanford Large-Scale 3D Indoor Spaces (S3DIS) [1] and
ScanNet [3]. S3DIS is composed of 272 point cloud rooms in six indoor
environments. The annotation for the point clouds has 12 semantic classes and
one background class annotated as a cluster. ScanNet contains 1,513 point
cloud scans from 707 special indoor scenes, which provides 20 semantic classes
in addition to one background class annotated with annotated space. Compared
with S3DIS, ScanNet includes more diverse room types, such as living rooms and
bathrooms. Our data pre-processing follows Zhao et al. [32] which splits S3DIS
and ScanNet into 7,547 and 36,359 blocks, respectively. From each block, 2,048
samples are randomly sampled. For each dataset, all semantic classes are
divided into two disjoint class subsets. Then we perform two-fold cross-
validation with one subset for training and the other for testing. We adopt a
similar strategy as in Zhao et al. [32] to sample multiple $N$-way $K$-shot
episodes for model training and 100 episodes for model evaluation.

Evaluate Metrics. We adopt the widely-used metric in point cloud semantic
segmentation,i.e., mean Interaction over Union (mean-IoU), to evaluate our
model. The mean-IoU is obtained by averaging over the set of testing classes.
Considering different performance on each class split, we also report mean
value of results from split-0 ($\text{S}^{0}$) and split-1 ($\text{S}^{1}$)
for a comprehensive comparison.

###  4.2 Implementation Details

The implementation of our model is divided into three stages, i.e., backbone
pretraining, meta-training, and meta-testing. For the pretraining stage, we
adopt DGCNN [23] as the feature extractor and additional MLP layers as the
classifier. The pretraining is conducted on the base (seen) classes, we set
the batchsize to 32 and learning rate is 0.001. We pretrain the model with
Adam optimizer ($\beta_{1}=0.9$, $\beta_{2}=0.999$) for 150 epochs on both
S3DIS and ScanNet. During the meta-training stage, we initialize the feature
extract with pretrained weights and adopt Adam optimizer to update all
parameters. The initial learning rate is set as 0.001 decays by half every
5,000 iterations. The hyper-parameters $\gamma$ and $L$ are set as 0.1 and 1
in all our experiments, respectively. We meta-train the model for 40,000
iterations, in which each episode is constructed based on randomly selected
classes. During meta-testing, we randomly sample 100 episodes from unseen
classes to perform model evaluation. All our models are implemented with
PyTorch and trained on one NVIDIA 3090 GPU.

###  4.3 Ablation Study

In this section, we perform an ablation study on the S3DIS dataset under the
2-way 1-shot setting to validate the effectiveness of our proposed DPA
(Dynamic Prototype Adaptation) method. As a baseline model, we use ProtoNet,
which directly employs support prototypes for segmentation without any
adaptation.

Effects of Prototype Adaptors. In our study, we evaluate the effectiveness of
prototype adaptation and compare our method with the previous prototype
adaptor, QGPA [6], as well as the baseline model without any adaptation. As
shown in Figure 3 (a), both QGPA and our method DPA significantly outperform
the baseline model. For instance, DPA surpasses the baseline by about 20% in
mean IoU. This indicates the crucial role of prototype adaptation in few-shot
point cloud segmentation, especially when support and query point clouds
exhibit large object variations. Moreover, our DPA method consistently
outperforms QGPA on both the S3DIS and ScanNet datasets, e.g., 70.19% vs.
62.76 in mean. This demonstrates the superiority of our approach in achieving
effective prototype adaptation and further improving segmentation performance.
We undertake a qualitative analysis of the adaptation process of our DPA
framework during the test time in Figure 3 (b) using t-SNE [21]. The
visualization underscores the importance of the prototype adaptation process
in attaining improved performance in few-shot point cloud segmentation task.

Benefits of Prototype Rectification (PR). As shown in Table 3, the inclusion
of prototype rectification in DPA significantly improves segmentation
performance. For instance, it leads to a 13.70% increase (64.49% vs. 50.79%)
in mean IoU for the split $S^{0}$ and $S^{1}$, showcasing the effectiveness of
prototype rectification in enhancing segmentation. With prototype
rectification, DPA aligns initial prototypes from the support set to the query
feature distribution, addressing the object variation issue and producing more
accurate and task-specific prototypes for improved segmentation results.

Figure 3: Ablation study of modules and hyper-parameters under 2-way 1-shot
setting on S3DIS dataset. (a) Effects of different adaptors. (b) t-SNE
visualization of adaptation process on 3-way 5-shot task. (c) Effects of
coefficient of prototype distillation (PD) loss $\gamma$. (d) Effects of
number of decoder layers $L$.

Benefits of Prototype-to-Query Attention (P2QA). In Table 3, the prototype-to-
query attention further improves the model’s performance by 3.05% (67.54% vs.
64.49%). This improvement is attributed to the prototype-to-query attention
mechanism, which aggregates task-specific context information from the query
point cloud into the prototypes. As a result, the model generates more
accurate and task-specific prototypes, leading to improved segmentation
performance. The prototype-to-query attention enhances the model’s ability to
capture relevant information from the query point cloud, contributing to
better segmentation.

Benefits of Prototype Distillation (PD). In Table 3, we demonstrate the
effects of the prototype distillation term. As shown in the table, the
prototype distillation term between prototype from prototype rectification
(PR) and prototype-to-query attention (P2QA) results in a 2.65% performance
gain. This validates that self-distillation between prototypes effectively
enables knowledge transfer and enhances the adaptation of initial prototypes
to task-specific prototypes, leading to superior performance. The self-
distillation mechanism plays a critical role in refining the prototypes and
improving the segmentation results by facilitating information exchange
between prototypes from different adaptation stages.

Effects of Hyper-parameters. In Figure 3 (c) and (d), we illustrate the
effects of hyperparameters $\gamma$ (coefficient of prototype distillation
loss) and $L$ (number of decoder layers), respectively. As shown in Figure 3
(c), we experiment with $\gamma$ values in the range [0.1, 0.3, 0.5, 0.7].
Larger values of $\gamma$ tend to cause unstable training, resulting in
degraded performance. It is essential to find an appropriate balance for
$\gamma$ to ensure stable and effective training. For the number of decoder
layers $L$ (Figure 3 (d)), we observe that increasing $L$ initially leads to
improved performance. However, after a certain point, further increasing $L$
can cause overfitting and deteriorate the results. Therefore, selecting an
optimal value for $L$ is crucial to achieving the best trade-off between model
complexity and performance.

Figure 4: Quantitative results of our method in 2-way 1-shot point cloud semantic segmentation in comparison to Ground Truth and AttMPTI [32]. Left: S3DIS. Right: ScanNet. Our method achieves consistently better segmentation results on both S3DIS and ScanNet. Table 4: Analysis of computational cost and experimental results under 2-way 1-shot setting. Our method achieves better balance between computation cost and experiment results. Methods | Memory | #Params | FPS | S3DIS | ScanNet
---|---|---|---|---|---
attMPTI | 4.06G | 357.82K | 10.8 | 54.86 | 41.69
QGPA | 3.49G | 2.79M | 16.9 | 61.09 | 56.12
DPA | 4.21G | 4.85M | 15.3 | 70.19 | 62.90

###  4.4 Results and Analysis

Results on S3DIS. In Table 1, we present a comprehensive comparison of our
proposed method with state-of-the-art (SOTA) methods, showcasing our
quantitative results on the S3DIS dataset. Our proposed method consistently
and significantly outperforms the compared methods across all four settings,
including 2/3-way 1/5-shot scenarios. Notably, our method achieves a
performance boost of 7.43% (70.19% vs. 62.76%) and 6.22% (74.07% vs. 67.85%)
over the previous SOTA method QGPA [6] in the 2-way 1-shot and 2-way 5-shot
settings, respectively. Furthermore, our model demonstrates superior
performance compared to QGPA by 2.32% and 2.89% in the 3-way 1-shot and 3-way
5-shot settings, respectively. Additionally, our model exhibits substantial
improvement over the baseline model AttProtoNet [4], for instance, with a
notable margin of 15.1% for the split 0 in the 2-way 1-shot setting.

Results on ScanNet. Compared with S3DIS, ScanNet is a more challenging
dataset, which contains more complex scenes. In Table 2, we present the
quantitative results on the ScanNet dataset. Our proposed method consistently
and significantly outperforms the compared methods across all four settings.
Specifically, when compared to the previous state-of-the-art method QGPA, our
model achieves improvements of 6.39% and 3.81% in the 2-way 1-shot and 2-way
5-shot settings, respectively. These improvements are further extended to
6.41% and 9.31% in the 3-way 1-shot and 3-way 5-shot settings, respectively.
Moreover, our method demonstrates significant performance gains over AttMPTI
[32] and BFG [16], surpassing them by 21.21% and 21.56% in the 2-way setting,
respectively. Overall, our proposed method achieves superior performance on
the ScanNet dataset, showcasing its effectiveness and superiority compared to
existing state-of-the-art methods.

Qualitative Results. Figure 4 illustrates the qualitative results of our
proposed method in 2-way 1-shot point cloud semantic segmentation on the S3DIS
and ScanNet datasets. We compare our results with the ground truths and the
predictions from AttMPTI. Across both datasets, our method consistently
outperforms AttMPTI. For instance, in the first row of S3DIS examples, our
method effectively distinguishes between the floor and the sofa, while AttMPTI
mistakenly predicts some parts of the floor as the sofa. Similar results are
observed in the first row of ScanNet examples. Furthermore, our method excels
in segmenting small objects, such as chairs (the third row) in S3DIS and
toilets (the third row) in ScanNet. The superior performance of our method
stems from the task-specific prototypes, which are tailored to each specific
query point cloud by dynamic prototype adaptation with prototype distillation,
leading to improved segmentation performance.

Computational Complexity. In Table 4, we present the number of parameters and
computational complexity of our proposed method, along with a comparison to
the previous state-of-the-art (SOTA) methods, AttMPTI [32] and QGPA [6]. Our
method has more parameters, primarily due to the prototype decoder, compared
to AttMPTI and QGPA. However, despite the increase in parameters, our model
achieves significantly better performance. For example, on the S3DIS dataset,
our model outperforms AttMPTI and QGPA by 15.33% and 9.10%, respectively.
Additionally, our model achieves comparable Frames Per Second (FPS) with QGPA
(15.3 vs 16.9), which is significantly better than attMPTI. Our proposed model
strikes a good balance between model performance and computational complexity,
offering superior segmentation results with reasonable computational
efficiency.

Limitations. While our method consistently outperforms previous methods, we
have observed some challenges in distinguishing semantic-similar objects. For
example, in the second row of ScanNet in Figure 4, our method mistakenly
recognizes chairs as desks. To address this issue, future work could explore
incorporating multi-scale information to capture long-range context.
Additionally, introducing semantic information, such as class name
representations, as auxiliary constraints could also enhance the model’s
ability to handle semantic-similar objects and achieve more accurate
segmentation results.

##  5 Conclusion

We propose dynamic prototype adaptation (DPA) for few-shot point cloud
segmentation, addressing feature variations between support and query point
clouds through effective prototype adaptation. DPA consists of three key
components, prototype rectification, prototype-to-query attention, and
prototype distillation. Prototype rectification aligns initial prototypes from
the support with the query feature distribution, adapting them to the query
point cloud. Prototype-to-query attention aggregates context information from
the query feature into prototypes, generating task-specific prototypes for
each point cloud. To further enhance the adaptation process, we propose
prototype distillation as a regularization technique that refines the
prototypes by promoting information exchange between different adaptation
stages. Extensive evaluations on the S3DIS and ScanNet datasets demonstrate
that DPA achieves state-of-the-art performance, significantly surpassing
previous methods.

## References

  * Armeni et al. [2016] Iro Armeni, Ozan Sener, Amir R Zamir, Helen Jiang, Ioannis Brilakis, Martin Fischer, and Silvio Savarese.  3d semantic parsing of large-scale indoor spaces.  In _IEEE CVPR_ , pages 1534–1543, 2016.
  * Buciluǎ et al. [2006] Cristian Buciluǎ, Rich Caruana, and Alexandru Niculescu-Mizil.  Model compression.  In _Proceedings of the 12th ACM SIGKDD international conference on Knowledge discovery and data mining_ , pages 535–541, 2006.
  * Dai et al. [2017] Angela Dai, Angel X Chang, Manolis Savva, Maciej Halber, Thomas Funkhouser, and Matthias Nießner.  Scannet: Richly-annotated 3d reconstructions of indoor scenes.  In _IEEE CVPR_ , pages 5828–5839, 2017.
  * Dong and Xing [2018] Nanqing Dong and Eric P Xing.  Few-shot semantic segmentation with prototype learning.  In _BMVC_ , 2018.
  * Engelmann et al. [2019] Francis Engelmann, Theodora Kontogianni, and Bastian Leibe.  Dilated point convolutions: On the receptive field of point convolutions.  _arXiv preprint arXiv:1907.12046_ , 2, 2019.
  * He et al. [2023] Shuting He, Xudong Jiang, Wei Jiang, and Henghui Ding.  Prototype adaption and projection for few-and zero-shot 3d point cloud semantic segmentation.  _IEEE Transactions on Image Processing_ , 2023.
  * Hinton et al. [2015] Geoffrey Hinton, Oriol Vinyals, and Jeff Dean.  Distilling the knowledge in a neural network.  _arXiv preprint arXiv:1503.02531_ , 2015.
  * Hu et al. [2020] Qingyong Hu, Bo Yang, Linhai Xie, Stefano Rosa, Yulan Guo, Zhihua Wang, Niki Trigoni, and Andrew Markham.  Randla-net: Efficient semantic segmentation of large-scale point clouds.  In _IEEE CVPR_ , pages 11108–11117, 2020.
  * Jiang et al. [2021] Li Jiang, Shaoshuai Shi, Zhuotao Tian, Xin Lai, Shu Liu, Chi-Wing Fu, and Jiaya Jia.  Guided point contrastive learning for semi-supervised point cloud semantic segmentation.  In _Proceedings of the IEEE/CVF international conference on computer vision_ , pages 6423–6432, 2021.
  * Jiang et al. [2018] Mingyang Jiang, Yiran Wu, Tianqi Zhao, Zelin Zhao, and Cewu Lu.  Pointsift: A sift-like network module for 3d point cloud semantic segmentation.  _arXiv preprint arXiv:1807.00652_ , 2018.
  * Landrieu and Simonovsky [2018] Loic Landrieu and Martin Simonovsky.  Large-scale point cloud semantic segmentation with superpoint graphs.  In _IEEE CVPR_ , pages 4558–4567, 2018.
  * Lee et al. [2020] Hankook Lee, Sung Ju Hwang, and Jinwoo Shin.  Self-supervised label augmentation via input transformations.  In _International Conference on Machine Learning_ , pages 5714–5724. PMLR, 2020.
  * Li et al. [2023] Li Li, Hubert PH Shum, and Toby P Breckon.  Less is more: Reducing task and model complexity for 3d point cloud semantic segmentation.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 9361–9371, 2023.
  * Li et al. [2022] Mengtian Li, Yuan Xie, Yunhang Shen, Bo Ke, Ruizhi Qiao, Bo Ren, Shaohui Lin, and Lizhuang Ma.  Hybridcr: Weakly-supervised 3d point cloud semantic segmentation via hybrid contrastive regularization.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 14930–14939, 2022.
  * Luan et al. [2019] Yunteng Luan, Hanyu Zhao, Zhi Yang, and Yafei Dai.  Msd: Multi-self-distillation learning via multi-classifiers within deep neural networks.  _arXiv preprint arXiv:1911.09418_ , 2019.
  * Mao et al. [2022] Yongqiang Mao, Zonghao Guo, LU Xiaonan, Zhiqiang Yuan, and Haowen Guo.  Bidirectional feature globalization for few-shot semantic segmentation of 3d point cloud scenes.  In _2022 International Conference on 3D Vision (3DV)_ , pages 505–514. IEEE, 2022.
  * Qi et al. [2017a] Charles R Qi, Hao Su, Kaichun Mo, and Leonidas J Guibas.  Pointnet: Deep learning on point sets for 3d classification and segmentation.  In _IEEE CVPR_ , pages 652–660, 2017a.
  * Qi et al. [2017b] Charles R Qi, Li Yi, Hao Su, and Leonidas J Guibas.  Pointnet++: Deep hierarchical feature learning on point sets in a metric space.  _arXiv preprint arXiv:1706.02413_ , 2017b.
  * Shen et al. [2022] Yiqing Shen, Liwu Xu, Yuzhe Yang, Yaqian Li, and Yandong Guo.  Self-distillation from the last mini-batch for consistency regularization.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pages 11943–11952, 2022.
  * Thomas et al. [2019] Hugues Thomas, Charles R Qi, Jean-Emmanuel Deschaud, Beatriz Marcotegui, François Goulette, and Leonidas J Guibas.  Kpconv: Flexible and deformable convolution for point clouds.  In _IEEE ICCV_ , pages 6411–6420, 2019.
  * Van der Maaten and Hinton [2008] Laurens Van der Maaten and Geoffrey Hinton.  Visualizing data using t-sne.  _Journal of machine learning research_ , 9(11), 2008.
  * Wang et al. [2019a] Lei Wang, Yuchun Huang, Yaolin Hou, Shenman Zhang, and Jie Shan.  Graph attention convolution for point cloud semantic segmentation.  In _IEEE CVPR_ , pages 10296–10305, 2019a.
  * Wang et al. [2019b] Yue Wang, Yongbin Sun, Ziwei Liu, Sanjay E Sarma, Michael M Bronstein, and Justin M Solomon.  Dynamic graph cnn for learning on point clouds.  _TOG_ , 38(5):1–12, 2019b.
  * Wu et al. [2019] Wenxuan Wu, Zhongang Qi, and Li Fuxin.  Pointconv: Deep convolutional networks on 3d point clouds.  In _IEEE CVPR_ , pages 9621–9630, 2019.
  * Xiong et al. [2019] Yuwen Xiong, Mengye Ren, Renjie Liao, Kelvin Wong, and Raquel Urtasun.  Deformable filter convolution for point cloud reasoning.  _arXiv preprint arXiv:1907.13079_ , 2019.
  * Xu and Liu [2019] Ting-Bing Xu and Cheng-Lin Liu.  Data-distortion guided self-distillation for deep neural networks.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , pages 5565–5572, 2019.
  * Ye et al. [2018] Xiaoqing Ye, Jiamao Li, Hexiao Huang, Liang Du, and Xiaolin Zhang.  3d recurrent neural networks with context fusion for point cloud semantic segmentation.  In _ECCV_ , pages 403–417, 2018.
  * Zhang et al. [2023] Canyu Zhang, Zhenyao Wu, Xinyi Wu, Ziyu Zhao, and Song Wang.  Few-shot 3d point cloud semantic segmentation via stratified class-specific attention based transformer network.  _arXiv preprint arXiv:2303.15654_ , 2023.
  * Zhang et al. [2019a] Linfeng Zhang, Jiebo Song, Anni Gao, Jingwei Chen, Chenglong Bao, and Kaisheng Ma.  Be your own teacher: Improve the performance of convolutional neural networks via self distillation.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 3713–3722, 2019a.
  * Zhang et al. [2021] Yachao Zhang, Yanyun Qu, Yuan Xie, Zonghao Li, Shanshan Zheng, and Cuihua Li.  Perturbed self-distillation: Weakly supervised large-scale point cloud semantic segmentation.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pages 15520–15528, 2021.
  * Zhang et al. [2019b] Zhiyuan Zhang, Binh-Son Hua, and Sai-Kit Yeung.  Shellnet: Efficient point cloud convolutional neural networks using concentric shells statistics.  In _Proceedings of the IEEE/CVF international conference on computer vision_ , pages 1607–1616, 2019b.
  * Zhao et al. [2021] Na Zhao, Tat-Seng Chua, and Gim Hee Lee.  Few-shot 3d point cloud semantic segmentation.  In _IEEE CVPR_ , pages 8873–8882, 2021.
  * Zhao et al. [2022] Ziyu Zhao, Zhenyao Wu, Xinyi Wu, Canyu Zhang, and Song Wang.  Crossmodal few-shot 3d point cloud semantic segmentation.  In _Proceedings of the 30th ACM International Conference on Multimedia_ , pages 4760–4768, 2022.


# A preliminary study about gravitational wave radiation and cosmic heat death

Jianming Zhang 1, Qiyue Qian 1†, Yiqing Guo 1, Xin Wang 1‡,Xiao-Dong Li 1∗
1 School of Physics and Astronomy, Sun Yat-Sen University, Guangzhou 510297,
P. R. China
†Corresponding author<EMAIL_ADDRESS>
‡Corresponding author<EMAIL_ADDRESS>
∗Corresponding author<EMAIL_ADDRESS>
###### Abstract

We study the role of gravitational waves (GW) in the heat death of the
universe. Due to the GW emission, in a very long period, dynamical systems in
the universe suffer from persistent mechanical energy dissipation, evolving to
a state of universal rest and death. With N-body simulations, we adopt a
simple yet representative scheme to calculate the energy loss due to the GW
emission. For current dark matter systems with mass $\sim
10^{12}-10^{15}M_{\odot}$, we estimate their GW emission timescale as $\sim
10^{19}-10^{25}$ years. This timescale is significantly larger than any baryon
processes in the universe, but still $\sim 10^{80}$ times shorter than that of
the Hawking radiation. We stress that our analysis could be invalid due to
many unknowns such as the dynamical chaos, the quadrupole momentum of halos,
the angular momentum loss, the dynamic friction, the central black hole
accretion, the dark matter decays or annihilations, the property of dark
energy and the future evolution of the universe.

###### keywords:

gravitational waves, cosmology, astrophysics

††pagerange: A preliminary study about gravitational wave radiation and cosmic
heat death–LABEL:lastpage††pubyear: 2020

##  1 Introduction

The heat death fate of the universe, also known as the "Big Chil" or "Big
Freeze", is a conjecture suggesting that the universe would end up with a
state of no thermodynamic free energy. In such situation the universe would be
unable to sustain any process that increases entropy.

The idea of heat death was first studied by Kelvin, who extrapolated the
second law of thermodynamics of mechanical energy dissipation to a cosmic
scale in the 1850s, and found it inevitably led to "a state of universal rest
and death". Along with this conjecture, Helmholtz (Thomson, 1857) envisioned a
stable, thermodynamic equilibrium state in the end, stating that "the universe
from that time forward would be condemned to a state of eternal rest." Similar
viewpoint was held by Clausius, who believed that the universe would evolve to
"a state of unchanging death" once its entropy reaches the maximum.

Discussions on the heat death hypothesis never stops since it was proposed.
Jeans (1930); Eddington (1931) brought the notion of the heat death into
relativistic cosmology. Barrow & Tipler (1978) demonstrated that the Hawking
black hole evaporation process would cause a vortical instability to develop
in spatially homogeneous spacetimes, and furthermore provided a novel picture
of the universal heat death. Also, there were opposing viewpoints to the idea
of heat death, such as "rise and fall theory" by L.E.Boltzman and the
conjecture of "Maxwell’s demon" (Bennett, 1982), etc.

With the development of modern cosmology, it becomes commonly accepted that
the ultimate fate of the universe crucially depends on its energy components.
Heat death is likely to happen in a forever expanding universe, which is
expected to occur if the topology of the universe is open or flat (in a
matter-dominated universe), or if the dark energy component keeps dominating.
The latter possibility is currently supported by multiple cosmological
observations (see Ade et al. (2016) and the references therein).

Dyson (1979) provided a comprehensive summary of physical processes related to
the heat death, which included the stellar evolution, the detachment of
planets from stars and stars from galaxies, the decay of object orbits by
gravitational radiation, the evaporation of black holes by the Hawking
process, the liquification of all matter at zero temperature, the decay of
matter to iron, as well as the collapse of iron stars to neutron stars and the
collapse of ordinary matter to black holes. While most of these processes are
of great interest and worth more detailed investigation, in this paper, we
focus on the detailed investigation of the gravitational waves. As is
mentioned in Barrow & Tipler (1978), “as time passes, clusters, galaxies and
other stellar systems will become increasingly bound by gravitational forces
since they will radiate away their binding energy in the form of gravitational
waves”.

Gravitational waves (GWs) generated by accelerated mass are spacetime
curvature disturbances which propagate as waves outward from the source at the
speed of light (Eistein, 1916, 1918). On September 14th, 2015, the Laser
Interferometer Gravitational Wave Observatory (LIGO) made the first direct
detection of a GW event (Abbott et al., 2016), opening a new window for us to
test the predictions of general relativity and probe into the structures of
the universe. GWs take place in all dynamical systems, causing persistent
mechanical energy dissipation in the long-term evolution of our universe, and
therefore are expected to become an essential process to the heat death.

GWs carry energy away from their sources and, in the case of orbiting objects,
lead to an in-spiral or the shrinking of the orbit’s radius. Before the
detection made by LIGO, pulsar timing observations over decade had shown a
gradual decay of the pulsar binary orbital period that matched the loss of
energy and angular momentum in GW radiation predicted by general relativity;
this offered the first indirect evidence of the existence of GWs (Hulse &
Taylor, 1975; Taylor & Weisberg, 1982). This process ends up with the merger
of the objects, which would form a heavier compact object (e.g. a black hole)
and produce strong GW signals detectable via experiments like LIGO.

The in-spiral caused by GW emission is a universal process of mechanical
energy loss, regardless of the nature of the objects. Therefore, it will
become the key mechanism for dark matter systems to reach heat death. Unlike
the baryonic systems such as gas, stars, galaxies and galaxy clusters, dark
matter can not lose mechanical energy via thermal radiation and (non-
dynamical) friction. Therefore, the basic picture regarding our thermal fate
would be that dark matter systems gradually lose mechanical energy, become
more compact, and finally collapse into black holes (BHs). Then BHs evaporate,
and the universe eventually enters the "state of eternal rest".

In this work we conduct a preliminary yet representative investigation on the
role of GW radiation in the cosmic death. In 2 we briefly introduce how GW
emission affects the dynamical systems. In 3, we estimate the GW emission
timescale of dark matter systems in the current universe with the aid of
simulations. In 4 we discuss the caveats of our analysis as well as some
closely related issues. We summarize and conclude in 5.

##  2 GW emission in dynamical systems

By carrying energy away, the GW radiation causes a persistent loss of mechanic
energy in a system. As a gravitational effect, this process only involves the
mass and dynamical properties of the system. In the most simplified case of a
two-body system with orbital velocity $v$ and mass of objects $m_{1}$ and
$m_{2}$, the rate at which energy is carried away by GWs is given as follows,

${d\over dt}E_{GW}={32\over 5}{G\over c^{5}}u^{2}r^{4}\omega^{6}~{},$ (1)

where we define $u=m_{1}m_{2}/M$, $M=m_{1}+m_{2}$, and $\omega$ being the
angular velocity. Here the constants $G=6.67\times 10^{-11}\rm m^{3}/s^{2}\
kg$, $c=2.998\times 10^{8}\rm m/s$. Notice that the mechanic orbital energy
takes the form of

$E_{orb}={-GMu\over 2r}~{}.$ (2)

Considering the energy loss is only caused by GW radiation, we have

${d\over dt}E_{orb}={GMu\over 2r^{2}}\dot{r}={-d\over dt}E_{Gw}~{}.$ (3)

Now we are ready to estimate the timescale that the system takes to form a
compact object, which we denote as $\Delta_{t}\equiv t_{s}-t_{0}$, where
$t_{0}$ is the current epoch and $t_{s}$ is the time when the system
collapses. From 1, 2,3, we find

$\dot{r}=-{64\over 5}{G^{3}\over c^{5}r^{3}}(m_{1}+m_{2})m_{1}m_{2}~{},$ (4)

which yields to

$\Delta t={5(r_{0}^{4}-r_{s}^{4})c^{5}\over 256G^{3}Mm_{1}m_{2}}~{},$ (5)

where $r_{0}$, $r_{s}$ are the initial and final radius of the system,
respectively. If we take $r_{s}$ as the Schwarzschild BH radius with mass of
the system, i.e. $r_{s}={2MG\over c^{2}}$ , then the timescale of evolution is
completely determined. Notice that the time for such BH to evaporate is

$\Delta t_{s}={5120\pi G^{2}M^{3}\over\hbar c^{4}}~{},$ (6)

where $\hbar$ is the reduced Planck constant.

##  3 Application to simulation data

From the above estimation, we see that the timescale depends on the mass and
orbital radius change. The cosmic structure formation results in a series of
collapsed dark matter halos whose mass is distributed in a wide range. These
objects would become stable after entering the state of virial equilibrium,
and are expected to stay stable for a long period if no major merger happens.

1 illustrates the distribution of dark matter halos at redshift $z=0$ with
$M>10^{12}M_{\odot}$ in a small volume $(256h^{-1}{\rm Mpc})^{3}$, created by
dark matter simulation (Springel, 2005; Tassev et al., 2013; Klypin et al.,
2016) with cosmological parameters of $\Omega_{m}=0.307115$,
$\Omega_{b}=0.048206$, $\sigma_{8}=0.8288$, $n_{s}=0.9611$, and $H_{0}=67.77\
{\rm km}\ s^{-1}{\rm Mpc}^{-1}$.

In total we select a sample of 56,549 halos, using the ROCKSTAR halo finder
(Behroozi et al., 2013) that allows for robust tracking of substructure based
on adaptive hierarchical refinement of friends-of-friends groups in six phase-
space dimensions and one time dimension. In general, the halos we obtain well
obey the virial theorem, which states that a stabilized dynamical system
should obey $\frac{GM}{R}\approx\sigma^{2}_{v}$, with $\sigma_{v}^{2}$ being
the velocity dispersion. The mass-velocity distribution measured from the
simulation is presented in 2.

Figure 1:  Mass function of the halo samples used in this analysis. The
samples include a set of 56,549 dark matter halos at redshift $z=0$ in a
volume of $(256\ h^{-1}{\rm Mpc})^{3}$, created using N-body method and the
ROCKSTAR halo finder.  Figure 2:  Mass-velocity distribution of the halo
samples used in our analysis.

To simplify the situation and make everything calculable, in what follows we
will equate the whole dark matter halo to a binary system of two objects with
equal mass orbiting around the center, and then the virial theorem leads to 7.
If we assume the distance between the two objects $R={1\over 2}r_{0}$, where
$r_{0}$ being the initial radius of the system, then we have

${1\over 2}m(3\sigma^{2}_{v})\sim{1\over 2}{GMm\over({1\over 2}r_{0})}~{},$
(7)

where $3\sigma^{2}_{v}$ is the $3-D$ velocity squared in the radial direction
and $M,\ m$ both equal to half of the halo mass. Then we have
${\sigma^{2}_{v}={2GM\over 3r_{0}}}~{},$ which is a good approximation for
what we have measured from the N-body simulation.

As consequences of GW emission, we believe that as time elapses, the radius of
the system $r$ would shrink, the orbital velocity $v$ would increase, and the
magnitudes of potential energy $V$ and kinetic energy $T$ would both increase.
The gradual changes of the physical quantities are rather small at the
beginning, and dramatically increase at the end of the evolution.

3 shows the evolution of three typical systems selected from the sample,
having mass around $10^{12}$, $10^{13}$ and $10^{14}$ $M_{\odot}$,
respectively. Their circular velocity and kinetic energy are presented in 1.

Table 1: Dynamical properties of the three selected halos at the present time  $i$ |  Mass $M_{\odot}$ |  Radius of the system $(\rm km)$ |  Circular Velocity $(\rm km/s)$ |  Kinetic Energy $(MJ)$
---|---|---|---|---
1 | $1.01\times 10^{12}$ | $8.15\times 10^{18}$ | 86.4 | $7.53\times 10^{45}$
2 | $1.00\times 10^{13}$ | $1.75\times 10^{19}$ | 197.8 | $3.90\times 10^{47}$
3 | $1.00\times 10^{14}$ | $3.77\times 10^{19}$ | 584.8 | $3.41\times 10^{49}$

We find that the results are consistent with what we expected. As demonstrated
in 3, with time elapsing, the radius of the system $r$ shrinks, velocity $v$
and kinetic energy $T$ increase, and the changes are rapid at the end of the
evolution.

From 5 and 6, we obtain the timescale distribution of halo-size binaries to
evolve into BHs via GW radiation and BHs evaporation through Hawking
radiation.

The left panel of the 4 illustrates halo distribution with the time period of
GW radiation, which is estimated as $\sim 10^{19}-10^{25}$ years. The right
panel shows that the time for BHs to evaporate is much longer, which
distributes in $\sim 10^{100}-10^{112}$ years.

Figure 3:  The evolution of three dynamical systems selected from the halo
sample (see 1 for detailed data). With time elapsing, their radius $r$
shrinks, while the velocity $v$ and kinetic energy $T$ increase. The evolution
becomes rapid at the end of the evolution. Figure 4: The left panel shows the
time needed for the halos to evolve to BHs through GW radiation. Time for
these BHs to evaporate via Hawking radiation is demonstrated in the right
panel.

##  4 Caveats and Discussions

In the above section, the typical timescale that dark matter systems take to
emit GWs before ceasing evolution is estimated using a two-body model. While
the model is oversimplified, our result is still representative and makes
physical sense due to two reasons. First, no matter how complicated a many-
body system is, as long as the motions (velocities, accelerations) of the
objects within are comparable to those in a two-body system with similar mass,
size and kinetic energy, the evolution of these two systems should not differ
from each other significantly. Secondly, the two-body system is the only case
of a gravitational system that is everlastingly stable. It is possible that
most complicated systems would finally evolve to a stage similar to a two-body
system after a long enough period.

However, besides the two-body simplification we made in the calculation, there
are more caveats that could potentially invalidate our results. We will
summarize and discuss them as follows.

###  4.1 Dynamical Evolution, Chaos, Stability

Using the N-body methods, one can start with a certain distribution of mass
points, and trace their evolution by integrating the equations of motion
(Aarseth & Binney, 1978). Yet we are not clear whether this kind of stability
can sustain within a timescale of $10^{19}-10^{25}$ years due to many well-
known difficulties, e.g., singular gravitational potential when two particles
become too close to each other, and chaotic behavior of the N-body problem
when $N>2$. Although some studies (Voglis et al., 2002; Kalapotharakos &
Voglis, 2005; Kalapotharakos et al., 2008; Muzzio, 2012) have shown that it is
possible to obtain models of elliptical galaxies which are highly stable over
time intervals of the order of a Hubble time (even with a black hole growing
in the center; see Merritt & Quinlan (1998); Poon & Merritt (2002, 2004)), so
far it is impossible to trace the evolution of the system within the very long
period of the GW radiation.

###  4.2 Dark matter life-time

Possibility exists that dark matter consists of particles that actually decay
on a very long time scale. Pandey et al. (2019); Mukherjee et al. (2019)
showed that dark matter with decay lifetime $\sim 10^{28}-10^{29}$ seconds can
help to explain some detected excesses (Chen & Takahashi, 2009; Yin et al.,
2009; Ishiwata et al., 2009; Ibarra & Tran, 2009; Chen et al., 2009; Nardi et
al., 2009; Arvanitaki et al., 2009), while some other work (Cohen et al.,
2017; Cadena et al., 2020) suggested that dark matter should be stable within
a long timescale of $\sim 10^{24}-10^{28}$ seconds. These timescales are
already close to the GW emission timescale ($\sim 10^{26}-10^{32}$ in unit of
second).

###  4.3 Dark matter accretion by central black hole

There have been scenarios envisaged in which dark matter could give a
significant contribution to the mass accretion of the central black hole of
the galaxies. Balberg & Shapiro (2002) showed that black holes with mass $\geq
10^{6}M_{\odot}$ could be formed directly as a consequence of relativistic
core collapse of halos if halos embedding galaxies are constituted of self-
interacting dark matter (Dave et al., 2001), while Zelnikov & Vasiliev (2005)
and Munyaneza & Biermann (2005) showed that a significant accretion may happen
in the case when dark matter particles are scattered by stars in molecular
clouds near the central black hole. Although in general the accretion of
collisionless dark matter particles into black holes are thought to be less
efficient than that expected from the dissipative baryonic fluid (Peirani & de
Freitas Pacheco, 2008), current studies can not rule out the possibility that
the dynamical system would be significantly affected or even terminated due to
the black hole accretion in the long timescale of GW emission.

###  4.4 Quadrupole moment in halos

To be precise, the rate of GW emission depends on the changing rate of the
quadrupole moment (Eistein, 1916, 1918) of the system, i.e.

${d\over dt}E_{GW}={32\over 5}{G\over c^{5}}I^{2}e^{2}\omega^{6}~{},$ (8)

where $I$ is the moment of inertia of the halo, $e$ is the eccentricity of the
halo, and $\omega$ is the angular velocity. Using the above formula, we re-
calculate the GW emission efficiency of the halo sample using the positions
and velocities of their member particles, and find the result is actually
$34\pm 15$ times larger than the result obtained using the simple two-body
model adopted in this analysis. In any case, these two results do not differ
for many orders of magnitude. Therefore our main conclusion remains valid.

The calculation of the quadrupole moment is based on the current mass
distribution within the halo samples, and it is very likely that this mass
distribution will change dramatically within the long period of GW emission.
Thus, in this paper we still adopt the simple two-body model to estimate the
power of the GW emission.

###  4.5 Angular momentum loss, Dynamic friction and the Final parsec problem

Although halos may lose their angular momentum through interactions with the
other objects, we believe that this will not have significant effect on our
main conclusion. As the universe evolves, the shrinking of the horizon size
will isolate halos, and prohibit their interactions with each other. The above
process is expected to happen in a timescale much shorter than the GW emission
timescale.

In particular, the future event horizon, i.e., the region that light emitted
from an object can reach in the infinite future, takes the form of

$r_{\rm FEH}\equiv\int_{t=t_{\rm emit}}^{+\infty}\frac{cdt}{a}.$ (9)

If the $\Lambda$CDM is adopted as the theoretical model, the future event
horizon would shrink to $1$ Mpc within $\sim 10^{12}$ years, which is much
shorter compared with the GW emission timescale.

Another process that may contribute to the collapse is the dynamic friction.
In general, its timescale depends inversely on the mass of the system
(Chandrasekhar, 1942). Once a cluster is collapsed, violent relaxation is
ineffective, and further relaxation occurs through two-body interactions.
Schechter (1976); Schechter & P.J.E.Peebles (1976) calculated the drag force
and the dynamic friction timescale of a system composed by homogeneous,
isotropic, Maxwellian distributed particles. Following their calculation, we
find that the dynamic friction of typical galaxy clusters is about $2\times
10^{8}$ year, which is also far less than the GW emission timescale.

Finally, one may wonder whether our results is related with the final parsec
problem. For that, our opinions are as follows. On one hand, if there exists
any mechanism that may fasten the merger of supermassive binary black holes
(SBBHs) when they are extremely close to each other, such mechanism may also
fasten the final stage of the evolution of a halo to a BH. In this case, the
thermal death may occur a bit earlier. On the other hand, considering that the
timescale of the GW emission is $10^{19}-10^{25}$ years, it is extremely
difficult to predict the status of the dynamic system after such a long time
evolution. One possible situation is that, since two-body system is the only
stable gravitational system, by the time all halos would exist in a form of
two-body system. Without a third object interacting with them, there maybe no
mechanism that can accelerate the evolution of the system.

###  4.6 Other Fates of the universe

In the case that the nature of dark energy is phantom like ($w<-1$,
$\dot{\rho}>0$), its density will reach infinity in a finite time, disrupt any
bounded system, and cause a "big rip" fate of the universe (Caldwell et al.,
2003; Li et al., 2012), making our analysis completely meaningless. Similarly,
our analysis is also invalid in the scenario of bouncing cosmology (see Cai
(2014) and the references therein).

##  5 Conclusion

In this paper, we study the role of GW in the heat death. GWs happen in almost
all dynamical systems in the universe, causing persistent mechanical energy
dissipation in the long-term evolution of the universe and driving the
universe to "a state of universal rest and death". With the N-body
simulations, we adopt a simple yet representative scheme to compute the GW
emission process, and estimate its timescale as $\sim 10^{19}-10^{25}$ years,
which depends on the mass of the system. This timescale is significantly
larger than any baryon process in the universe, but still $\sim 10^{80}$ times
shorter than the timescale of Hawking radiation.

By taking the GW emission into consideration, our work extends the scope of
heat death. This means that, similar to the baryon systems, dark matter has to
lose mechanical energy via radiation and therefore cannot persist forever.

In this work, we study typical halos with mass $\sim
10^{12}-10^{15}M_{\odot}$. But the results may be different if using a
simulation with higher resolution and bigger size, or if the nature of dark
matter is not as assumed in the standard cold dark matter scheme.

Base on our current knowledge and technology, it is unlikely to obtain a
comprehensive understand of the physics process of dynamical chaos, dark
matter decays or annihilations, central black hole accretion, and the nature
of dark energy. Neither do we know the roles they are going to play in the GW
emission process. Thus, our analysis would be invalid under certain
circumstances.

We adopt a rather simplified model to estimate the timescale of the GW
emission. A further study may require running a N-body simulation with the GW
emission considered, which is of great challenge and, not that necessary,
given that there are so many unknown factors that may affect the process.

Several issues about the gravitational wave and thermal death are not
discussed in this short work. In a generic anisotropic universe without
positive cosmological constant, it is possible to generate an infinite amount
of entropy by taking advantage of cosmological shear and curvature anisotropy,
allowing civilizations to exist forever (Ellis & Barrow, 2002; Barrow &
Hervik, 2003). This provides a novel method to avoid thermal death by
utilizing the gravitational waves. Neither do we discuss the "gravitational
field entropy" (Barrow et al., 1988; M.Patel & Lineweaver, 2017), which is
associated with the increase of irregularity in the universe and provides a
novel picture of the universal heat death.

In all, heat death is an interesting conjecture about the fate of the
universe, and we expect more studies on this issue with the progress of our
understanding about the physics processes in the universe.

##  6 *Data Availability

The data used to support the findings of this study are available from the
corresponding authors upon request.

## Acknowledgements

We thank Prof. Rongxin Miao for helpful discussions. We acknowledge the use of
Kunlun cluster located in School of Physics and Astronomy, Sun Yat-Sen
University. This work is supported by National SKA Program of China No.
2020SKA0110401. XDL acknowledges support from the NSFC grant (No. 11803094),
the Science and Technology Program of Guangzhou, China (No. 202002030360).

## References

  * Aarseth & Binney (1978) Aarseth, S. J., & Binney, J. 1978, Mon. Not. Roy. Astron. Soc., 185, 227
  * Abbott et al. (2016) Abbott, B. P., Abbott, R., Abbott, T. D., et al. 2016, Phys. Rev. Lett., 116, 061102
  * Ade et al. (2016) Ade, P. A., Aghanim, N., Arnaud, M., et al. 2016, Astronomy & Astrophysics, 594, A13
  * Arvanitaki et al. (2009) Arvanitaki, A., Dimopoulos, S., Dubovsky, S., et al. 2009, Phys. Rev. D, 79, 105022
  * Balberg & Shapiro (2002) Balberg, S., & Shapiro, S. L. 2002, Phys. Rev. Lett., 88, 101301
  * Barrow & Hervik (2003) Barrow, J., & Hervik, S. 2003, Physics Letters B, 566, 1
  * Barrow & Tipler (1978) Barrow, J. D., & Tipler, F. J. 1978, Nature, 276, 453
  * Barrow et al. (1988) Barrow, J. D., Tipler, F. J., & Anderson, J. L. 1988, The Anthropic Cosmological Principle (Oxford University press), 613–658
  * Behroozi et al. (2013) Behroozi, P. S., Wechsler, R. H., & Wu, H.-Y. 2013, ApJ, 762, 109
  * Bennett (1982) Bennett, C. H. 1982, International Journal of Theoretical physics, 21, 905
  * Cadena et al. (2020) Cadena, S. H., Franco, J. S., Alfaro Molina, R., et al. 2020, PoS, ICRC2019, 520
  * Cai (2014) Cai, Y.-F. 2014, Sci. China Phys. Mech. Astron., 57, 1414
  * Caldwell et al. (2003) Caldwell, R. R., Kamionkowski, M., & Weinberg, N. N. 2003, Phys. Rev. Lett., 91, 071301
  * Chandrasekhar (1942) Chandrasekhar, S. 1942, university of chicago, 231
  * Chen et al. (2009) Chen, C.-R., Nojiri, M. M., Takahashi, F., & Yanagida, T. 2009, Prog. Theor. Phys., 122, 553
  * Chen & Takahashi (2009) Chen, C.-R., & Takahashi, F. 2009, JCAP, 02, 004
  * Cohen et al. (2017) Cohen, T., Murase, K., Rodd, N. L., Safdi, B. R., & Soreq, Y. 2017, Phys. Rev. Lett., 119, 021102
  * Dave et al. (2001) Dave, R., Spergel, D. N., Steinhardt, P. J., & Wandelt, B. D. 2001, The Astrophysical Journal, 547, 574
  * Dyson (1979) Dyson, F. J. 1979, Rev. Mod. Phys., 51, 447
  * Eddington (1931) Eddington, A. S. 1931, Nature, 127, 447
  * Eistein (1916) Eistein, A. 1916, Sitzungsber. Preuss. Akad. Wiss., 1, 688
  * Eistein (1918) —. 1918, Sitzungsber. Preuss. Akad. Wiss., 1, 154
  * Ellis & Barrow (2002) Ellis, E. G. F. R., & Barrow, J. D. 2002, The Far-Future Universe: The Far, Far Future (Pontifical Academy of Sciences, the Vatican), 23
  * Hulse & Taylor (1975) Hulse, R. A., & Taylor, J. H. 1975, The Astrophysical Journal, 195, L51
  * Ibarra & Tran (2009) Ibarra, A., & Tran, D. 2009, JCAP, 02, 021
  * Ishiwata et al. (2009) Ishiwata, K., Matsumoto, S., & Moroi, T. 2009, Phys. Lett. B, 675, 446
  * Jeans (1930) Jeans, J. 1930, Cambridge University Press
  * Kalapotharakos et al. (2008) Kalapotharakos, C., Efthymiopoulos, C., & Voglis, N. 2008, Mon. Not. Roy. Astron. Soc., 383, 971
  * Kalapotharakos & Voglis (2005) Kalapotharakos, C., & Voglis, N. 2005, Celest. Mech. & Dynam. Astron., 92, 157
  * Klypin et al. (2016) Klypin, A., Yepes, G., Gottlöber, S., Prada, F., & Heß, S. 2016, Mon. Not. Roy. Astron. Soc., 457, 4340
  * Li et al. (2012) Li, X.-D., Wang, S., Huang, Q.-G., Zhang, X., & Li, M. 2012, Sci. China Phys. Mech. Astron., 55, 1330
  * Merritt & Quinlan (1998) Merritt, D., & Quinlan, G. D. 1998, The Astrophysical Journal, 498, 625
  * M.Patel & Lineweaver (2017) M.Patel, V., & Lineweaver, C. H. 2017, Physics, Computer Science. Entropy, 19, 411
  * Mukherjee et al. (2019) Mukherjee, T., Pandey, M., Majumdar, D., & Halder, A. 2019, arXiv:1911.10148
  * Munyaneza & Biermann (2005) Munyaneza, F., & Biermann, P. L. 2005, Astron. Astrophys., 436, 805
  * Muzzio (2012) Muzzio, J. 2012, arXiv:1204.0709
  * Nardi et al. (2009) Nardi, E., Sannino, F., & Strumia, A. 2009, JCAP, 01, 043
  * Pandey et al. (2019) Pandey, M., Majumdar, D., Halder, A., & Banerjee, S. 2019, Phys. Lett. B, 797, 134910
  * Peirani & de Freitas Pacheco (2008) Peirani, S., & de Freitas Pacheco, J. 2008, Phys. Rev. D, 77, 064023
  * Poon & Merritt (2002) Poon, M., & Merritt, D. 2002, The Astrophysical Journal, 568, 89
  * Poon & Merritt (2004) —. 2004, The Astrophysical Journal, 606, 774
  * Schechter (1976) Schechter, P. L. 1976, Astrophys.J, 209, 297
  * Schechter & P.J.E.Peebles (1976) Schechter, P. L., & P.J.E.Peebles. 1976, Astrophys.J, 203, 670
  * Springel (2005) Springel, V. 2005, Mon. Not. Roy. Astron. Soc., 364, 1105
  * Tassev et al. (2013) Tassev, S., Zaldarriaga, M., & Eisenstein, D. J. 2013, Journal of Cosmology and Astroparticle Physics, 6, 036
  * Taylor & Weisberg (1982) Taylor, J. H., & Weisberg, J. M. 1982, Annu. Rev. Astron. Astrophys., 253, 908
  * Thomson (1857) Thomson, W. 1857, Proceedings of the Royal Society of Edinburgh, 3, 139
  * Voglis et al. (2002) Voglis, N., Kalapotharakos, C., & Stavropoulos, I. 2002, Mon. Not. Roy. Astron. Soc., 337, 619
  * Yin et al. (2009) Yin, P.-f., Yuan, Q., Liu, J., et al. 2009, Phys. Rev. D, 79, 023512
  * Zelnikov & Vasiliev (2005) Zelnikov, M., & Vasiliev, E. A. 2005, Int. J. Mod. Phys. A, 20, 4217