# Diffusion2: Dynamic 3D Content Generation via Score Composition of
Orthogonal Diffusion Models

Zeyu Yang Zijie Pan11footnotemark: 1 Chun Gu Li Zhang
Fudan University
https://github.com/fudan-zvg/diffusion-square Equally contributedLi Zhang
<EMAIL_ADDRESS>is the corresponding author with School of Data
Science, Fudan University.

###### Abstract

Recent advancements in 3D generation are predominantly propelled by
improvements in 3D-aware image diffusion models which are pretrained on
Internet-scale image data and fine-tuned on massive 3D data, offering the
capability of producing highly consistent multi-view images. However, due to
the scarcity of synchronized multi-view video data, it is impractical to adapt
this paradigm to 4D generation directly. Despite that, the available video and
3D data are adequate for training video and multi-view diffusion models that
can provide satisfactory dynamic and geometric priors respectively. In this
paper, we present Diffusion2, a novel framework for dynamic 3D content
creation that leverages the knowledge about geometric consistency and temporal
smoothness from these models to directly sample dense multi-view and multi-
frame images which can be employed to optimize continuous 4D representation.
Specifically, we design a simple yet effective denoising strategy via score
composition of video and multi-view diffusion models based on the probability
structure of the images to be generated. Owing to the high parallelism of the
image generation and the efficiency of the modern 4D reconstruction pipeline,
our framework can generate 4D content within few minutes. Furthermore, our
method circumvents the reliance on 4D data, thereby having the potential to
benefit from the scalability of the foundation video and multi-view diffusion
models. Extensive experiments demonstrate the efficacy of our proposed
framework and its capability to flexibly adapt to various types of prompts.

##  1 Introduction

Figure 1:  Diffusion2 is designed to generate dynamic content by creating a
dense multi-frame multi-view image matrix in a highly parallel denoising
diffusion process with the combination of the foundation video diffusion model
and multi-view diffusion model. The generated image can be used to construct a
full 4D representation by being fed into an off-the-shelf 4D reconstruction
pipeline.

Spurred by the advances from generative image models ho2020denoising ;
song2020denoising ; song2020score ; karras2022elucidating ; zhang2023adding ,
automatic 3D content creation poole2022dreamfusion ; wang2024prolificdreamer ;
tang2023dreamgaussian ; hong2023lrm  has witnessed remarkable progress in
terms of efficiency, fidelity, diversity, and controllability. Coupled with
the breakthroughs in 4D representation yang2023deformable ; wu20234d , these
advancements further foster substantial development in dynamic content (4D)
generation singer2023text ; bahmani20234d ; jiang2023consistent4d ;
zhao2023animate124 ; ren2023dreamgaussian4d ; gao2024gaussianflow , which
holds significant value across a wide range of applications in animation,
film, game, and MetaVerse.

Recently, 3D content generation has achieved considerable breakthroughs in
efficiency. Some works liu2023zero ; liu2023syncdreamer ; shi2023mvdream ;
wang2023imagedream ; tang2024mvdiffusion++  inject stereo knowledge into the
image generation model, enabling these 3D-aware image generators to produce
consistent multi-view images, thereby effectively stabilizing and accelerating
the optimization. Other efforts hong2023lrm ; chen2024v3d ; voleti2024sv3d ;
zuo2024videomv ; tang2024lgm  attempt to directly generate 3D representations,
such as triplane chen2022tensorf  or Gaussian splatting kerbl3Dgaussians .
However, the efficiency improvement from these works is largely data-driven
wu2023omniobject3d ; reizenstein2021common ; yu2023mvimgnet ;
deitke2023objaverse . Consequently, it is infeasible to adapt these approaches
to 4D generation due to the scarcity of synchronized multi-view video data.
Therefore, most existing optimization-based 4D generation works still suffer
from slow and unstable optimization.

However, despite the paucity of 4D data, there are vast available monocular
video data and static multi-view image data. Existing works have demonstrated
that it is feasible to train diffusion-based generative models to learn the
distribution of these two classes of data separately voleti2024sv3d ;
liu2023syncdreamer ; tang2024mvdiffusion++ ; blattmann2023align ;
blattmann2023stable . Considering that video diffusion model stores the prior
of motion and temporal smoothness, and multi-view diffusion model has sound
knowledge of geometrical consistency, combining the two generative models to
generate 4D assets becomes a highly promising and appealing approach.

Therefore, in this paper we propose a novel 4D generation framework, which
combines both the video diffusion model and multi-view diffusion model to
directly sample multi-frame multi-view image array imitating the photographing
process of 4D content. To demonstrate how we realize this combination, we
assume that such an image matrix has a nice structure: the elements not in the
same row and column are conditionally independent of each other. Based on the
property, we design a simple yet effective denoising strategy in which the
estimated score is just the convex combination of the scores predicted by two
foundation diffusion models. Our formulation is easy to be adapted to various
prompts including single image, single-view video, and static 3D content.
Unlike the existing optimization-based counterparts, our image generation is
highly parallel. Combined with efficient modern 4D reconstruction methods, we
can generate high-fidelity and diverse 4D assets within just several minutes.
Besides, our approach can also potentially benefit from the further
development of foundation diffusion models videoworldsimulators2024 .

Our contributions can be summarized into threefold: (i) We present a novel 4D
content generation framework that achieves zero-shot generation of realistic
multi-view multi-frame image arrays, which can be integrated with an off-the-
shelf modern 4D reconstruct pipeline to efficiently create 4D content. (ii) We
identify the conditional independence that existed in the distribution of the
elements composing the image arrays. And based on it, we design a simple yet
effective joint denoising strategy that combines video diffusion model and
multi-view diffusion model to directly sample multi-view multi-frame image
arrays from their natural distribution. (iii) Systematic experiments
demonstrate that our proposed method can achieve satisfactory results under
different types of prompts, including single image, single-view video, and
static 3D content.

##  2 Reletaed work

#### 3D generation

3D generation aims at creating static 3D content from different prompts like
text or image. Early efforts employed GAN-based approaches gao2022get3d ;
schwarz2020graf . Recently, significant breakthroughs have been achieved
alongside the emergence of diffusion models ho2020denoising  in this domain.
DreamFusion poole2022dreamfusion  introduced score distillation sampling (SDS)
to unleash the creativity in diffusion models. Although such approach has
exhibited promising results, the original form of SDS encounters challenges
such as mode collapse, multi-face Janus issues, and the slow optimization. A
series of subsequent works wang2024prolificdreamer ; shi2023mvdream ;
wang2023imagedream ; pan2024enhancing ; tang2023dreamgaussian ;
yi2023gaussiandreamer  try to address these problems by modifying this
mechanism. On the other hand, some studies nichol2022point ; jun2023shap ;
hong2023lrm ; tang2024lgm ; wang2024crm  have explored the direct generation
of 3D representations using diffusion models. Another line of research
liu2023zero ; liu2023syncdreamer ; long2023wonder3d ; chen2024v3d ;
tang2024mvdiffusion++  focuses on generating dense multi-view images with
sufficient 3D consistency by training or fine-tuning 2D diffusion models on 3D
datasets to make them more suitable for 3D generation tasks. The generated
images can be used for reconstruction to obtain textured meshes, point clouds
or implicit radiance fields. We also adopt this approach of directly
generating consistent images for reconstruction. But unlike these 3D
counterparts, there is no large-scale multi-view synchronized video data.
Therefore, we opt to combine geometrical consistency priors and video dynamic
priors to generate images.

#### Video generation

Video generation and prediction is an active field that has gained increasing
popularity. Recent diffusion model-based video generation methods have
exhibited unprecedented levels of realism, diversity, and controllability.
Notably, recent breakthroughs have demonstrated the scalability of video
generation models combined with Transformers and their potential as physical
world simulators. Video LDM blattmann2023align  was among the first works to
apply the latent diffusion framework rombach2022high  to video generation. The
subsequent work SVD blattmann2023stable  followed its architecture and made
effective improvements to the training recipe. W.A.L.T gupta2023photorealistic
employed a transformer with window attention tailored for spatiotemporal
generative modeling to generate high-resolution videos. VDT lu2023vdt
introduced the video diffusion transformer to flexibly capture long-distance
spatiotemporal context in videos and a spatial-temporal mask mechanism to
uniformly handle different video generation tasks. The recently introduced
SORA videoworldsimulators2024  demonstrated a remarkable capability to
generate arbitrarily sized long videos with intuitively physical fidelity.
Models trained on large-scale video data can generate videos with consistent
geometry and realistic dynamics. Furthermore, video diffusion models can also
be considered as effective 3D generators chen2024v3d ; han2024vfusion3d ;
voleti2024sv3d  to generate multi-view consistent images. Therefore, we build
our method on this flourishing domain.

#### 4D generation

Animating category-agnostic stuff is a challenging problem and has been
receiving a lot of attention from both academia and industry. Compared to 3D
generation, 4D generation requires not only predicting consistent geometry but
also generating realistic and diverse dynamics. Recent works on 4D generation
could be classified into two main streams based on the type of input prompt.
The first class of methods predicts 4D representations given a single image
and text description as input. For instance, MAV3D singer2023text  directly
deploys SDS into 4D generation and proposes a three-stage training scheme to
stably generate high-resolution videos. DreamGaussian4D ren2023dreamgaussian4d
adopt a similar three-stage training scheme but switches the underlying 4D
representation from Hexplane cao2023hexplane  to the deformable 3D Gaussian
and replaces the third stage with the refinement of high-resolution UV texture
maps, thereby achieving significantly efficiency improvement. 4DGen
yin20234dgen  first generates a set of candidate 3D assets according to the
image or text prompt and then grounds the 4D generation on user-specified
static 3D assets and monocular video sequences. 4D-fy bahmani20234d  also
adopts the idea of combining diffusion priors. But different from our method,
it combines them during SDS, resulting in an extremely slow generation process
that typically takes several days. Another line of work predicts dynamic
objects from a single-view videos. Although the motion is largely dictated
under this setting, generating spatiotemporal consistent structure still
entails considerable uncertainty due to the limited view. Therefore,
Consistent4D jiang2023consistent4d  is first proposes to use a generation
approach to address this task. Subsequently, Efficient4D pan2024fast  mimics a
photogrammetry-based 3D object capture pipeline by directly generating multi-
frame multi-view captures of dynamic 3D objects through SyncDreamer-T and
reconstructing 4D representations with them. Note that although Efficient4D
shares the same SDS-free philosophy as ours, it possesses a strong
architecture bias of the foundation diffusion model and is incapable of
synthesizing novel dynamics. Compared to previous works, our framework can
efficiently generate diverse dynamic 4D content, avoiding the slow, unstable,
and intricate multi-stage optimization, and has the potential to continuously
benefit from the scalability of the underlying diffusion model.

##  3 Method

Figure 2:  The overall pipeline of Diffusion2. (i) Given a reference image,
Diffusion2 first independently generates the animation under the reference
view (denoted $\mathcal{I}_{1:V,0}$) and the multi-view images at the
reference time (denoted $\mathcal{I}_{0,1:F}$) as the condition for the
subsequent generation of the full matrix, denoted $\mathcal{I}$. Depending on
the type of given prompt, the condition images $\mathcal{I}_{1:V,0}$ or
$\mathcal{I}_{0,1:F}$ can be specified by users. (ii) Then, Diffusion2
directly samples a dense multi-frame multi-view image array by blending the
estimated scores with a weighting scheduler from pretrained video and multi-
view diffusion models in the reverse-time SDE. (iii) The generated image
arrays are employed as supervision to optimize a continuous 4D content
representation.

In this section, we present a novel framework designed for efficient and
scalable generation of 4D content, which consists of two stages as depicted in
Figure 2. In Section 3.1 (stage-1), we will discuss how to generate a dense
multi-view multi-frame image array for reconstruction through a highly
parallelizable denoising process by integrating the pretrained video diffusion
model and multi-view diffusion model, and why it is feasible. In Section 3.2
(stage-2), we will briefly illustrate how to robustly reconstruct 4D content
from the image matrix produced in the first stage.

###  3.1 Image matrix generation

In this stage, our goal is to generate dense multi-frame multi-view images for
reconstruction, which can be denoted as a matrix of image

$\mathcal{I}=\left\\{I_{i,j}\in\mathbb{R}^{H\times W\times
3}\right\\}_{i=1,j=1}^{V,F}=\begin{bmatrix}{\color[rgb]{0,.5,.5}\definecolor[named]{pgfstrokecolor}{rgb}{0,.5,.5}I_{1,1}}&{\color[rgb]{0,.5,.5}\definecolor[named]{pgfstrokecolor}{rgb}{0,.5,.5}\cdots}&{\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}I_{1,j}}&{\color[rgb]{0,.5,.5}\definecolor[named]{pgfstrokecolor}{rgb}{0,.5,.5}\cdots}&{\color[rgb]{0,.5,.5}\definecolor[named]{pgfstrokecolor}{rgb}{0,.5,.5}I_{1,F}}\\\
{\color[rgb]{0,.5,.5}\definecolor[named]{pgfstrokecolor}{rgb}{0,.5,.5}\vdots}&{\color[rgb]{0,.5,.5}\definecolor[named]{pgfstrokecolor}{rgb}{0,.5,.5}\ddots}&{\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\vdots}&{\color[rgb]{0,.5,.5}\definecolor[named]{pgfstrokecolor}{rgb}{0,.5,.5}\ddots}&{\color[rgb]{0,.5,.5}\definecolor[named]{pgfstrokecolor}{rgb}{0,.5,.5}\vdots}\\\
{\color[rgb]{1,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,1}\pgfsys@color@cmyk@stroke{0}{1}{0}{0}\pgfsys@color@cmyk@fill{0}{1}{0}{0}I_{i,1}}&{\color[rgb]{1,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,1}\pgfsys@color@cmyk@stroke{0}{1}{0}{0}\pgfsys@color@cmyk@fill{0}{1}{0}{0}\cdots}&I_{i,j}&{\color[rgb]{1,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,1}\pgfsys@color@cmyk@stroke{0}{1}{0}{0}\pgfsys@color@cmyk@fill{0}{1}{0}{0}\cdots}&{\color[rgb]{1,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,1}\pgfsys@color@cmyk@stroke{0}{1}{0}{0}\pgfsys@color@cmyk@fill{0}{1}{0}{0}I_{i,F}}\\\
{\color[rgb]{0,.5,.5}\definecolor[named]{pgfstrokecolor}{rgb}{0,.5,.5}\vdots}&{\color[rgb]{0,.5,.5}\definecolor[named]{pgfstrokecolor}{rgb}{0,.5,.5}\ddots}&{\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\vdots}&{\color[rgb]{0,.5,.5}\definecolor[named]{pgfstrokecolor}{rgb}{0,.5,.5}\ddots}&{\color[rgb]{0,.5,.5}\definecolor[named]{pgfstrokecolor}{rgb}{0,.5,.5}\vdots}\\\
{\color[rgb]{0,.5,.5}\definecolor[named]{pgfstrokecolor}{rgb}{0,.5,.5}I_{V,1}}&{\color[rgb]{0,.5,.5}\definecolor[named]{pgfstrokecolor}{rgb}{0,.5,.5}\cdots}&{\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}I_{V,j}}&{\color[rgb]{0,.5,.5}\definecolor[named]{pgfstrokecolor}{rgb}{0,.5,.5}\cdots}&{\color[rgb]{0,.5,.5}\definecolor[named]{pgfstrokecolor}{rgb}{0,.5,.5}I_{V,F}}\end{bmatrix},$
(1)

where $V$ is the number of views, $F$ is the number of video frames, and
$(H,W)$ is the size of images. We aim to construct a generative model that
allows us to directly sample $\mathcal{I}\sim p(\mathcal{I})$.

Now, let us first distract our attention to reviewing existing diffusion-based
generators for video and multi-view images, which can be utilized for sampling
realistic images through the following probabilistic flow ODE:

$d\mathrm{x}=-\dot{\sigma}(t)\sigma(t)\nabla_{\mathrm{x}}\log
p\left(\mathrm{x};\sigma(t)\right)dt.$ (2)

Here, $\mathrm{x}=\left\\{I_{i}\in\mathbb{R}^{H\times W\times
3}\right\\}_{i=1}^{N}$ is a series of images with $N$ frames or $N$ views,
$\nabla_{\mathrm{x}}\log p\left(\mathrm{x};\sigma\right)$ is the score
function, which can be parameterized as $\nabla_{\mathrm{x}}\log
p\left(\mathrm{x};\sigma\right)\approx\left(D_{\theta}(\mathrm{x};\sigma)\right)/\sigma^{2}$
karras2022elucidating ; blattmann2023stable , where
$D_{\theta}(\mathrm{x};\sigma)$ is a neural network trained via denoising
score matching.

We want to extend the above formulation to the sampling of $\mathcal{I}$. The
question is, how do we estimate the score function of the joint distribution
of $V\times F$ images?

For simplicity, let $\mathcal{I}_{-i,j}\triangleq\\{I_{i^{\prime},j}|1\leq
i^{\prime}\leq V,i^{\prime}\neq i\\}$,
$\mathcal{I}_{i,-j}\triangleq\\{I_{i,j^{\prime}}|1\leq j^{\prime}\leq
F,j^{\prime}\neq j\\}$ and
$\mathcal{I}_{-i,-j}\triangleq\\{I_{i^{\prime},j^{\prime}}|1\leq
i^{\prime}\leq V,1\leq j^{\prime}\leq F,i^{\prime}\neq i,j^{\prime}\neq j\\}$.
We first make an assumption on the structure of $p(\mathcal{I})$.

######  Assumption 3.1.

Given any image $I_{i,j}$, the underlying geometry  $\mathcal{I}_{-i,j}$ and
the dynamics  $\mathcal{I}_{i,-j}$ are conditionally independent, i.e.,

$p\left({\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mathcal{I}_{-i,j}},{\color[rgb]{1,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,1}\pgfsys@color@cmyk@stroke{0}{1}{0}{0}\pgfsys@color@cmyk@fill{0}{1}{0}{0}\mathcal{I}_{i,-j}}|I_{i,j}\right)=p\left({\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\mathcal{I}_{-i,j}}|I_{i,j}\right)p\left({\color[rgb]{1,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,1}\pgfsys@color@cmyk@stroke{0}{1}{0}{0}\pgfsys@color@cmyk@fill{0}{1}{0}{0}\mathcal{I}_{i,-j}}|I_{i,j}\right).$
(3)

The assumption 3.1 implies that given the front view of a 3D object, its
motion as seen from the front does not correlate with its appearance from the
back, which aligns with our intuition. A natural corollary is that the
mollified distribution derived by adding Gaussian noise into the data
distribution still maintains conditional independence:

######  Corollary 3.1.

Denote $\hat{\mathcal{I}}$ as the noisy version of $\mathcal{I}$, i.e.,

$\displaystyle\hat{\mathcal{I}}=\\{\hat{I}_{i,j}\in\mathbb{R}^{H\times W\times
3}\\}_{i=1,j=1}^{V,F}\quad\text{with}~{}\hat{I}_{i,j}=\alpha
I_{i,j}+\varepsilon_{i,j},$ (4)

where $\alpha\in\mathbb{R}$ is a constant and
$\varepsilon_{i,j}\in\mathbb{R}^{H\times W\times 3}$ are independent Gaussian
noises. Then we have

$p\left({\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\hat{\mathcal{I}}_{-i,j}},{\color[rgb]{1,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,1}\pgfsys@color@cmyk@stroke{0}{1}{0}{0}\pgfsys@color@cmyk@fill{0}{1}{0}{0}\hat{\mathcal{I}}_{i,-j}}|\hat{I}_{i,j}\right)=p\left({\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\hat{\mathcal{I}}_{-i,j}}|\hat{I}_{i,j}\right)p\left({\color[rgb]{1,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,1}\pgfsys@color@cmyk@stroke{0}{1}{0}{0}\pgfsys@color@cmyk@fill{0}{1}{0}{0}\hat{\mathcal{I}}_{i,-j}}|\hat{I}_{i,j}\right).$
(5)

This nice property allows us to sample the desired image matrix by
progressively denoising from pure Gaussian noise through the combination of
two estimated scores of its marginal distribution, which can be obtained from
the pretrained video and multi-view diffusion models respectively. Therefore,
we can derive our main theorem.

######  Theorem 3.1.

For $\mathrm{x}=\hat{I}_{i,j}$, we have

$\nabla_{\mathrm{x}}\log
p\left(\hat{\mathcal{I}}\right)=\nabla_{\mathrm{x}}\log
p\left(\hat{\mathcal{I}}_{\\{1:V\\},j}\right)+\nabla_{\mathrm{x}}\log
p\left(\hat{\mathcal{I}}_{i,\\{1:F\\}}\right)-\nabla_{\mathrm{x}}\log
p\left(\hat{I}_{i,j}\right).$ (6)

###### Proof.

We first decompose $p\left(\hat{\mathcal{I}}\right)$ by

$\displaystyle p\left(\hat{\mathcal{I}}\right)$
$\displaystyle=p\left(\hat{I}_{i,j},{\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\hat{\mathcal{I}}_{-i,j}},{\color[rgb]{1,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,1}\pgfsys@color@cmyk@stroke{0}{1}{0}{0}\pgfsys@color@cmyk@fill{0}{1}{0}{0}\hat{\mathcal{I}}_{i,-j}},{\color[rgb]{0,.5,.5}\definecolor[named]{pgfstrokecolor}{rgb}{0,.5,.5}\hat{\mathcal{I}}_{-i,-j}}\right)$
(7)
$\displaystyle=p\left(\hat{I}_{i,j},{\color[rgb]{0,.5,.5}\definecolor[named]{pgfstrokecolor}{rgb}{0,.5,.5}\hat{\mathcal{I}}_{-i,-j}}|{\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\hat{\mathcal{I}}_{-i,j}},{\color[rgb]{1,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,1}\pgfsys@color@cmyk@stroke{0}{1}{0}{0}\pgfsys@color@cmyk@fill{0}{1}{0}{0}\hat{\mathcal{I}}_{i,-j}}\right)p\left({\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\hat{\mathcal{I}}_{-i,j}},{\color[rgb]{1,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,1}\pgfsys@color@cmyk@stroke{0}{1}{0}{0}\pgfsys@color@cmyk@fill{0}{1}{0}{0}\hat{\mathcal{I}}_{i,-j}}\right).$
(8)

Note that for any
$\hat{I}_{i^{\prime},j^{\prime}}\in{\color[rgb]{0,.5,.5}\definecolor[named]{pgfstrokecolor}{rgb}{0,.5,.5}\hat{\mathcal{I}}_{-i,-j}}$,
$I_{i^{\prime},j^{\prime}}$ and $I_{i,j}$ are independent conditioned on
$I_{i^{\prime},j}$ by corollary 3.1, hence

$p\left(\hat{I}_{i,j},{\color[rgb]{0,.5,.5}\definecolor[named]{pgfstrokecolor}{rgb}{0,.5,.5}\hat{\mathcal{I}}_{-i,-j}}|{\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\hat{\mathcal{I}}_{-i,j}},{\color[rgb]{1,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,1}\pgfsys@color@cmyk@stroke{0}{1}{0}{0}\pgfsys@color@cmyk@fill{0}{1}{0}{0}\hat{\mathcal{I}}_{i,-j}}\right)=p\left({\color[rgb]{0,.5,.5}\definecolor[named]{pgfstrokecolor}{rgb}{0,.5,.5}\hat{\mathcal{I}}_{-i,-j}}|{\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\hat{\mathcal{I}}_{-i,j}},{\color[rgb]{1,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,1}\pgfsys@color@cmyk@stroke{0}{1}{0}{0}\pgfsys@color@cmyk@fill{0}{1}{0}{0}\hat{\mathcal{I}}_{i,-j}}\right)p\left(\hat{I}_{i,j}|{\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\hat{\mathcal{I}}_{-i,j}},{\color[rgb]{1,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,1}\pgfsys@color@cmyk@stroke{0}{1}{0}{0}\pgfsys@color@cmyk@fill{0}{1}{0}{0}\hat{\mathcal{I}}_{i,-j}}\right).$
(9)

Since
$p\left({\color[rgb]{0,.5,.5}\definecolor[named]{pgfstrokecolor}{rgb}{0,.5,.5}\hat{\mathcal{I}}_{-i,-j}}|{\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\hat{\mathcal{I}}_{-i,j}},{\color[rgb]{1,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,1}\pgfsys@color@cmyk@stroke{0}{1}{0}{0}\pgfsys@color@cmyk@fill{0}{1}{0}{0}\hat{\mathcal{I}}_{i,-j}}\right)$
does not contain $I_{i,j}$, its derivative with respect to $I_{i,j}$ is zero.
Then combined with equation (8) and equation (9), taking the derivative of
$\log p\left(\hat{\mathcal{I}}\right)$ with respect to $\mathrm{x}$, we
achieve

$\displaystyle\nabla_{\mathrm{x}}\log p\left(\hat{\mathcal{I}}\right)$
$\displaystyle=\nabla_{\mathrm{x}}\log
p\left(\hat{I}_{i,j}|{\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\hat{\mathcal{I}}_{-i,j}},{\color[rgb]{1,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,1}\pgfsys@color@cmyk@stroke{0}{1}{0}{0}\pgfsys@color@cmyk@fill{0}{1}{0}{0}\hat{\mathcal{I}}_{i,-j}}\right)p\left({\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\hat{\mathcal{I}}_{-i,j}},{\color[rgb]{1,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,1}\pgfsys@color@cmyk@stroke{0}{1}{0}{0}\pgfsys@color@cmyk@fill{0}{1}{0}{0}\hat{\mathcal{I}}_{i,-j}}\right)$
(10) $\displaystyle=\nabla_{\mathrm{x}}\log
p\left(\hat{I}_{i,j},{\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\hat{\mathcal{I}}_{-i,j}},{\color[rgb]{1,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,1}\pgfsys@color@cmyk@stroke{0}{1}{0}{0}\pgfsys@color@cmyk@fill{0}{1}{0}{0}\hat{\mathcal{I}}_{i,-j}}\right).$
(11)

Finally, by further decomposing
$p\left(\hat{I}_{i,j},{\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\hat{\mathcal{I}}_{-i,j}},{\color[rgb]{1,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,1}\pgfsys@color@cmyk@stroke{0}{1}{0}{0}\pgfsys@color@cmyk@fill{0}{1}{0}{0}\hat{\mathcal{I}}_{i,-j}}\right)$
and directly applying corollary 3.1, we obtain

$\displaystyle\nabla_{\mathrm{x}}\log p\left(\hat{\mathcal{I}}\right)$
$\displaystyle=\nabla_{\mathrm{x}}\log
p\left({\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\hat{\mathcal{I}}_{-i,j}},{\color[rgb]{1,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,1}\pgfsys@color@cmyk@stroke{0}{1}{0}{0}\pgfsys@color@cmyk@fill{0}{1}{0}{0}\hat{\mathcal{I}}_{i,-j}}|\hat{I}_{i,j}\right)p\left(\hat{I}_{i,j}\right)$
(12) $\displaystyle=\nabla_{\mathrm{x}}\log
p\left({\color[rgb]{0,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{0,0,1}\hat{\mathcal{I}}_{-i,j}}|\hat{I}_{i,j}\right)p\left({\color[rgb]{1,0,1}\definecolor[named]{pgfstrokecolor}{rgb}{1,0,1}\pgfsys@color@cmyk@stroke{0}{1}{0}{0}\pgfsys@color@cmyk@fill{0}{1}{0}{0}\hat{\mathcal{I}}_{i,-j}}|\hat{I}_{i,j}\right)p\left(\hat{I}_{i,j}\right)$
(13)
$\displaystyle=\nabla_{\mathrm{x}}\log\frac{p\left(\hat{\mathcal{I}}_{\\{1:V\\},j}\right)p\left(\hat{\mathcal{I}}_{i,\\{1:F\\}}\right)}{p\left(\hat{I}_{i,j}\right)}$
(14) $\displaystyle=\nabla_{\mathrm{x}}\log
p\left(\hat{\mathcal{I}}_{\\{1:V\\},j}\right)+\nabla_{\mathrm{x}}\log
p\left(\hat{\mathcal{I}}_{i,\\{1:F\\}}\right)-\nabla_{\mathrm{x}}\log
p\left(\hat{I}_{i,j}\right).$ (15)

∎

Here $\nabla_{\mathrm{x}}\log p\left(\hat{\mathcal{I}}_{i,\\{1:F\\}}\right)$
and $\nabla_{\mathrm{x}}\log p\left(\hat{\mathcal{I}}_{\\{1:V\\},j}\right)$
are just the score functions of the video diffusion model and the multi-view
diffusion model respectively . And we use the convex combination of them to
estimate $\nabla_{\mathrm{x}}\log p\left(\hat{I}_{i,j}\right)$ as:

$\nabla_{\mathrm{x}}\log
p\left(\hat{I}_{i,j}\right)=(1-s)\nabla_{\mathrm{x}}\log
p\left(\hat{\mathcal{I}}_{i,\\{1:F\\}}\right)+s\nabla_{\mathrm{x}}\log
p\left(\hat{\mathcal{I}}_{\\{1:V\\},j}\right).$ (16)

In practice, we employ a logistic schedule to adjust the change of $s$ with
denoising step. Given the current denoising step $i$ and the number of total
steps $N$, we set $s=1-\frac{1}{1+e^{k(i/N-s_{0})}}$. This function has a
sigmoidal curve, which is relatively flat at the extremes away from middle
$s_{0}$ and changes sharply near it, with the derivative controlled by $k$.
This schedule decouples the generation of consistent geometry and temporally
smooth appearance to some extent.

#### Sampling in latent space

For convenience, the above theorem assumes that the sample object is the image
in the original RGB space. However, modern high-resolution diffusion models
typically generate images encoded into a latent space by VQVAE van2017neural .
The legitimacy of the aforementioned derivation requires that the multi-view
generator and the video generator share the same latent space. Although this
requirement is not met in the most current instantiation of them, we believe
that this condition will be increasingly satisfied by more multi-view
generation models in the future. Because as pointed out by SVD
blattmann2023stable ; chen2024v3d ; videoworldsimulators2024 ; li2024sora ,
video generation models trained on large-scale video datasets have learned a
strong stereo knowledge, thus can provide a better pretraining for fine-tuning
multi-view diffusion models than those trained solely on image data. And the
latent encoder is usually frozen during fine-tuning on the multi-view images.

#### Generation with various input conditions

Note that the formulation described above is based on unconditional
generation. However, we are more interested in controllable generation in
practice. Then we will extend the aforementioned process to conditional
generation. Consider the augmented matrix $\mathcal{I}_{\text{aug}}$ defined
by

$\mathcal{I}_{\text{aug}}=\begin{bmatrix}I_{0,0}&\mathcal{I}_{0,\\{1:V\\}}\\\
\mathcal{I}_{\\{1:F\\},0}&\mathcal{I}\end{bmatrix},$ (17)

where $I_{0,0}$ are the input image, and
$\mathcal{I}_{0,\\{1:V\\}},\mathcal{I}_{\\{1:F\\},0}$ is the first row and
column of $\mathcal{I}_{\text{aug}}$ we need to first create as the condition
for the subsequent generation of full matrix. Then we will demonstrate how we
obtain them from various conditions. For convenience, we denote $\mathbf{V}$
as the multi-view diffusion model and $\mathbf{F}$ as the video diffusion
model.

  * •

Single image. Given $I_{0,0}$ as input, we use $\mathbf{V}$ to generate
$\mathcal{I}_{0,\\{1:V\\}}$ which dictates the geometry of the generated 4D
assets and use $\mathbf{F}$ to generate $\mathcal{I}_{\\{1:F\\},0}$ which
endows the static image with dynamics.

  * •

Single-view video. Given $\mathcal{I}_{\\{0:F\\},0}$ as input, we use the last
frame $I_{0,0}$ as the condition of $\mathbf{V}$ to generate
$\mathcal{I}_{0,\\{1:V\\}}$.

  * •

Static 3D model. Similarly, Given $\mathcal{I}_{0,\\{0:V\\}}$ as input, we use
the front view $I_{0,0}$ as the condition of $\mathbf{F}$ to generate
$\mathcal{I}_{\\{1:F\\},0}$.

Assumption 3.1 ensures the safety of independently generating the geometry
$\mathcal{I}_{0,\\{1:V\\}}$ and the motion $\mathcal{I}_{\\{1:F\\},0}$.
Additionally, there is no computational or data dependency between these two
generation processes, allowing their total time cost to be reduced to a single
reverse diffusion process. Then we will denoise the rest part of
$\mathcal{I}_{\text{aug}}$ from pure Gaussian noise. In each step, we run the
score estimators for each row and column conditioned on the
$\mathcal{I}_{0,\\{1:V\\}}$ and $\mathcal{I}_{\\{1:F\\},0}$, and combine their
results as in Theorem 3.1 to update the noisy latent. Since the score
estimation for each row and column can also be parallelized, the time cost can
be decreased to running a single diffusion step. Therefore, with sufficient
GPU memory, the total time spent on the process illustrated in Figure 2(ii)
remains the same as that for generating a single video.

###  3.2 Robust reconstruction

#### 4D representation

Given generated synchronized multi-view videos conditioned on any type of
prompts, there are numerous methods that can be employed to reconstruct 4D
assets. Among the numerous candidates, we adopt the 4D Gaussian Splatting due
to its superior fitting capabilities and efficient optimization given the
dense multi-view supervision.

#### Optimization

Although the images generated in the first stage already have intuitively
satisfactory spatiotemporal consistency, the performance limitation of the
foundational multi-view generation components still makes it difficult to
achieve precise pixel-level matching across different views and frames.
Therefore, we follow chen2024v3d  to optimize the combination of perception
loss $\mathcal{L}_{lpips}$ zhang2018unreasonable  and D-SSIM
$\mathcal{L}_{ssim}$ wang2004image  while ignoring L1 loss. In addition, we
weight each term with the confidence score, then the total objective is
defined as
$\mathcal{L}_{total}=\lambda_{lpips}\mathcal{C}_{lpips}\mathcal{L}_{lpips}+\lambda_{ssim}\mathcal{C}_{ssim}\mathcal{L}_{ssim}$,
where $\mathcal{C}_{ssim}$ is just the ssim between ground truth and rendered
images and $\mathcal{C}_{lpips}$ is defined as $1-\mathcal{L}_{lpips}$.

##  4 Experiments

###  4.1 Implementation details

In the first stage, we use Stable Video Diffusion blattmann2023stable  as our
foundation video diffusion model, predicting 25 frames each time according to
the image prompt. SV3Dp voleti2024sv3d  is chosen as the foundation multi-view
diffusion model. For simplicity, we only generate orbital videos that have 21
uniformly spaced azimuths and fixed elevation with manual filter of side view
as these views typically contain thin structures that pose challenges for the
video generation model and subsequent reconstruction processes. By default, we
set the number of sampling steps to 50 for both generative models. In the
reconstruction stage, we optimized 4D Gaussian Splatting for 5k iterations
without bells and whistles. The image size is set to (576 $\times$ 576) in
both stages.

(a) Image-to-4D

(b) Video-to-4D

Figure 3:  Qualitative comparisons on (a) image-to-4D generation and (b)
video-to-4D generation.

###  4.2 4D generation from single image

Table 1:  User study on image-to-4D generation. The proportions of different methods that best match user preferences under three criteria are reported.  Method | Details | Geometrical | Temporal | Overall model | Generation
---|---|---|---|---|---
consistency | smoothness | quality | time
Animate124 zhao2023animate124  | 11.3% | 31.0% | 16.0% | 18.0% | 9h
DreamGaussian4D ren2023dreamgaussian4d  | 27.7% | 24.3% | 48.0% | 25.7% | 12m
Diffusion2 (Ours) | 61.0% | 44.7% | 36.0% | 56.3% | 10m

Figure 3 (a), we show the results generated by the proposed method and provide
the comparison with other alternatives. It can be observed that our concise
and elegant pipeline is capable of generating 4D assets of comparable quality
to those produced by state-of-the-art SDS-based methods with sophisticated
multi-stage optimization. Furthermore, the parallel evaluation of a large 2D
generative model can provide a potential efficiency advantage for our method.
We also conducted a user study (See appendix for details), the results of
which are reported in Table 1. It suggests that our method garnered the
highest human preference in the multi-view consistency, detail, and overall
model quality.

###  4.3 4D generation from single view video

Table 2:  Quantitative comparisons on video-to-4D generation.  Method | Type | CLIP Similarity $\uparrow$ | Generation time $\downarrow$
---|---|---|---
Consistent4D jiang2023consistent4d  | Optimization-based | 0.87 | 2h
4DGen yin20234dgen  | Optimization-based | 0.89 | 2h10m
Efficient4D pan2024fast  | Photogrammetry-based | 0.92 | 14m
Diffusion2 (Ours) | Photogrammetry-based | 0.94 | 10m

Generating 4D dynamic objects from fixed-view video is a practical task first
introduced in jiang2023consistent4d . Compared to generation from a single
image, this task additionally constrains the object’s motion. Our proposed
framework can be easily adapted to deal with this task as detailed in Section
3.1. We perform both quantitative and qualitative comparisons with other
counterparts under this setting. The qualitative result is shown in Figure 3
(b). It can be seen that our method slightly reduces the over-saturated
appearance. For qualitative evaluation, we report the CLIP-similarity between
generated views and ground truth images to indicate overall semantic
consistency and the recognizable of generated images. The quantitative metrics
also support our superiority.

###  4.4 4D generation from static 3D content

Figure 4:  Synthesized images from static 3D models.

Naturally, Diffusion2 can also animate static 3D models into dynamic objects
as described in Section 3.1, which has substantial practical implications.
From Figure 4 we can observe that our method is capable of endowing 3D models
with diverse and realistic dynamics while maintaining satisfactory temporal
and geometrical consistency.

###  4.5 Ablation studies

Figure 5:  Ablation studies. (a) The parameter controlling the the logistic
schedule. (b) Different type of scale schedule. Best viewed with zoom-in.

Since previous methods have explored dynamic content reconstruction well, we
mainly focus on the ablation of key design choices in the stage of image
matrix generation. In Figure 5 (a), we adjusted two parameters that control
the curve of the logistic schedule. The results reveal that: when only using
video prior, the generated images fail to ensure consistency in perspective,
geometry, and detail with other views. For example, in the second row and
second column of Figure 5 (a), the dress still evenly sags down both sides of
the body, unlike in the reference view where the dress flutters to the left
side of the body. This indicates that without the guidance of a geometry
prior, the dynamics of each view will be totally independent of each other.
Another extreme is when we set $s$ to 1, that is, using only multi-view prior,
where it can no longer guarantee the consistency of details between different
frames of invisible views. The comparison of the last column of the first two
rows of Figure 5 (a) shows a remarkable change in the shape of the tail of the
hair. So we finally adopt a compromised option to set $s$ to 0.5, i.e.,
starting to drastically reduce the weight of the multi-view score at halfway
of the denoising process. In most cases, this choice can achieve smooth
temporal transitions as well as geometrical consistency. In addition, the
decreasing speed of $s$ may also affect the quality of generated images, which
is controlled by another parameter $k$. Therefore, we test the image
generation under different $k$ given $s=0.5$. It can be seen that when $k$ is
relatively small, the generated images may exhibit ghosting effects. On the
other hand, when $k$ is too large, it may weaken the temporal smoothing effect
brought by the video diffusion model in the early stages of denoising.
Consequently, we adopt $s=0.5,k=20$ as the default setting. Finally, we also
examined the impact of different noise schedules, as shown in Figure 5 (b). It
can be observed that all other choices would lead to ghosting artifacts.

##  5 Conclusion

In this work, we introduce Diffusion2, a novel framework for creating dynamic
3D content. This framework first generates a dense array of multi-view and
multi-frame images with high parallelization, which are then used to build a
full 4D representation through a reconstruction process. The core assumption
behind this framework is that the motion of an object viewed from one angle
and its appearance from another are conditionally independent. Based on this
assumption, we prove that we can directly sample synchronized multi-view
videos in a denoising process by combining pretrained video and multi-view
diffusion models. Our experimental results demonstrate the flexibility and
effectiveness of our framework, showing that it can adapt to various prompts
to produce high-quality 4D content quickly and reliably. We believe our work
opens new pathways for future research, particularly in exploring the rich
geometric and dynamic knowledge embedded in foundational 3D-aware and video
diffusion models.

## References

  * (1) Sherwin Bahmani, Ivan Skorokhodov, Victor Rong, Gordon Wetzstein, Leonidas Guibas, Peter Wonka, Sergey Tulyakov, Jeong Joon Park, Andrea Tagliasacchi, and David B Lindell.  4d-fy: Text-to-4d generation using hybrid score distillation sampling.  In CVPR, 2024.
  * (2) Andreas Blattmann, Tim Dockhorn, Sumith Kulal, Daniel Mendelevitch, Maciej Kilian, Dominik Lorenz, Yam Levi, Zion English, Vikram Voleti, Adam Letts, et al.  Stable video diffusion: Scaling latent video diffusion models to large datasets.  arXiv preprint, 2023.
  * (3) Andreas Blattmann, Robin Rombach, Huan Ling, Tim Dockhorn, Seung Wook Kim, Sanja Fidler, and Karsten Kreis.  Align your latents: High-resolution video synthesis with latent diffusion models.  In CVPR, 2023.
  * (4) Tim Brooks, Bill Peebles, Connor Holmes, Will DePue, Yufei Guo, Li Jing, David Schnurr, Joe Taylor, Troy Luhman, Eric Luhman, Clarence Ng, Ricky Wang, and Aditya Ramesh.  Video generation models as world simulators.  https://openai.com/research/video-generation-models-as-world-simulators, 2024.
  * (5) Ang Cao and Justin Johnson.  Hexplane: A fast representation for dynamic scenes.  In CVPR, 2023.
  * (6) Anpei Chen, Zexiang Xu, Andreas Geiger, Jingyi Yu, and Hao Su.  Tensorf: Tensorial radiance fields.  In ECCV, 2022.
  * (7) Zilong Chen, Yikai Wang, Feng Wang, Zhengyi Wang, and Huaping Liu.  V3d: Video diffusion models are effective 3d generators.  arXiv preprint, 2024.
  * (8) Matt Deitke, Dustin Schwenk, Jordi Salvador, Luca Weihs, Oscar Michel, Eli VanderBilt, Ludwig Schmidt, Kiana Ehsani, Aniruddha Kembhavi, and Ali Farhadi.  Objaverse: A universe of annotated 3d objects.  In CVPR, 2023.
  * (9) Jun Gao, Tianchang Shen, Zian Wang, Wenzheng Chen, Kangxue Yin, Daiqing Li, Or Litany, Zan Gojcic, and Sanja Fidler.  Get3d: A generative model of high quality 3d textured shapes learned from images.  In NeurIPS, 2022.
  * (10) Quankai Gao, Qiangeng Xu, Zhe Cao, Ben Mildenhall, Wenchao Ma, Le Chen, Danhang Tang, and Ulrich Neumann.  Gaussianflow: Splatting gaussian dynamics for 4d content creation.  arXiv preprint, 2024.
  * (11) Agrim Gupta, Lijun Yu, Kihyuk Sohn, Xiuye Gu, Meera Hahn, Li Fei-Fei, Irfan Essa, Lu Jiang, and José Lezama.  Photorealistic video generation with diffusion models.  arXiv preprint, 2023.
  * (12) Junlin Han, Filippos Kokkinos, and Philip Torr.  Vfusion3d: Learning scalable 3d generative models from video diffusion models.  arXiv preprint, 2024.
  * (13) Jonathan Ho, Ajay Jain, and Pieter Abbeel.  Denoising diffusion probabilistic models.  In NeurIPS, 2020.
  * (14) Yicong Hong, Kai Zhang, Jiuxiang Gu, Sai Bi, Yang Zhou, Difan Liu, Feng Liu, Kalyan Sunkavalli, Trung Bui, and Hao Tan.  Lrm: Large reconstruction model for single image to 3d.  In ICLR, 2024.
  * (15) Yanqin Jiang, Li Zhang, Jin Gao, Weimin Hu, and Yao Yao.  Consistent4d: Consistent 360 $\\{$$\backslash$deg$\\}$ dynamic object generation from monocular video.  In ICLR, 2024.
  * (16) Heewoo Jun and Alex Nichol.  Shap-e: Generating conditional 3d implicit functions.  arXiv preprint, 2023.
  * (17) Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine.  Elucidating the design space of diffusion-based generative models.  In NeurIPS, 2022.
  * (18) Bernhard Kerbl, Georgios Kopanas, Thomas Leimkühler, and George Drettakis.  3d gaussian splatting for real-time radiance field rendering.  In ACM TOG, 2023.
  * (19) Xuanyi Li, Daquan Zhou, Chenxu Zhang, Shaodong Wei, Qibin Hou, and Ming-Ming Cheng.  Sora generates videos with stunning geometrical consistency.  arXiv preprint, 2024.
  * (20) Ruoshi Liu, Rundi Wu, Basile Van Hoorick, Pavel Tokmakov, Sergey Zakharov, and Carl Vondrick.  Zero-1-to-3: Zero-shot one image to 3d object.  In ICCV, 2023.
  * (21) Yuan Liu, Cheng Lin, Zijiao Zeng, Xiaoxiao Long, Lingjie Liu, Taku Komura, and Wenping Wang.  Syncdreamer: Generating multiview-consistent images from a single-view image.  In ICLR, 2024.
  * (22) Xiaoxiao Long, Yuan-Chen Guo, Cheng Lin, Yuan Liu, Zhiyang Dou, Lingjie Liu, Yuexin Ma, Song-Hai Zhang, Marc Habermann, Christian Theobalt, et al.  Wonder3d: Single image to 3d using cross-domain diffusion.  arXiv preprint, 2023.
  * (23) Haoyu Lu, Guoxing Yang, Nanyi Fei, Yuqi Huo, Zhiwu Lu, Ping Luo, and Mingyu Ding.  Vdt: General-purpose video diffusion transformers via mask modeling.  In ICLR, 2024.
  * (24) Alex Nichol, Heewoo Jun, Prafulla Dhariwal, Pamela Mishkin, and Mark Chen.  Point-e: A system for generating 3d point clouds from complex prompts.  arXiv preprint, 2022.
  * (25) Zijie Pan, Jiachen Lu, Xiatian Zhu, and Li Zhang.  Enhancing high-resolution 3d generation through pixel-wise gradient clipping.  In ICLR, 2024.
  * (26) Zijie Pan, Zeyu Yang, Xiatian Zhu, and Li Zhang.  Fast dynamic 3d object generation from a single-view video.  arXiv preprint, 2024.
  * (27) Ben Poole, Ajay Jain, Jonathan T Barron, and Ben Mildenhall.  Dreamfusion: Text-to-3d using 2d diffusion.  In ICLR, 2023.
  * (28) Jeremy Reizenstein, Roman Shapovalov, Philipp Henzler, Luca Sbordone, Patrick Labatut, and David Novotny.  Common objects in 3d: Large-scale learning and evaluation of real-life 3d category reconstruction.  In ICCV, 2021.
  * (29) Jiawei Ren, Liang Pan, Jiaxiang Tang, Chi Zhang, Ang Cao, Gang Zeng, and Ziwei Liu.  Dreamgaussian4d: Generative 4d gaussian splatting.  arXiv preprint, 2023.
  * (30) Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.  High-resolution image synthesis with latent diffusion models.  In CVPR, 2022.
  * (31) Katja Schwarz, Yiyi Liao, Michael Niemeyer, and Andreas Geiger.  Graf: Generative radiance fields for 3d-aware image synthesis.  In NeurIPS, 2020.
  * (32) Yichun Shi, Peng Wang, Jianglong Ye, Mai Long, Kejie Li, and Xiao Yang.  Mvdream: Multi-view diffusion for 3d generation.  arXiv preprint, 2023.
  * (33) Uriel Singer, Shelly Sheynin, Adam Polyak, Oron Ashual, Iurii Makarov, Filippos Kokkinos, Naman Goyal, Andrea Vedaldi, Devi Parikh, Justin Johnson, et al.  Text-to-4d dynamic scene generation.  arXiv preprint, 2023.
  * (34) Jiaming Song, Chenlin Meng, and Stefano Ermon.  Denoising diffusion implicit models.  In ICLR, 2021.
  * (35) Yang Song, Jascha Sohl-Dickstein, Diederik P Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole.  Score-based generative modeling through stochastic differential equations.  In ICLR, 2021.
  * (36) Jiaxiang Tang, Zhaoxi Chen, Xiaokang Chen, Tengfei Wang, Gang Zeng, and Ziwei Liu.  Lgm: Large multi-view gaussian model for high-resolution 3d content creation.  arXiv preprint, 2024.
  * (37) Jiaxiang Tang, Jiawei Ren, Hang Zhou, Ziwei Liu, and Gang Zeng.  Dreamgaussian: Generative gaussian splatting for efficient 3d content creation.  In ICLR, 2024.
  * (38) Shitao Tang, Jiacheng Chen, Dilin Wang, Chengzhou Tang, Fuyang Zhang, Yuchen Fan, Vikas Chandra, Yasutaka Furukawa, and Rakesh Ranjan.  Mvdiffusion++: A dense high-resolution multi-view diffusion model for single or sparse-view 3d object reconstruction.  arXiv preprint, 2024.
  * (39) Aaron Van Den Oord, Oriol Vinyals, et al.  Neural discrete representation learning.  In NeurIPS, 2017.
  * (40) Vikram Voleti, Chun-Han Yao, Mark Boss, Adam Letts, David Pankratz, Dmitry Tochilkin, Christian Laforte, Robin Rombach, and Varun Jampani.  Sv3d: Novel multi-view synthesis and 3d generation from a single image using latent video diffusion.  arXiv preprint, 2024.
  * (41) Peng Wang and Yichun Shi.  Imagedream: Image-prompt multi-view diffusion for 3d generation.  arXiv preprint, 2023.
  * (42) Zhengyi Wang, Cheng Lu, Yikai Wang, Fan Bao, Chongxuan Li, Hang Su, and Jun Zhu.  Prolificdreamer: High-fidelity and diverse text-to-3d generation with variational score distillation.  In NeurIPS, 2023.
  * (43) Zhengyi Wang, Yikai Wang, Yifei Chen, Chendong Xiang, Shuo Chen, Dajiang Yu, Chongxuan Li, Hang Su, and Jun Zhu.  Crm: Single image to 3d textured mesh with convolutional reconstruction model.  arXiv preprint, 2024.
  * (44) Zhou Wang, Alan C Bovik, Hamid R Sheikh, and Eero P Simoncelli.  Image quality assessment: from error visibility to structural similarity.  In IEEE TIP, 2004.
  * (45) Guanjun Wu, Taoran Yi, Jiemin Fang, Lingxi Xie, Xiaopeng Zhang, Wei Wei, Wenyu Liu, Qi Tian, and Xinggang Wang.  4d gaussian splatting for real-time dynamic scene rendering.  In CVPR, 2024.
  * (46) Tong Wu, Jiarui Zhang, Xiao Fu, Yuxin Wang, Liang Pan Jiawei Ren, Wayne Wu, Lei Yang, Jiaqi Wang, Chen Qian, Dahua Lin, and Ziwei Liu.  Omniobject3d: Large-vocabulary 3d object dataset for realistic perception, reconstruction and generation.  In CVPR, 2023.
  * (47) Ziyi Yang, Xinyu Gao, Wen Zhou, Shaohui Jiao, Yuqing Zhang, and Xiaogang Jin.  Deformable 3d gaussians for high-fidelity monocular dynamic scene reconstruction.  In CVPR, 2024.
  * (48) Taoran Yi, Jiemin Fang, Guanjun Wu, Lingxi Xie, Xiaopeng Zhang, Wenyu Liu, Qi Tian, and Xinggang Wang.  Gaussiandreamer: Fast generation from text to 3d gaussian splatting with point cloud priors.  In CVPR, 2024.
  * (49) Yuyang Yin, Dejia Xu, Zhangyang Wang, Yao Zhao, and Yunchao Wei.  4dgen: Grounded 4d content generation with spatial-temporal consistency.  arXiv preprint, 2023.
  * (50) Xianggang Yu, Mutian Xu, Yidan Zhang, Haolin Liu, Chongjie Ye, Yushuang Wu, Zizheng Yan, Chenming Zhu, Zhangyang Xiong, Tianyou Liang, et al.  Mvimgnet: A large-scale dataset of multi-view images.  In CVPR, 2023.
  * (51) Lvmin Zhang, Anyi Rao, and Maneesh Agrawala.  Adding conditional control to text-to-image diffusion models.  In ICCV, 2023.
  * (52) Richard Zhang, Phillip Isola, Alexei A Efros, Eli Shechtman, and Oliver Wang.  The unreasonable effectiveness of deep features as a perceptual metric.  In CVPR, 2018.
  * (53) Yuyang Zhao, Zhiwen Yan, Enze Xie, Lanqing Hong, Zhenguo Li, and Gim Hee Lee.  Animate124: Animating one image to 4d dynamic scene.  arXiv preprint, 2023.
  * (54) Qi Zuo, Xiaodong Gu, Lingteng Qiu, Yuan Dong, Zhengyi Zhao, Weihao Yuan, Rui Peng, Siyu Zhu, Zilong Dong, Liefeng Bo, et al.  Videomv: Consistent multi-view generation based on large video generative model.  arXiv preprint, 2024.


* _Hut et al._ (1987) Hut P., Alvarez W., Elder W. P. et al. (1987) _Comet showers as a cause of mass extinctions_ , _Nature_ , _329_ , 118–126.
  * _Ivezić et al._ (2019) Ivezić et al. Ž. (2019) _LSST: From Science Drivers to Reference Design and Anticipated Data Products_ , _ApJ_ , _873_ , 111\.
  * _Jewitt_ (2009) Jewitt D. (2009) _The Active Centaurs_ , _AJ_ , _137_ , 4296–4312.
  * _Joss_ (1973) Joss P. C. (1973) _On the Origin of Short-period Comets_ , _A &A_, _25_ , 271.
  * _Kaib et al._ (2009) Kaib N. A., Becker A. C., Jones R. L. et al. (2009) _2006 SQ 372: A Likely Long-Period Comet from the Inner Oort Cloud_, _ApJ_ , _695_ , 268–275.
  * _Kaib et al._ (2019) Kaib N. A., Pike R., Lawler S. et al. (2019) _OSSOS. XV. Probing the Distant Solar System with Observed Scattering TNOs_ , _AJ_ , _158_ , 43.
  * _Kaib and Quinn_ (2008) Kaib N. A. and Quinn T. (2008) _The formation of the Oort cloud in open cluster environments_ , _Icarus_ , _197_ , 221–238.
  * _Kaib and Quinn_ (2009) Kaib N. A. and Quinn T. (2009) _Reassessing the Source of Long-Period Comets_ , _Science_ , _325_ , 1234.
  * _Kaib et al._ (2011) Kaib N. A., Roškar R., and Quinn T. (2011) _Sedna and the Oort Cloud around a migrating Sun_ , _Icarus_ , _215_ , 491–507.
  * _Kaib and Sheppard_ (2016) Kaib N. A. and Sheppard S. S. (2016) _Tracking Neptune’s Migration History through High-perihelion Resonant Trans-Neptunian Objects_ , _AJ_ , _152_ , 133.
  * _Kavelaars et al._ (2021) Kavelaars J., Petit J.-M., Gladman B. et al. (2021) _OSSOS finds an Exponential Cutoff in the Size Distribution of the Cold Classical Kuiper belt_ , _arXiv_ , arXiv:2107.06120.
  * _Kenyon and Bromley_ (2004) Kenyon S. J. and Bromley B. C. (2004) _Stellar encounters as the origin of distant Solar System objects in highly eccentric orbits_ , _Nature_ , _432_ , 598–602.
  * _Kowal and Gehrels_ (1977) Kowal C. T. and Gehrels T. (1977) _Slow-Moving Object Kowal_ , _IAUC_ , _3129_ , 1.
  * _Kozai_ (1962) Kozai Y. (1962) _Secular perturbations of asteroids with high inclination and eccentricity_ , _AJ_ , _67_ , 591–598.
  * _Kresák_ (1972) Kresák L. (1972) _Jacobian integral as a classificational and evolutionary parameter of interplanetary bodies_ , _Bulletin of the Astronomical Institutes of Czechoslovakia_ , _23_ , 1.
  * _Kresak and Pittich_ (1978) Kresak L. and Pittich E. M. (1978) _The Intrinsic Number Density of Active Long-period Comets in the Inner Solar System_ , _Bulletin of the Astronomical Institutes of Czechoslovakia_ , _29_ , 299.
  * _Kronk_ (1999) Kronk G. W. (1999) _Cometography: A Catalog of Comets, Volume 1: Ancient-1799_.
  * _Kuiper_ (1951) Kuiper G. P. (1951) _On the Origin of the Solar System_ , _Proceedings of the National Academy of Science_ , _37_ , 1–14.
  * _Lada and Lada_ (2003) Lada C. J. and Lada E. A. (2003) _Embedded Clusters in Molecular Clouds_ , _ARA &A_, _41_ , 57–115.
  * _Lawler et al._ (2018a) Lawler S. M., Kavelaars J. J., Alexandersen M. et al. (2018a) _OSSOS: X. How to use a Survey Simulator: Statistical Testing of Dynamical Models Against the Real Kuiper Belt_ , _FrASS_ , _5_ , 14.
  * _Lawler et al._ (2019) Lawler S. M., Pike R. E., Kaib N. et al. (2019) _OSSOS. XIII. Fossilized Resonant Dropouts Tentatively Confirm Neptune’s Migration Was Grainy and Slow_ , _AJ_ , _157_ , 253.
  * _Lawler et al._ (2018b) Lawler S. M., Shankman C., Kavelaars J. J. et al. (2018b) _OSSOS. VIII. The Transition between Two Size Distribution Slopes in the Scattering Disk_ , _AJ_ , _155_ , 197.
  * _Lehner et al._ (2012) Lehner M. J., Wang S.-Y., Alcock C. A. et al. (2012) in _Ground-based and Airborne Telescopes IV_ (L. M. Stepp, R. Gilmozzi, and H. J. Hall, eds.), vol. 8444 of _Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series_ , p. 84440D.
  * _Levison_ (1996) Levison H. F. (1996) in _Completing the Inventory of the Solar System_ (T. Rettig and J. M. Hahn, eds.), vol. 107 of _Astronomical Society of the Pacific Conference Series_ , pp. 173–191.
  * _Levison et al._ (2001) Levison H. F., Dones L., and Duncan M. J. (2001) _The Origin of Halley-Type Comets: Probing the Inner Oort Cloud_ , _AJ_ , _121_ , 2253–2267.
  * _Levison and Duncan_ (1994) Levison H. F. and Duncan M. J. (1994) _The Long-Term Dynamical Behavior of Short-Period Comets_ , _Icarus_ , _108_ , 18–36.
  * _Levison and Duncan_ (1997) Levison H. F. and Duncan M. J. (1997) _From the Kuiper Belt to Jupiter-Family Comets: The Spatial Distribution of Ecliptic Comets_ , _Icarus_ , _127_ , 13–32.
  * _Levison et al._ (2010) Levison H. F., Duncan M. J., Brasser R. et al. (2010) _Capture of the Sun’s Oort Cloud from Stars in Its Birth Cluster_ , _Science_ , _329_ , 187–190.
  * _Levison et al._ (2006a) Levison H. F., Duncan M. J., Dones L. et al. (2006a) _The scattered disk as a source of Halley-type comets_ , _Icarus_ , _184_ , 619–633.
  * _Levison et al._ (2008) Levison H. F., Morbidelli A., Van Laerhoven C. et al. (2008) _Origin of the structure of the Kuiper belt during a dynamical instability in the orbits of Uranus and Neptune_ , _Icar_ , _196_ , 258–273.
  * _Levison et al._ (2006b) Levison H. F., Terrell D., Wiegert P. A. et al. (2006b) _On the origin of the unusual orbit of Comet 2P/Encke_ , _Icar_ , _182_ , 161–168.
  * _Lidov_ (1962) Lidov M. L. (1962) _The evolution of orbits of artificial satellites of planets under the action of gravitational perturbations of external bodies_ , _Planet. Space Sci._ , _9_ , 719–759.
  * _Lowry et al._ (2008) Lowry S., Fitzsimmons A., Lamy P. et al. (2008) _Kuiper Belt Objects in the Planetary Region: The Jupiter-Family Comets_ , p. 397.
  * _Luu et al._ (1997) Luu J., Marsden B. G., Jewitt D. et al. (1997) _A new dynamical class of object in the outer Solar System_ , _Natur_ , _387_ , 573–575.
  * _Lykawka and Mukai_ (2007) Lykawka P. S. and Mukai T. (2007) _Resonance sticking in the scattered disk_ , _Icar_ , _192_ , 238–247.
  * _Malhotra_ (1993) Malhotra R. (1993) _The origin of Pluto’s peculiar orbit_ , _Natur_ , _365_ , 819–821.
  * _Malhotra_ (1995) Malhotra R. (1995) _The Origin of Pluto’s Orbit: Implications for the Solar System Beyond Neptune_ , _AJ_ , _110_ , 420.
  * _Marsden_ (2009) Marsden B. G. (2009) _Orbital properties of Jupiter-family comets_ , _Planet. Space Sci._ , _57_ , 1098–1105.
  * _Marsden and Williams_ (2008) Marsden B. G. and Williams G. V. (2008) _, Catalogue of Cometary Orbits 2008 (17th ed.; Cambridge, MA: IAU Minor Planet Center/Central Bureau for Astronomical Telegrams)_.
  * _Martínez-Barbosa et al._ (2017) Martínez-Barbosa C. A., Jílková L., Portegies Zwart S. et al. (2017) _The rate of stellar encounters along a migrating orbit of the Sun_ , _MNRAS_ , _464_ , 2290–2300.
  * _Matese et al._ (1995) Matese J. J., Whitman P. G., Innanen K. A. et al. (1995) _Periodic modulation of the Oort cloud comet flux by the adiabatically changing galactic tide._ , _Icarus_ , _116_ , 255–268.
  * _McKinnon et al._ (2020) McKinnon W. B., Richardson D. C., Marohnic J. C. et al. (2020) _The solar nebula origin of (486958) Arrokoth, a primordial contact binary in the Kuiper Belt_ , _Sci_ , _367_ , aay6620.
  * _Morbidelli and Levison_ (2004) Morbidelli A. and Levison H. F. (2004) _Scenarios for the Origin of the Orbits of the Trans-Neptunian Objects 2000 CR 105 and 2003 VB12 (Sedna)_, _AJ_ , _128_ , 2564–2576.
  * _Morbidelli et al._ (2021) Morbidelli A., Nesvorny D., Bottke W. F. et al. (2021) _A re-assessment of the Kuiper belt size distribution for sub-kilometer objects, revealing collisional equilibrium at small sizes_ , _Icarus_ , _356_ , 114256.
  * _Morris and Muller_ (1986) Morris D. E. and Muller R. A. (1986) _Tidal gravitational forces: The infall of “new” comets and comet showers_ , _Icarus_ , _65_ , 1–12.
  * _Murray and Dermott_ (1999) Murray C. D. and Dermott S. F. (1999) _Solar system dynamics_.
  * _Murray-Clay and Chiang_ (2005) Murray-Clay R. A. and Chiang E. I. (2005) _A Signature of Planetary Migration: The Origin of Asymmetric Capture in the 2:1 Resonance_ , _ApJ_ , _619_ , 623–638.
  * _Napier et al._ (2021) Napier K. J., Gerdes D. W., Lin H. W. et al. (2021) _No Evidence for Orbital Clustering in the Extreme Trans-Neptunian Objects_ , _PSJ_ , _2_ , 59.
  * _Nesvorný_ (2011) Nesvorný D. (2011) _Young Solar System’s Fifth Giant Planet?_ , _ApJ_ , _742_ , L22.
  * _Nesvorný_ (2015a) Nesvorný D. (2015a) _Evidence for Slow Migration of Neptune from the Inclination Distribution of Kuiper Belt Objects_ , _AJ_ , _150_ , 73.
  * _Nesvorný_ (2015b) Nesvorný D. (2015b) _Jumping Neptune Can Explain the Kuiper Belt Kernel_ , _AJ_ , _150_ , 68.
  * _Nesvorný_ (2018) Nesvorný D. (2018) _Dynamical Evolution of the Early Solar System_ , _ARA &A_, _56_ , 137–174.
  * _Nesvorný and Morbidelli_ (2012) Nesvorný D. and Morbidelli A. (2012) _Statistical Study of the Early Solar System’s Instability with Four, Five, and Six Giant Planets_ , _AJ_ , _144_ , 117.
  * _Nesvorný and Vokrouhlický_ (2016) Nesvorný D. and Vokrouhlický D. (2016) _Neptune’s Orbital Migration Was Grainy, Not Smooth_ , _ApJ_ , _825_ , 94.
  * _Nesvorny et al._ (2017) Nesvorny D., Vokrouhlicky D., Dones H. C. L. et al. (2017) in _AAS/Division for Planetary Sciences Meeting Abstracts #49_ , vol. 49 of _AAS/Division for Planetary Sciences Meeting Abstracts_ , p. 401.01.
  * _Nesvorný et al._ (2017) Nesvorný D., Vokrouhlický D., Dones L. et al. (2017) _Origin and Evolution of Short-period Comets_ , _ApJ_ , _845_ , 27\.
  * _Nesvorný et al._ (2016) Nesvorný D., Vokrouhlický D., and Roig F. (2016) _The Orbital Distribution of Trans-Neptunian Objects Beyond 50 au_ , _ApJL_ , _827_ , L35.
  * _Nihei et al._ (2007) Nihei T. C., Lehner M. J., Bianco F. B. et al. (2007) _Detectability of Occultations of Stars by Objects in the Kuiper Belt and Oort Cloud_ , _AJ_ , _134_ , 1596–1612.
  * _Noll et al._ (2020) Noll K., Grundy W. M., Nesvorný D. et al. (2020) _Trans-Neptunian binaries (2018)_ , pp. 201–224.
  * _Oikawa and Everhart_ (1979) Oikawa S. and Everhart E. (1979) _Past and future orbit of 1977 UB, object Chiron._ , _AJ_ , _84_ , 134–139.
  * _Oort_ (1950) Oort J. H. (1950) _The structure of the cloud of comets surrounding the Solar System and a hypothesis concerning its origin_ , _Bull. Astron. Inst. Netherlands_ , _11_ , 91–110.
  * _Öpik_ (1932) Öpik E. (1932) _Note on Stellar Perturbations of Nearly Parabolic Orbits_ , _Proceedings of the American Academy of Arts and Sciences_ , _67_ , 169.
  * _Opik_ (1976) Opik E. J. (1976) _Interplanetary encounters : close-range gravitational interactions_.
  * _Ortiz et al._ (2020) Ortiz J. L., Sicardy B., Camargo J. I. B. et al. (2020) _Stellar occultation by TNOs: from predictions to observations_ , pp. 413–437.
  * _Peixinho et al._ (2020) Peixinho N., Thirouin A., Tegler S. C. et al. (2020) _From Centaurs to Comets - 40 years_ , pp. 307–329.
  * _Petit et al._ (2011) Petit J. M., Kavelaars J. J., Gladman B. J. et al. (2011) _The Canada-France Ecliptic Plane Survey—Full Data Release: The Orbital Structure of the Kuiper Belt_ , _AJ_ , _142_ , 131.
  * _Pike et al._ (2017) Pike R. E., Fraser W. C., Schwamb M. E. et al. (2017) _Col-OSSOS: z-Band Photometry Reveals Three Distinct TNO Surface Types_ , _AJ_ , _154_ , 101.
  * _Pike et al._ (2015) Pike R. E., Kavelaars J. J., Petit J. M. et al. (2015) _The 5:1 Neptune Resonance as Probed by CFEPS: Dynamics and Population_ , _AJ_ , _149_ , 202.
  * _Quinn et al._ (1990) Quinn T., Tremaine S., and Duncan M. (1990) _Planetary Perturbations and the Origin of Short-Period Comets_ , _ApJ_ , _355_ , 667.
  * _Rickman_ (1976) Rickman H. (1976) _Stellar Perturbations of Orbits of Long-period Comets and their Significance for Cometary Capture_ , _Bulletin of the Astronomical Institutes of Czechoslovakia_ , _27_ , 92.
  * _Rickman et al._ (1990) Rickman H., Fernandez J. A., and Gustafson B. A. S. (1990) _Formation of stable dust mantles on short-period comet nuclei_ , _A &A_, _237_ , 524–535.
  * _Rickman et al._ (2008) Rickman H., Fouchard M., Froeschlé C. et al. (2008) _Injection of Oort Cloud comets: the fundamental role of stellar perturbations_ , _Celestial Mechanics and Dynamical Astronomy_ , _102_ , 111–132.
  * _Rickman et al._ (2012) Rickman H., Fouchard M., Froeschlé C. et al. (2012) _Gaia and the new comets from the Oort cloud_ , _Planet. Space Sci._ , _73_ , 124–129.
  * _Roškar et al._ (2008) Roškar R., Debattista V. P., Quinn T. R. et al. (2008) _Riding the Spiral Waves: Implications of Stellar Migration for the Properties of Galactic Disks_ , _ApJ_ , _684_ , L79.
  * _Sarid et al._ (2019) Sarid G., Volk K., Steckloff J. K. et al. (2019) _29P/Schwassmann-Wachmann 1, A Centaur in the Gateway to the Jupiter-family Comets_ , _ApJL_ , _883_ , L25.
  * _Schlichting et al._ (2012) Schlichting H. E., Ofek E. O., Sari R. et al. (2012) _Measuring the Abundance of Sub-kilometer-sized Kuiper Belt Objects Using Stellar Occultations_ , _ApJ_ , _761_ , 150.
  * _Schlichting et al._ (2009) Schlichting H. E., Ofek E. O., Wenz M. et al. (2009) _A single sub-kilometre Kuiper belt object from a stellar occultation in archival data_ , _Nature_ , _462_ , 895–897.
  * _Scholl_ (1979) Scholl H. (1979) _History and evolution of Chiron’s orbit_ , _Icar_ , _40_ , 345–349.
  * _Schwamb et al._ (2019) Schwamb M. E., Fraser W. C., Bannister M. T. et al. (2019) _Col-OSSOS: The Colors of the Outer Solar System Origins Survey_ , _ApJS_ , _243_ , 12.
  * _Sellwood and Binney_ (2002) Sellwood J. A. and Binney J. J. (2002) _Radial mixing in galactic discs_ , _MNRAS_ , _336_ , 785–796.
  * _Shankman et al._ (2013) Shankman C., Gladman B. J., Kaib N. et al. (2013) _A Possible Divot in the Size Distribution of the Kuiper Belt’s Scattering Objects_ , _ApJL_ , _764_ , L2.
  * _Shankman et al._ (2016) Shankman C., Kavelaars J., Gladman B. J. et al. (2016) _OSSOS. II. A Sharp Transition in the Absolute Magnitude Distribution of the Kuiper Belt’s Scattering Population_ , _AJ_ , _151_ , 31.
  * _Shankman et al._ (2017a) Shankman C., Kavelaars J. J., Bannister M. T. et al. (2017a) _OSSOS. VI. Striking Biases in the Detection of Large Semimajor Axis Trans-Neptunian Objects_ , _AJ_ , _154_ , 50.
  * _Shankman et al._ (2017b) Shankman C., Kavelaars J. J., Lawler S. M. et al. (2017b) _Consequences of a Distant Massive Planet on the Large Semimajor Axis Trans-Neptunian Objects_ , _AJ_ , _153_ , 63.
  * _Sheppard et al._ (2019) Sheppard S. S., Trujillo C. A., Tholen D. J. et al. (2019) _A New High Perihelion Trans-Plutonian Inner Oort Cloud Object: 2015 TG387_ , _AJ_ , _157_ , 139.
  * _Shoemaker and Wolfe_ (1984) Shoemaker E. M. and Wolfe R. F. (1984) in _Lunar and Planetary Science Conference_ , Lunar and Planetary Science Conference, pp. 780–781.
  * _Silsbee and Tremaine_ (2018) Silsbee K. and Tremaine S. (2018) _Producing Distant Planets by Mutual Scattering of Planetary Embryos_ , _AJ_ , _155_ , 75.
  * _Singer et al._ (2019) Singer K. N., McKinnon W. B., Gladman B. et al. (2019) _Impact craters on Pluto and Charon indicate a deficit of small Kuiper belt objects_ , _Sci_ , _363_ , 955–959.
  * _Smullen and Volk_ (2020) Smullen R. A. and Volk K. (2020) _Machine learning classification of Kuiper belt populations_ , _MNRAS_ , _497_ , 1391–1403.
  * _Snodgrass et al._ (2011) Snodgrass C., Fitzsimmons A., Lowry S. C. et al. (2011) _The size distribution of Jupiter Family comet nuclei_ , _MNRAS_ , _414_ , 458–469.
  * _Sosa and Fernández_ (2011) Sosa A. and Fernández J. A. (2011) _Masses of long-period comets derived from non-gravitational effects - analysis of the computed results and the consistency and reliability of the non-gravitational parameters_ , _MNRAS_ , _416_ , 767–782.
  * _Spencer et al._ (2020) Spencer J., Grundy W. M., Nimmo F. et al. (2020) _The Pluto system after New Horizons_ , pp. 271–288.
  * _Stern and Campins_ (1996) Stern A. and Campins H. (1996) _Chiron and the Centaurs: escapees from the Kuiper belt_ , _Natur_ , _382_ , 507–510.
  * _Tegler et al._ (2003) Tegler S. C., Romanishin W., and Consolmagno G. J. (2003) _Color Patterns in the Kuiper Belt: A Possible Primordial Origin_ , _ApJL_ , _599_ , L49–L52.
  * _Tiscareno and Malhotra_ (2003) Tiscareno M. S. and Malhotra R. (2003) _The Dynamics of Known Centaurs_ , _AJ_ , _126_ , 3122–3131.
  * _Torbett_ (1989) Torbett M. V. (1989) _Chaotic motion in a comet disk beyong Neptune : the delivery of short-period comets._ , _AJ_ , _98_ , 1477–1481.
  * _Trujillo and Sheppard_ (2014) Trujillo C. A. and Sheppard S. S. (2014) _A Sedna-like body with a perihelion of 80 astronomical units_ , _Nature_ , _507_ , 471–474.
  * _Tsiganis et al._ (2005) Tsiganis K., Gomes R., Morbidelli A. et al. (2005) _Origin of the orbital architecture of the giant planets of the Solar System_ , _Natur_ , _435_ , 459–461.
  * _Vokrouhlický et al._ (2019) Vokrouhlický D., Nesvorný D., and Dones L. (2019) _Origin and Evolution of Long-period Comets_ , _AJ_ , _157_ , 181.
  * _Volk and Malhotra_ (2008) Volk K. and Malhotra R. (2008) _The Scattered Disk as the Source of the Jupiter Family Comets_ , _ApJ_ , _687_ , 714–725.
  * _Volk et al._ (2018) Volk K., Murray-Clay R. A., Gladman B. J. et al. (2018) _OSSOS. IX. Two Objects in Neptune’s 9:1 Resonance—Implications for Resonance Sticking in the Scattering Population_ , _AJ_ , _155_ , 260.
  * _Weissman_ (1980) Weissman P. R. (1980) _Physical loss of long-period comets_ , _A &A_, _85_ , 191–196.
  * _Weissman_ (1996) Weissman P. R. (1996) in _Completing the Inventory of the Solar System_ (T. Rettig and J. M. Hahn, eds.), vol. 107 of _Astronomical Society of the Pacific Conference Series_ , pp. 265–288.
  * _Zhang et al._ (2013) Zhang Z. W., Lehner M. J., Wang J. H. et al. (2013) _The TAOS Project: Results from Seven Years of Survey Data_ , _AJ_ , _146_ , 14\.
  * _Zheng et al._ (1990) Zheng J.-Q., Valtonen M. J., and Valtaoja L. (1990) _Capture of Comets during the Evolution of a Star Cluster and the Origin of the Oort Cloud_ , _Celestial Mechanics and Dynamical Astronomy_ , _49_ , 265–272.