#  Conditional Diffusion with Less Explicit Guidance
via Model Predictive Control

Max W. Shen , Ehsan Hajiramezanali,
Gabriele Scalia, Alex Tseng, Nathaniel Diamant, Tommaso Biancalani, Andreas
Loukas
Genentech
<EMAIL_ADDRESS>Corresponding author<EMAIL_ADDRESS>
###### Abstract

How much explicit guidance is necessary for conditional diffusion? We consider
the problem of conditional sampling using an unconditional diffusion model and
limited explicit guidance (e.g., a noised classifier, or a conditional
diffusion model) that is restricted to a small number of time steps. We
explore a model predictive control (MPC)-like approach to approximate guidance
by simulating unconditional diffusion forward, and backpropagating explicit
guidance feedback. MPC-approximated guides have high cosine similarity to real
guides, even over large simulation distances. Adding MPC steps improves
generative quality when explicit guidance is limited to five time steps.

##  1 Introduction

Diffusion models are a class of generative models that have achieved
remarkable sample quality, particularly for text-to-image generation [1],
where diffusion has been guided using classifier guidance or classifier-free
guidance to sample images $\mathbf{x}\sim p(\mathbf{x}|\mathbf{c})$ for a
conditioning variable $\mathbf{c}$ (e.g., text) [2, 3]. Controlling generative
models is important for applications such as text generation and drug
discovery, where multiple distinct conditional variables
$\mathbf{c}_{1},\mathbf{c}_{2},...\mathbf{c}_{n}$ can be important: e.g., drug
activity and permeability [4].

For each new conditioning information source $\mathbf{c}$ of interest,
classifier guidance and classifier-free guidance require training a new
explicit guidance model over all diffusion time steps $t\in[0,...,T]$ (often,
$T=$100 to 1,000), and sample using the explicit guide at each generative time
step (often, 25-100) [2, 3]. Here, we explore whether conditional sampling is
achievable without explicit guidance at every generative step, and if it is
achievable with very few steps. This line of inquiry may make it easier to
condition on new variables by reducing the training burden of new explicit
guidance models.

Rejection sampling and Langevin "churning" have been explored for image
editing, inpainting, and conditional sampling on new variables without
training a new model over diffusion time steps, but lack general applicability
[5, 1, 6, 7, 8, 9, 10]: churning appears limited to "local" edits, while
rejection sampling is inefficient for rare events. Separately, scheduler
advances have reduced sampling steps from 100-1000 to 25-50 while retaining
high sample quality [11, 12]. This work aims to be generally applicable and
synergistic with scheduler improvements.

#### Diffusion models.

Diffusion models are trained on noise-corrupted data, and learn an iterative
denoising process to generate samples. We give a non-precise introduction
following [13], and refer interested readers to [11] for a precise
description. A diffusion model $\hat{\mathbf{x}}_{\theta}$ is trained to
optimize:

$\operatorname*{\mathbb{E}}_{\mathbf{x},\mathbf{c},\boldsymbol{\epsilon},t}[w_{t}\|\hat{\mathbf{x}}_{\theta}(\alpha_{t}\mathbf{x}+\sigma_{t}\boldsymbol{\epsilon},\mathbf{c})-\mathbf{x}\|_{2}^{2}]$
(1)

where $(\mathbf{x},\mathbf{c})$ are data-conditioning pairs,
$t\sim\mathcal{U}([0,1])$, $\bm{\epsilon}\sim\mathcal{N}(\mathbf{0,I})$, and
$\alpha_{t},\sigma_{t}$, and $w_{t}$ are time-varying weights that influence
sample quality. In the $\bm{\epsilon}$-prediction parameterization,
$\hat{\mathbf{x}}_{\theta}(\mathbf{z}_{t},\mathbf{c})=(\mathbf{z}_{t}-\sigma_{t}\bm{\epsilon}_{\theta}(\mathbf{z}_{t},\mathbf{c}))/\alpha_{t}$
where $\bm{\epsilon}_{\theta}$ is the learned function. Notably, this training
procedure has an expectation over $t$, which can be hundreds to thousands of
time steps.

To sample, a simple scheduler starts at
$\mathbf{z}_{T}\sim\mathcal{N}(\mathbf{0,I})$ and iteratively generates
$\mathbf{z}_{t-1}=(\mathbf{z}_{t}-\sigma\tilde{\bm{\epsilon}}_{\theta})/\alpha_{t}$
where the choice of $\tilde{\bm{\epsilon}}_{\theta}$ distinguishes sampling
strategies. In general, schedulers can jump to $\mathbf{z}_{t-\Delta}$ as a
function of starting time $t$, jump size $\Delta$, latent $\mathbf{z}_{t}$,
and predicted noise $\tilde{\bm{\epsilon}}_{\theta}$.

#### Diffusion guidance.

Classifier guidance [2] requires training a noised classifier
$p_{t}(\mathbf{c}|\mathbf{z}_{t})$ over $T$ time steps, and uses
$\tilde{\bm{\epsilon}}_{\theta}=\bm{\epsilon}(\mathbf{z}_{t},\mathbf{c})-\nabla_{\mathbf{z}_{t}}\log
p_{t}(\mathbf{c}|\mathbf{z}_{t})$. Notably, pre-trained clean-data classifiers
cannot be directly used for guidance. Classifier-free guidance [3] learns both
a conditional and unconditional diffusion model by setting
$\mathbf{c}=\mathbf{0}$ with $10\%$ probability during training;
$\tilde{\bm{\epsilon}}_{\theta}=\bm{\epsilon}_{\theta}(\mathbf{z}_{t}):=\bm{\epsilon}_{\theta}(\mathbf{z}_{t},\mathbf{c}=\mathbf{0})$
achieves unconditional sampling. Classifier-free guidance with weight $w$ uses

$\tilde{\bm{\epsilon}}_{\theta}=(1+w)\bm{\epsilon}(\mathbf{z}_{t},\mathbf{c})-w\bm{\epsilon}_{\bm{\theta}}(\mathbf{z}_{t}).$
(2)

#### Model predictive control (MPC).

Model predictive control aims at controlling a time-evolving system in an
optimized manner, by using a predictive dynamics model of the system and
solving an optimization problem online to obtain a sequence of control
actions. Typically, the first control action is applied at the current time,
then the optimization problem is solved again to act at the next time step
[14]. The general formalized MPC problem is:

$\operatorname*{arg\,min}_{\mathbf{s}_{1:T},\mathbf{a}_{1:T}}\sum_{t=1}^{T}\ell_{t}(\mathbf{s}_{t},\mathbf{a}_{t})\textrm{
subject to
}\mathbf{s}_{t+1}=f(\mathbf{s}_{t},\mathbf{a}_{t});\mathbf{s}_{1}=\mathbf{s}_{\texttt{init}}$
(3)

where $\mathbf{s}_{t},\mathbf{a}_{t}$ are the state and control action at time
$t$, $\ell_{t}$ is a cost function, $f$ is a dynamics model, and
$\mathbf{s}_{\texttt{init}}$ is the initial state of the system. MPC can be
solved with gradient methods [15, 16].

##  2 Approximate conditional guidance via model predictive control

Our problem is performing conditional diffusion on a latent $\mathbf{z}_{t}$
with only access to an unconditional diffusion model. In particular, we do not
have an explicit conditional guide
$\bm{\epsilon}_{\bm{\theta}}(\mathbf{z}_{t},\mathbf{c})$ at time $t$; instead,
we can evaluate it only at $t-\delta$. Our method, MPC guidance, optimizes an
approximation
$\boldsymbol{\xi}_{t}\approx\bm{\epsilon}_{\bm{\theta}}(\mathbf{z}_{t},\mathbf{c})$,
which is used in classifier-free guidance (eq. 2) to apply one generative step
on $\mathbf{z}_{t}$ to obtain $\mathbf{z}_{t-\Delta}$. This can be applied
repeatedly to reach $\mathbf{z}_{0}$. In terms of MPC, we view
$\mathbf{z}_{t}$ as states, control actions as
$\tilde{\bm{\epsilon}}_{\theta}$, the dynamics model $f$ as the diffusion
generative process given $\mathbf{z}_{t}$ and
$\tilde{\bm{\epsilon}}_{\theta}$, and define loss $\ell$ at time $t-\delta$
using the explicit guide (Fig. 2).

#### Noised classifier.

With a noised classifier $p_{t-\delta}(\mathbf{c}|\mathbf{z}_{t})$, the
explicit guide
$\bm{\epsilon}_{\bm{\theta}}(\mathbf{z}_{t-\delta},\mathbf{c})=\nabla_{\mathbf{z}_{t-\delta}}\log
p_{t-\delta}(\mathbf{c}|\mathbf{z}_{t-\delta})$. We propose to unconditionally
generate $\mathbf{z}_{t-\delta}$ from $\mathbf{z}_{t}$ and evaluate $\log
p_{t-\delta}(\mathbf{c}|\mathbf{z}_{t-\delta})$ which we treat as "inverse
loss". Our MPC guide at time $t$ is a first-order, one-step optimization of
this loss:

$\boldsymbol{\xi}_{t}=-\nabla_{\mathbf{z}_{t}}\ell(\mathbf{z}_{t-\delta})=-\nabla_{\mathbf{z}_{t}}\log
p_{t-\delta}(\mathbf{c}|\mathbf{z}_{t-\delta})$ (4)

    def approx_guide(zt, t, dt, noised_classifier):
        z = denoise(zt, t, dt)    # differentiable; denoise zt to time t-dt
        return autograd(noised_classifier(z), zt)   # grad wrt zt

Algorithm 1 Approximate guide with noised classifier

#### Conditional diffusion model.

When the explicit guide is a conditional diffusion model
$\bm{\epsilon}_{\theta}(\mathbf{z}_{t-\delta},\mathbf{c})$, we denoise
$\mathbf{z}_{t}$ to $\mathbf{z}_{t-\delta}$ and construct the MPC guide as:

$\boldsymbol{\xi}_{t}=-\nabla_{\mathbf{z}_{t}}\ell(\mathbf{z}_{t-\delta})=-\nabla_{\mathbf{z}_{t}}\|\mathbf{z}_{t-\delta}-\mathbf{z}^{*}\|^{2}$
(5)

where gradients with respect to $\mathbf{z}_{t}$ are blocked for the target
$\mathbf{z}^{*}:=\mathbf{z}_{t-\delta}+\bm{\epsilon}_{\bm{\theta}}(\mathbf{z}_{t-\delta},\mathbf{c})$.

    def approx_guide(zt, t, dt, cond_score):
        z = denoise(zt, t, dt)    # differentiable; denoise zt to time t-dt
        with no_grad():
            target = z + cond_score(z, t-dt)
        loss = (z - target)**2
        return autograd(loss, zt)   # grad wrt zt

Algorithm 2 Approximate guide with conditional diffusion model

#### Backpropagation through diffusion.

To compute gradients with respect to $\mathbf{z}_{t}$, we must backpropagate
through unconditional diffusion. This incurs memory cost linear in the number
of denoising steps used. In practice, five to ten denoising steps enabled good
performance without memory issues.

##  3 Experiments

We perform experiments on Stable Diffusion [1], an open-source text-to-image
latent diffusion model trained on LAION-5B [17] with a pre-trained text
conditional and unconditional model. Latent diffusion occurs over 1000 time
steps: $\mathbf{z}_{0}\rightleftharpoons\mathbf{z}_{1000}$, and an
adversarially-trained autoencoder encodes and decodes
$\mathbf{x}\rightleftharpoons\mathbf{z}_{0}$. We treat the conditional
diffusion model as the explicit guide. We use the pseudo linear multi-step
(PLMS) scheduler [12] which is deterministic.

#### Approximate guides have high accuracy.

In figure 1, we compare our approximated guide $\boldsymbol{\xi}_{t}$ to
Stable Diffusion’s conditional guide
$\bm{\epsilon}_{\bm{\theta}}(\mathbf{z}_{t},\mathbf{c})$ using the cosine
similarity between the two gradients (see appendix for full details).
Approximate guides obtained by denoising $\mathbf{z}_{t}$ to
$\mathbf{z}_{t-\delta}$ are very similar to Stable Diffusion’s guide, with
cosine similarity above 0.99 even as $\delta$ increases to 500 time steps out
of 1000 total diffusion steps. At $\delta=900$, similarity is maintained above
0.80.

In contrast, approximate guides formed by denoising and decoding
$\mathbf{z}_{t}$ to images $\mathbf{x}$, applying CLIP [18] spherical loss,
and backpropagating back to $\mathbf{z}_{t}$ are essentially orthogonal to
$\bm{\epsilon}_{\bm{\theta}}(\mathbf{z}_{t},\mathbf{c})$, with mean similarity
around 0.01. This is consistent with observations that the manifold of natural
images is complex in pixel space, and gradients on images are difficult to use
for optimizing latents [19]. This highlights the challenge of conditional
diffusion sampling using only clean-data classifiers.

Figure 1: MPC guides have high cosine similarity to real guides

#### Approximate guides improve robustness to sample quality damage with
reduced explicit guidance.

We evaluated conditional sampling with explicit guidance restricted to just
$n=5$ time steps, with classifier weight $w=2$. We compare to using $k=3$
additional MPC-guided generative steps (with a total of $n+k=8$ steps), and a
reference with full explicit guidance on $n+k$ steps - if MPC is accurate,
then samples should look similar to the reference. We also generated gold
standard samples with 50 explicit guidance steps. We evaluated PLMS baselines
with both $n$ and $n+k$ generative steps (see appendix). Each approach was
initialized with identical $z_{T}$; as each approach is deterministic, quality
can be judged by similarity to the reference and gold standard.

On random MS-COCO prompts, adding MPC generative steps significantly improved
visual sample quality over the baseline (Fig. 2) and improve FID to the
reference and gold standard (Table 3). MPC samples are more visually similar
to the reference than the baseline, and intriguingly, in some cases seem to
outcompete the reference in visual similarity to the gold standard.

FID ($\downarrow$) | Reference | Gold standard
---|---|---
Baseline ($n=5$) | 400.0 | 443.28
\+ MPC ($k=3$) | 282.4 | 312.84
Figure 2: Comparison of samples (Stable Diffusion, pseudo linear multi-step
scheduler, guidance weight $w=2$)

##  4 Discussion

We described a method for approximating guidance for conditionally sampling
from diffusion models with model predictive control, and showed preliminary
evidence that approximated guidance improves sample quality when access to a
conditional guide is severely restricted to just five time steps.

Looking forward, future work may be interested in addressing instabilities and
divergence. In some settings, we found that approximate guides tended to cause
divergence to reference latent trajectories over time. We found this issue to
be particularly problematic with larger classifier guidance weights $w$: even
if $\boldsymbol{\xi}_{t}$ is very similar to
$\bm{\epsilon}(\mathbf{z}_{t},\mathbf{c})$ (e.g., 0.9999), and
$\bm{\epsilon}_{\bm{\theta}}(\mathbf{z}_{t})$ is identical, the adjusted
prediction
$\tilde{\bm{\epsilon}}(\mathbf{z}_{t},\mathbf{c})=(1+w)\boldsymbol{\xi}_{t}-w\bm{\epsilon}_{\bm{\theta}}(\mathbf{z}_{t})$
can have significantly lower similarity (e.g., 0.992). We also observed that
divergence increased with the number of approximate guidance steps.

Our results suggest the possibility of conditional diffusion using explicit
guidance (e.g., a conditional diffusion model) trained on a small number of
time steps. Future work can explore this by restricting conditional training;
here, we only restricted the time steps at which we queried the ground-truth
guide which was trained on all time steps.

## References

  * Rombach et al. [2021] Robin Rombach, Andreas Blattmann, Dominik Lorenz, Patrick Esser, and Björn Ommer.  High-resolution image synthesis with latent diffusion models, 2021.
  * Dhariwal and Nichol [2021] Prafulla Dhariwal and Alexander Quinn Nichol.  Diffusion models beat gans on image synthesis.  In A. Beygelzimer, Y. Dauphin, P. Liang, and J. Wortman Vaughan, editors, _Advances in Neural Information Processing Systems_ , 2021.
  * Ho and Salimans [2021] Jonathan Ho and Tim Salimans.  Classifier free diffusion guidance.  In _NeurIPS 2021 Workshop on Deep Generative Models and Downstream Applications_ , 2021.
  * Stanton et al. [2022] Samuel Stanton, Wesley Maddox, Nate Gruver, Phillip Maffettone, Emily Delaney, Peyton Greenside, and Andrew Gordon Wilson.  Accelerating bayesian optimization for biological sequence design with denoising autoencoders.  In _Proceedings of the 39th International Conference on Machine Learning_ , Proceedings of Machine Learning Research. PMLR, 17–23 Jul 2022.
  * Meng et al. [2022] Chenlin Meng, Yutong He, Yang Song, Jiaming Song, Jiajun Wu, Jun-Yan Zhu, and Stefano Ermon.  SDEdit: Guided image synthesis and editing with stochastic differential equations.  In _International Conference on Learning Representations_ , 2022.
  * Choi et al. [2021] Jooyoung Choi, Sungwon Kim, Yonghyun Jeong, Youngjune Gwon, and Sungroh Yoon.  Ilvr: Conditioning method for denoising diffusion probabilistic models.  In _ICCV_ , 2021.  doi: 10.48550/ARXIV.2108.02938.  URL https://arxiv.org/abs/2108.02938.
  * Singh et al. [2022] Vedant Singh, Surgan Jandial, Ayush Chopra, Siddharth Ramesh, Balaji Krishnamurthy, and Vineeth N. Balasubramanian.  On conditioning the input noise for controlled image generation with diffusion models, 2022.  URL https://arxiv.org/abs/2205.03859.
  * Chung et al. [2022] Hyungjin Chung, Byeongsu Sim, and Jong Chul Ye.  Come-closer-diffuse-faster: Accelerating conditional diffusion models for inverse problems through stochastic contraction.  In _CVPR_ , 2022.
  * Sinha et al. [2021] Abhishek Sinha, Jiaming Song, Chenlin Meng, and Stefano Ermon.  D2c: Diffusion-decoding models for few-shot conditional generation.  In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan, editors, _Advances in Neural Information Processing Systems_ , volume 34, pages 12533–12548. Curran Associates, Inc., 2021.  URL https://proceedings.neurips.cc/paper/2021/file/682e0e796084e163c5ca053dd8573b0c-Paper.pdf.
  * Lugmayr et al. [2022] Andreas Lugmayr, Martin Danelljan, Andres Romero, Fisher Yu, Radu Timofte, and Luc Van Gool.  Repaint: Inpainting using denoising diffusion probabilistic models, 2022\.  URL https://arxiv.org/abs/2201.09865.
  * Karras et al. [2022] Tero Karras, Miika Aittala, Timo Aila, and Samuli Laine.  Elucidating the design space of diffusion-based generative models, 2022\.  URL https://arxiv.org/abs/2206.00364.
  * Liu et al. [2022] Luping Liu, Yi Ren, Zhijie Lin, and Zhou Zhao.  Pseudo numerical methods for diffusion models on manifolds.  In _International Conference on Learning Representations_ , 2022.  URL https://openreview.net/forum?id=PlKWVd2yBkY.
  * Saharia et al. [2022] Chitwan Saharia, William Chan, Saurabh Saxena, Lala Li, Jay Whang, Emily Denton, Seyed Kamyar Seyed Ghasemipour, Burcu Karagol Ayan, S. Sara Mahdavi, Rapha Gontijo Lopes, Tim Salimans, Jonathan Ho, David J Fleet, and Mohammad Norouzi.  Photorealistic text-to-image diffusion models with deep language understanding, 2022.  URL https://arxiv.org/abs/2205.11487.
  * Amos et al. [2018] Brandon Amos, Ivan Jimenez, Jacob Sacks, Byron Boots, and J. Zico Kolter.  Differentiable mpc for end-to-end planning and control.  In S. Bengio, H. Wallach, H. Larochelle, K. Grauman, N. Cesa-Bianchi, and R. Garnett, editors, _Advances in Neural Information Processing Systems_ , volume 31. Curran Associates, Inc., 2018.  URL https://proceedings.neurips.cc/paper/2018/file/ba6d843eb4251a4526ce65d1807a9309-Paper.pdf.
  * Piche et al. [1999] Stephen Piche, James Keeler, Greg Martin, Gene Boe, Doug Johnson, and Mark Gerules.  Neural network based model predictive control.  In S. Solla, T. Leen, and K. Müller, editors, _Advances in Neural Information Processing Systems_ , volume 12. MIT Press, 1999.  URL https://proceedings.neurips.cc/paper/1999/file/db957c626a8cd7a27231adfbf51e20eb-Paper.pdf.
  * Bharadhwaj et al. [2020] Homanga Bharadhwaj, Kevin Xie, and Florian Shkurti.  Model-predictive control via cross-entropy and gradient-based optimization.  In Alexandre M. Bayen, Ali Jadbabaie, George Pappas, Pablo A. Parrilo, Benjamin Recht, Claire Tomlin, and Melanie Zeilinger, editors, _Proceedings of the 2nd Conference on Learning for Dynamics and Control_ , volume 120 of _Proceedings of Machine Learning Research_ , pages 277–286. PMLR, 10–11 Jun 2020.  URL https://proceedings.mlr.press/v120/bharadhwaj20a.html.
  * Schuhmann et al. [2022] Christoph Schuhmann, Romain Beaumont, Cade W Gordon, Ross Wightman, mehdi cherti, Theo Coombes, Aarush Katta, Clayton Mullis, Patrick Schramowski, Srivatsa R Kundurthy, Katherine Crowson, Mitchell Wortsman, Richard Vencu, Ludwig Schmidt, Robert Kaczmarczyk, and Jenia Jitsev.  LAION-5b: An open large-scale dataset for training next generation image-text models.  In _Thirty-sixth Conference on Neural Information Processing Systems Datasets and Benchmarks Track_ , 2022.  URL https://openreview.net/forum?id=M3Y74vmsMcY.
  * Radford et al. [2021] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever.  Learning transferable visual models from natural language supervision, 2021.  URL https://arxiv.org/abs/2103.00020.
  * Plumerault et al. [2020] Antoine Plumerault, Hervé Le Borgne, and Céline Hudelot.  Controlling generative models with continuous factors of variations.  In _International Conference on Learning Representations_ , 2020.  URL https://openreview.net/forum?id=H1laeJrKDB.
  * Bühlmann [2022] Matthias Bühlmann.  Stable diffusion based image compression, 2022.  URL https://matthias-buehlmann.medium.com/stable-diffusion-based-image-compresssion-6f1f0a399202.

##  Appendix A Appendix

###  A.1 Experiments

We used an Nvidia A100 with 80 GB memory for our experiments. Backpropagating
through diffusion requires backpropagating through Stable Diffusion’s U-Net
several times. We found that roughly 10 or more denoising steps exceeded the
memory of our A100, but that five denoising steps was sufficient for
performance.

We used classifier-free guidance weight $w=2$, following [3]. In practice, we
scale our approximate guide $\boldsymbol{\xi}_{t}$ at time $t$ to match the
norm of the unconditional score $\bm{\epsilon}_{\bm{\theta}}(\mathbf{z}_{t})$.

We will release our code in a future version.

#### Details on Stable Diffusion.

Stable Diffusion was trained with classifier-free guidance, conditioning on
CLIP-embedded text prompts [18], with $1000$ diffusion time steps. An
adversarially-trained autoencoder encodes and decodes images , which is an
$8\times$ down-sampled latent space. Latents $z_{0}$ were very weakly
regularized ($10^{-6}$ weight) towards a unit Gaussian. Despite this, when
visualized as images, latents $\mathbf{z}_{0}$ appear as fuzzy versions of the
decoded image $\mathbf{x}=\texttt{decoder}(\mathbf{z}_{0})$ [20].

#### Similarity study.

At each starting time $t$, we initialized $z_{t}$ by unconditionally denoising
from the prior $z_{T}$. We obtained an approximate guide for various $\delta$,
also called $dt$. Stable diffusion has 1000 total diffusion timesteps, so we
varied $t$ in $[200,400,600,800,1000]$. We varied $\delta$ in increments of
$100$, and performed 10 replicates for each experimental condition. We used
the following text prompts, some of which were from the Stable Diffusion paper
[1]: ’a photo of a cat’, ’a photo of an astronaut riding a horse on mars’, ’a
street sign that reads latent diffusion’, ’a zombie in the style of picasso’,
’a watercolor painting of a chair that looks like an octopus’, ’an
illustration of a slightly conscious neural network’. We observed similar
results for all prompts. The plot depicts data for $t=1000$, for varying
$\delta$ on the x-axis, across prompts and replicates: there are 60 datapoints
for each violin plot, which is smoothed with kernel density estimation using
seaborn.

#### Restricted explicit guidance experiments.

Our approach used an eight-step schedule evenly divided from $t=$1000 to 0:
[875, 750, 625, 500, 375, 250, 125, 0], with explicit guidance at times [750,
500, 250, 125, 0] and MPC at [875, 625, 375]. We compare to a reference with
the same eight-step schedule with full explicit guidance. Our PLMS baseline
uses the five-step schedule [800, 600, 400, 200, 0] with explicit guidance. We
also tried another baseline using the eight-step schedule, explicit guidance
at five time steps, and unconditional steps at times [875, 625, 375], but
found that this baseline ignored prompts.

#### Wall-clock time (for one sample).

50 generative steps takes about 12 seconds. 10 generative steps takes about
2.5 seconds. We find that with 5 generative steps, adding 3 MPC steps adds
negligible runtime, with all runs finishing in 1-3 seconds. In a separate
unreported experimental setting, our method, with 25 total generative
denoising steps, guidance at 10 time steps, 10 unconditional denoising steps
for approximating the guide, and churning, takes about 34 seconds. The same
setting, without churning, takes about 18 seconds.


$\psi(z_{i})$ is equivalent to a power series in $\varrho$ variables after a
linear homogeneous invertible transformation of the $z_{i}$.

If $\psi(z_{i})$ is of rank $\varrho$, then every divisor of $\psi(z_{i})$ is
also of rank $\varrho$. Since if we transform the variables $z_{i}$ into new
variables $\zeta_{i}$, every divisor of $\psi(z_{i})$ becomes a divisor of the
transformed power series; but if the transformed power series is equivalent to
one that only depends on $\varrho$ variables $\zeta_{i}$, then the same
applies to each of its divisors, according to the theorem about the uniqueness
of the decomposition of a power series into irreducible factors.

We now return to our question and choose a differential equation $\Phi(z_{i})$
formally satisfied by $y$, with $\Phi(z_{i})$ being of as small a rank
$\varrho$ as possible. If $\Phi(z_{i})=\Phi^{(1)}(z_{i})\Phi^{(2)}(z_{i})$,
where $\Phi^{(1)}$ and $\Phi^{(2)}$ are power series, then $y$ formally
satisfies one of the equations

$\Phi^{(1)}(z_{i})=0,\hskip 30.0pt\Phi^{(2)}(z_{i})=0.$

Therefore we can take $\Phi(z_{i})$ to be an irreducible power series of rank
$\varrho$. There thus exists a linear homogeneous invertible transformation

(61) $\displaystyle z_{i}=\sum_{ik}\zeta_{k},\hskip 30.0pt\zeta_{i}=\sum
s_{ik}z_{k},$

by which $\Phi(z_{i})$ is mapped to a power series
$\overline{\Phi}(\zeta_{i})$ to which the representation

$\Phi(z_{i})=\overline{\Phi}(\zeta_{i})=\Psi(\zeta_{i})E(\zeta_{i})$

applies, where $\Psi(\zeta_{i})$ only depends on $\varrho$ of the variables
$\zeta_{i}$, say $\zeta_{1},\ldots,\zeta_{\varrho}$, and $E$ does not vanish
at the origin. If $\frac{1}{E(\zeta_{i})}=e(z_{i})$, by replacing
$\Phi(z_{i})$ with $\Phi(z_{i})e(z_{i})$ if necessary, we can convert
$\Phi(z_{i})$ directly into a power series $\Psi(\zeta_{i})$ which only
depends on $\varrho$ variables $\zeta_{1},\ldots,\zeta_{\varrho}$, directly
using $(61)$. The differential equation $\Phi(z_{i})=0$ then has the desired
property. Since assuming this, $y$ also formally satisfies an equation
$\Phi_{1}(z_{i})=\frac{\partial\Phi(z_{i})}{\partial z_{i}}=0$ for instance.
From the identity:

$\Phi_{1}(z_{i})=\frac{\partial\Phi(z_{i})}{\partial z_{i}}=\sum
s_{k1}\frac{\partial\Psi(\zeta_{i}}{\partial\zeta_{k}}=\Psi^{*}(\zeta_{i})$

it follows that $\Phi_{1}(z_{i})$ becomes a power series $\Psi^{*}(\zeta_{i})$
in $\zeta_{1},\ldots,\zeta_{\varrho}$ using (61). We now apply another linear
homogeneous substitution to $\Psi(\zeta_{i})$ and $\Psi^{*}(\zeta_{i})$

(62) $\displaystyle\zeta_{i}=\sum\tau_{ik}u_{k},\hskip 30.0ptu_{i}=\sum
t_{ik}\zeta_{k},\hskip 30.0pt(i=1,\ldots\varrho;k=1,\ldots,\varrho),$

through which we achieve that Weierstrass’s preparation theorem becomes
applicable to both power series. Namely, we can achieve that both $\Psi$ and
$\Psi^{*}$, after the transformation, contain isolated powers of the same
variable, say $u_{1}$. Since if the sums of the terms of the lowest dimension
in $\Psi(\zeta_{i})$ and $\Psi^{*}(\zeta_{i})$ are $a(\zeta_{i})$ and
$b(\zeta_{i})$, we only need (62) to be set up in such a way that after the
transformation (62), all products of powers of $u_{i}$ that are permissible
according to dimension actually occur in the homogeneous forms $a$ and $b$ .
We can therefore write

$\displaystyle\Psi(\zeta_{i})=X(u_{i})$
$\displaystyle=E(u_{i})\Pi(u_{i})=E(u_{i})(u_{1}^{\mu}+A^{(1)}(u_{i})u_{1}^{\mu-1}+\ldots+A^{(\mu)}(u_{i}))$
$\displaystyle\Psi^{*}$
$\displaystyle=X^{*}(u_{i})=E^{*}(u_{i})\Pi^{*}(u_{i})=$
$\displaystyle=E^{*}(u_{i})(u_{1}^{\nu}+A^{*(1)}(u_{i})u_{1}^{\nu-1}+\ldots+A^{*(\nu)}(u_{i})).$

Here $E(u_{i}),E^{*}(u_{i})$ are power series that do not vanish at the
origin, $A^{(i)}$ and $A^{*(i)}$ however are power series only in
$u_{2},\ldots,u_{\varrho}$. If we now form the resultant
$P(u_{2},\ldots,u_{\varrho})$ from $\Pi$ and $\Pi^{*}$ with respect to
$u_{1}$, the following identity holds:

$P(u_{2},\ldots,u_{\varrho})=B(u_{i})\Pi(u_{i})+B^{*}(u_{i})\Pi^{*}(u_{i})$

or

(63) $\displaystyle
P(u_{2},\ldots,u_{\varrho})=C(u_{i})X(u_{i})+C^{*}(u_{i})X^{*}(u_{i}).$

If we now go back to the $\zeta_{i}$ and $z_{i}$,
$P(u_{2},\ldots,u_{\varrho})$ yields a power series $R(z_{i})$ whose rank is
at most $\varrho-1$, and for which (63) gives an identical relation of the
form

$R(z_{i})=D(z_{i})\Phi(z_{i})+D_{1}(z_{i})\Phi_{1}(z_{i}).$

From this it would follow, however, that $y$ also satisfies the equation
$R(z_{i})=0$ formally, while $R(z_{i})$ is of the rank $\varrho-1$. Thus the
above claim has been proven.

We can therefore assume that none of the equations (60) are formally
satisfied. Therefore, if one plugs the series expansion for $y$ into one of
the power series $\Phi_{i}(z)$ in (60), and sorts the result according to the
real parts of the exponents of the individual terms, which is possible
according to our assumptions, not all terms vanish identically, and the
expansion will start with terms in which the real parts of the exponents are
as small as possible, say $\nu_{i}$. Let these terms be:

(64) $\displaystyle c_{i,\kappa}x^{\lambda_{i,\kappa}}.$

From our premises it thus follows: If one truncates the series for $y$
somewhere sufficiently far along, the initial terms of the expansion of
$\Phi_{i}(z)$ do not change. The exponents $\lambda_{i,\kappa}$ are thus
homogeneous integer linear combinations of certain $n_{i}$. We now split the
series for $y$ into two parts

$y=A_{t}(x)+R_{t}(x)=\sum_{i=0}^{t-1}a_{i}x^{n_{i}}+\sum_{i=t}^{\infty}a_{i}x^{n_{i}}$

and take $t$ to be so large that starting with this $t$, when $A_{t}(x)$ is
substituted for $y$, the initial terms of the $\Phi_{i}$ are the sums of the
expressions (64). We now denote
$A_{t}(x),A_{t}^{\prime}(x),\ldots,A_{t}^{(n)}(x)$ as
$u_{2},u_{3},\ldots,u_{n+2},$ and
$R_{t}(x),R_{t}^{\prime}(x),\ldots,R_{t}^{(n)}(x)$ as
$v_{2},v_{3},\ldots,v_{n+2}$, where the indices are chosen in correspondance
with those of $z_{i}$. The $u_{i}$ and $v_{i}$ do still depend on $t$.

We want to expand $\Phi(z_{i})$ in terms of powers of $v_{i}$. For this
purpose we consider that as long as the real part of the initial term of
$y^{(i)}$ is not negative, $z_{i+2}=y^{(i)}-c_{i}=(u_{i+2}-c_{i})+v_{i+2}$, so
that we simply have to use Taylor’s theorem directly to expand into powers of
these $v_{i+2}$. If, on the other hand, the initial term $b_{i}x^{\mu_{i}}$ of
$y^{(i)}$ has an exponent with a negative real part, then

(65) $\displaystyle
z_{i+2}=\frac{1}{u_{i+2}+v_{i+2}}=\frac{1}{b_{i}}x^{-\mu_{i}}\frac{1}{\frac{u_{i+2}}{b_{i}x^{\mu_{i}}}+\frac{v_{i+2}}{b_{i}x^{\mu_{i}}}}=\frac{1}{b_{i}}x^{-\mu_{i}}\sum_{k=0}^{\infty}(\frac{u_{i+2}}{b_{i}x^{\mu_{i}}}-1+\frac{v_{i+2}}{b_{i}x^{\mu_{i}}})^{k}.$

Here $\frac{u_{i+2}}{b_{i}x^{\mu_{i}}}-1$ and
$\frac{v_{i+2}}{b_{i}x^{\mu_{i}}}$ have many terms with different exponents,
and the real parts of the exponents of all terms of
$\frac{v_{i+2}}{b_{i}x^{\mu_{i}}}$ are, as soon as $t$ is taken sufficiently
large, greater than the exponents of the terms of
$\frac{u_{i+2}}{b_{i}x^{\mu_{i}}}$. We finally note that the expansions of the
derivatives of $z_{i+2}$ in $v_{i+2}$, formally constructed from (65), for
$v_{i+2}=0$, have exponents whose real parts are not negative. The formula
(65) shows that one can use Taylor’s theorem when expanding $\Phi(z_{i})$ in
powers of $v_{i}$. For $i\geq 0$ we denote generally $u_{i+2}-c_{i}$ or
$\frac{1}{u_{i+2}}$ as $w_{i+2}$ (thus $w_{i+2}=(z_{i+2})_{v_{i+2}=0}$), then
the expansion

(66) $\displaystyle\Phi(z)=\Phi(x,w)+\sum
v_{i}(\frac{\partial\Phi(x,z)}{\partial v_{i}})_{z_{s}=w_{s}}+\sum
v_{i}v_{k}(\frac{\partial^{2}\Phi(x,z)}{\partial v_{i}\partial
v_{k}})_{z_{s}=w_{s}}+\ldots$

holds. Here all differentiations are to be carried out in such a way that one
first differentiates in $z_{i}$ and then differentiates $z_{i}$ in $v_{i}$
(either from $z_{i}=u_{i}+v_{i}$, or from $z_{i}=\frac{1}{u_{i}+v_{i}}$). Next
we consider, however, that the exponents of the individual terms in the
expansions of the derivatives (in $v_{i}$) of $z_{i}$ have a non-negative real
part. Likewise, the partial derivatives of $\Phi(z_{i})$ with respect to the
$z_{i}$ only contain terms whose exponents have non-negative real parts, and
this is even the case when we replace the $z_{i}$ with the $w_{i}$ . From all
this it follows that the real parts of the exponents of the individual terms
in the sums

$\sum v_{i}v_{k}(\frac{\partial^{2}\Phi(x,z)}{\partial v_{i}\partial
v_{k}})_{z_{s}=w_{s}},\hskip 30.0pt\sum
v_{i}v_{k}v_{l}(\frac{\partial^{2}\Phi(x,z)}{\partial v_{i}\partial
v_{k}\partial v_{l}})_{z_{s}=w_{s}},\ldots$

are no smaller than $2\sigma_{t}$, where $\sigma_{t}$ is the real part of the
exponent of $a_{t}^{n_{t}}$.

We have applied Taylor’s theorem above, even the meaning of the expansion (66)
is of course quite different from what is commonplace. The $u_{i}$ and $v_{i}$
are not sufficiently small numbers, but rather power series in $x$. Equation
(66) says that if you replace the $z_{i}$ and the $v_{i}$ with their series
expansions in $x$ on the right and left-hand sides, and rearrange them purely
formally, exactly the same terms appear on the right and left-hand sides.
However, this rearrangement is always possible without taking limits, since
according to our assumptions about the $u_{i}$, only a finite number of terms
can occur that have the same exponent.

We could also interpret equations (65), (66) as congruences modulo
sufficiently large positive powers $x^{\varrho}$ of $x$, in the sense that for
all powers of $x$, we are to ignore those for whose exponents the real parts
are larger than $\varrho$. Then on both sides of equations (65) and (66) only
finitely many terms are considered for any $\varrho$, and equations (65), (66)
say that the corresponding congruences hold for each $\varrho$. - And since
the expansions resulting in equation (66) always have coefficients that are
power series in $x$ in which the exponents of the individual terms have non-
negative real parts, it is clear why we are allowed to use Taylor’s theorem
(and the rule on differentiating function with respect to a function).

We now draw our attention to the first sum on the right hand side of (66)

$\sum v_{i}(\frac{\partial\Phi(x,z)}{\partial v_{i}})_{z_{s}=w_{s}}.$

If for any $i>1$ we have that $z_{i}=u_{i}+v_{i}$, then the coefficient of
$v_{i}$ equals

$\Phi_{i}(x,w_{2},\ldots,w_{n+2}).$

But if $z_{i}=\frac{1}{u_{i}+v_{i}}$, then the coefficient of $v_{i}$ equals

$\Phi_{i}(x,w_{2},\ldots,w_{n+2})\cdot\frac{-1}{u_{i+2}^{2}}.$

We now wish to find the terms in (66) whose exponents have the smallest real
part. For $i$ such that $z_{i}=u_{i}+v_{i}$, the initial terms of
$v_{i}(\frac{\partial\Phi}{\partial v_{i}})_{z_{s}=w_{s}}$ are equal to

(67)
$\displaystyle(i-2)!\binom{n_{i}}{i-2}a_{t}x^{n_{t}-i+2}c_{i,\kappa}x^{\lambda_{i,\kappa}}\hskip
30.0pt(\kappa=1,2,\ldots)$

for sufficiently large $t$. If the real part of $n_{t}$ is $\sigma_{t}$, then
the real part of the exponents of the terms (67) is $\sigma_{t}+\nu_{t}-i+2$.
But if $z_{i}=\frac{1}{u_{i}+v_{i}}$, then

(68)
$\displaystyle-(i-2)!\binom{n_{i}}{i-2}a_{t}x^{n_{t}-i+2}c_{i,\kappa}x^{\lambda_{i,\kappa}}\frac{1}{b_{i}^{2}}x^{-2\mu_{i}}\hskip
30.0pt(\kappa=1,2,\ldots),$

where $b_{i}x^{\mu_{i}}$ is the initial term of $u_{i}$. If the real part of
$\mu_{i}$ equals $\varrho_{i}$, then the real part of the exponents of (68) is
equal to

$\sigma_{t}+\nu_{i}-2\varrho_{i}-i+2.$

Here the numbers $\nu_{i},\varrho_{i}$ are independent of $t$. We now take the
sum of the terms (67) and (68), bringing $a_{t}x^{n_{t}}$ in front of the
brackets, then subsequently sorting the terms inside the brackets in order of
their exponents. We then obtain

$a_{t}x^{n_{t}}\sum\varphi_{s}(n_{t})x^{k_{s}},$

where $k_{s}$ are all distinct complex numbers and $\varphi_{s}(n_{t})$ are
polynomials in $n_{t}$ with fixed numerical coefficients that are independent
of $t$. Evidently, not all $\varphi_{s}(n_{t})$ vanish identically in $n_{t}$,
since in (67) and (68) $i$ takes on many different values, and therefore
$n_{t}$ appears only once in the $n$-th power, namely $\binom{n_{t}}{n}$. If
$k$ is a $k_{s}$ for which the real part is as small as possible, and the
coefficient $\varphi(n_{t})$ of $x^{k}$ is non-zero, then $\varphi(n_{t})$ can
only vanish for a finitely many $n_{t}$. From a certain $t$ onward, the term

$a_{t}\varphi(n_{t})x^{n_{t}+k}$

either cancels with a term from the second, third, $\ldots$ sum on the right-
hand side of (66), or with a term of $\Phi(x,w)$. The real parts of the
exponents of the terms in the second, third, $\ldots$ sum of the right-hand
side of (66) are no less than $2\sigma_{t}$, that is from a certain $t$ onward
they are certainly greater than the real part of $\sigma_{t}+k$. Hence, the
expansion of $F(x,w_{2},\ldots,w_{n+2})$ must result in at least one term
whose exponent is equal to $n_{t}+k$. The exponents of the terms $\Phi(x,w)$
are made up of integer linear combinations of $1,n_{0},n_{1},\ldots,n_{t-1}$.
Likewise, $k$ is made up of $1$ and finitely many $n_{i}$ linearly using
integer coefficients.

∎

In Theorem 21 we formulated the property that, for very extensive classes of
series iterated over arbitrary powers of the argument, they do not formally
satisfy any analytic partial differential equation. We now want to extend this
property to functions that can be represented by such series.

To achieve this, we first of all require of the exponents
$n_{i}=\sigma_{i}+\sqrt{-1}\tau_{i}$ that
$\lim_{i\to\infty}\sigma_{i}=+\infty$, secondly that
$|\frac{\tau_{i}}{\sigma_{i}}|$ remains bounded from a certain $i$ on. This
means, geometrically speaking, that all $n_{i}$ from a certain $i$ onwards lie
within an angle from the aperture $\gamma<\pi$, containing positive part of
the real axis, while leaving the imaginary axis (except for the origin)
completely untouched.

In order to formulate the assumption regarding the convergence of the series
(57), we introduce the concept of absolute convergence in the sharp sense. We
call a series $\sum a_{i}x^{n_{i}}$ with complex exponents
$n_{i}=\sigma_{i}+\sqrt{-1}\tau_{i}$, which satisfy the above conditions,
absolutely convergent in the sharp sense for $|x|=\nu$, if the series
$\sum|a_{i}|\nu^{\sigma_{i}}e^{2\pi|\tau_{i}|}$ converges. Such a series
converges absolutely on the whole circle $|x|=\nu$, as well as in every non-
zero point within this circle, and absolutely in the sharp sense on every
concentric circle of smaller radius. If $\sum a_{i}x^{n_{i}}$ converges
absolutely in the sharp sense for $|x|=\nu$, then the formally constructed
derivative $\sum n_{i}a_{i}x^{n_{t}-1}$ of this series converges absolutely in
the sharp sense for $|x|=\nu-\epsilon$, where $\epsilon$ is an arbitrarily
small fixed positive quantity. Since in order to prove this, from the
convergence of $\sum\alpha_{i}\nu^{\sigma_{i}}e^{2\pi|\tau_{i}|}$, where
$|a_{i}|=\alpha_{i}$ is set, the convergence of the series
$\sum\alpha_{i}|\sigma_{i}+\sqrt{-1}\tau_{i}|(\nu-\epsilon)^{\sigma_{t}-1}e^{2\pi|\tau_{i}|}$
must be deduced. But this follows from

$\lim_{i\to\infty}|\sigma_{i}+\sqrt{-1}\tau_{i}|\frac{(\nu-\epsilon)^{\sigma_{i}-1}}{\nu^{\sigma_{i}}}=|\frac{\sigma_{i}+\sqrt{-1}\tau_{i}}{\sigma_{i}}|\sigma_{i}\frac{(\nu-\epsilon)^{\sigma_{i}-1}}{\nu^{\sigma_{i}}}=0.$

Since $\frac{\sigma_{i}+\sqrt{-1}\tau_{i}}{\sigma_{i}}$ is absolutely bounded
from a certain $i$ onward and
$\lim_{i\to\infty}\sigma_{i}(\frac{\nu-\epsilon}{\nu})^{\sigma_{i}}=0.$ It
follows from this that every formally constructed derivative of $\sum
a_{i}x^{n_{i}}$ is absolutely convergent in the sharp sense for
$|x|=\nu-\epsilon$. And from the uniformity of the convergence, which results
from the proof of the uniqueness theorem below, it follows that the formally
constructed derivatives represent the derivatives of the function represented
by the original series.

If $\sum a_{i}x^{n_{i}}$ converges absolutely in the sharp sense, and if all
$\sigma_{i}$ are positive, then, as we shall immediately prove in detail, the
majorant $\sum\alpha_{i}\nu^{\sigma_{i}}e^{2\pi|\tau_{i}|}$ also becomes
arbitrarily small with decreasing $\nu$. From this it follows that the result
of substituting (57) in the left-hand side of (58) converges absolutely in the
sharp sense on a certain circle around the origin, and there represents the
result of substituting the function represented by (57) in (58). Thus it
remains only to prove the uniqueness: If a series $\sum a_{i}x^{n_{i}}$
converges absolutely in the sharp sense on a circle around the origin , and if
the function represented by it equals 0, then all coefficients $a_{i}$ vanish
.

We can obviously assume that all $n_{i}=\sigma_{i}+\sqrt{-1}\tau_{i}$ are
mutually distinct and that $\sigma_{i}$ are ordered in non-decreasing order.
We can also assume that the first $k(k>0)\sigma_{i}$ vanish, since this
otherwise can be achieved by multiplying by $x^{-\sigma_{0}}$. Thus let
$\sigma_{0}=\ldots=\sigma_{k}=0$, $\sigma_{k+1}>0$. We claim first that every
positive $\epsilon$ can be assigned such a $\delta$ that
$|\sum_{k+1}^{\infty}a_{i}x^{n_{i}}|<\epsilon$ as soon as $|x|<\delta$. In
general, let $|a_{i}|=\alpha_{i}$. From the absolute convergence in the sharp
sense of $\sum a_{i}x^{n_{i}}$ it follows that the Dirichlet series

(69)
$\displaystyle\sum_{i=k+1}^{\infty}\alpha_{i}e^{2\pi|\tau_{i}|}e^{-\sigma_{i}s}$

converges absolutely and therefore uniformly in a certain $s$-half-plane. From
this it follows that one can specify $\delta_{1}$ such that the absolute value
of (69) becomes smaller than $\epsilon$ as soon as the real part of $s$
becomes greater than $\delta_{1}$. From this it follows, however, that the
absolute value of $\sum_{i=k+1}^{\infty}a_{i}x^{n_{i}}$ becomes smaller than
$\epsilon$ as soon as $|x|<\delta=e^{-\delta_{1}}$.

As a result, if all the coefficients $a_{0},\ldots,a_{k}$ were non-zero, the
sum $a_{0}x^{\sqrt{-1}\tau_{0}}+\ldots+a_{k}x^{\sqrt{-1}\tau_{k}}$ would
converge to $0$ with $x$. So it suffices to show that there is such a sequence
of real values of $s$ that increase to infinity such that

$f(s)=a_{0}x^{\sqrt{-1}\tau_{0}}+\ldots+a_{k}x^{\sqrt{-1}\tau_{k}}$

converges to a value other than $0$. The entire function $f(s)$ cannot vanish
identically, since otherwise it would also have to vanish for purely imaginary
$s=\sqrt{-1}t$, and one term in $f(s)$ would dominate for sufficiently large
$t$ , since $\tau_{0},\ldots,\tau_{k}$ are mutually distinct. For real
$s=s_{0}$, assume $f(s_{0})\neq 0$. Let $m$ be an arbitrarily large positive
integer. Then, as is well-known, there is a non-zero positive integer $s_{m}$,
which can be found with the help of the famous Dirichlet approximation, such
that the numbers $s_{m}\tau_{0},\ldots,s_{m}\tau_{k}$ differ from certain
integer multiples of $2\pi$ by less than $\frac{1}{m^{2}}$. And consequently
the value of $f(s_{0}+ms_{m})$ will converge to $f(s_{0})\neq 0$ with
increasing $m$.

We can summarize this result as

######  Theorem 22.

If an absolutely convergent series in the sharp sense

$y(x)=\sum a_{i}x^{n_{i}}\hskip 30.0pt(n_{i}=\sigma_{i}+\sqrt{-1}\tau_{i})$

has the property that

  1. (1)

$\lim\sigma_{i}=+\infty$;

  2. (2)

$|\frac{\tau_{i}}{\sigma_{i}}|$ remains bounded from a certain $i$ onward;

  3. (3)

if $\sigma_{i}$ is the smallest $\sigma$ corresponding to a non-zero $\tau$,
and if all smaller $\sigma$ are non-negative integers, then $\sigma_{i}$ is
not a non-negative integer and there is no second exponent $n$ with the same
$\sigma_{i}$;

and if furthermore the function $y(x)$ satisfies a differential equation, the
left-hand side of which is analytic in
$x,y-c,y^{\prime}-c_{1},\ldots,y^{(n)}-c_{n},\frac{1}{y^{(n+1)}},\ldots,\frac{1}{y^{(n+m)}}$,
where the real parts of the exponents of the initial terms of
$x,y-c,y^{\prime}-c_{1},\ldots,y^{(n)}-c_{n},\frac{1}{y^{(n+1)}},\ldots,\frac{1}{y^{(n+m)}}$
are non-negative and $c_{1},\ldots,c_{n}$ are finite constants, then the
exponents $n_{i}$ have a finite integer basis.

At the same time, this theorem contains a generalization of Theorem 12 to the
case where the exponents are complex252525In fact, every algebraic
differential equation whose left-hand side is a polynomial in $x$, the unknown
function and its derivatives, can be written in the form (58), possibly by
division with a product of powers of the unknown function and its
derivatives.. The substitution $x=e^{-s}$ then leads to an analogous theorem
about Dirichlet series $\sum a_{i}e^{-\lambda_{i}s}$, where $\lambda_{i}$ may
be complex, and an analogous requirement of absolute convergence in the sharp
sense must be made.

In formulating the theorem at the beginning of this section, we made
assumptions about the form of the differential equation (58), which could
possibly be formulated a little more generally. To that end, the theorem that
among the exponents $n_{i}$ there are only finitely many linearly independent
ones probably holds even if $F$ is a power series iterated over any integer
(whether positive or negative) powers of its arguments converging in a certain
neighborhood of the origin. However, this case no longer permits use of the
methods used in this section. The reason why the validity of Theorems 21 and
22 also appears plausible in this case is easy to overlook. If there were an
infinite number of linearly independent exponents among the exponents $n_{i}$,
then we would be able to (at least formally) introduce an infinite number of
arbitrary constants into the integral by repeatedly winding around the origin.
This conclusion can be carried out completely in many cases. For example, with
their help one can prove that the function
$\sum_{i=1}^{\infty}x^{\text{log}(p_{i})}$, where $p_{i}$ is the $i$-th prime
number, satisfies no differential equation whose left-hand side is an
unambiguous, generally analytic function of the derivatives in a neighborhood
of the origin. And analogous power series in several variables can be
constructed. For example, the analogous fact can be proven for the series

$\sum x^{n}y^{\text{log}(n)},$

which arises from the series $\sum x^{n}n^{-s}$ by a simple substitution. And
with that a new - third - proof of the property that the series $\sum
x^{n}n^{-s}$ does not satisfy any algebraic partial differential equation has
been given. I will elaborate on these remarks elsewhere and in doing so I will
delve into their close connection with a known conclusion, first used by Mr.
H. Bohr in the theory of Riemann’s $\zeta$ function.

(Received February 3, 1919.)