# Parameter-Efficient Fine-Tuning via Circular Convolution

Aochuan Chen♠ Ziqi Gao♠ Zijing Liu${}^{\vardiamondsuit}$ Yu
Li${}^{\vardiamondsuit}$ Jia Li♠
♠The Hong Kong University of Science and Technology (Guangzhou)
${}^{\vardiamondsuit}$International Digital Economy Academy

###### Abstract

Low-Rank Adaptation (LoRA) has gained popularity for fine-tuning large
foundation models, leveraging low-rank matrices $\mathbf{A}$ and $\mathbf{B}$
to represent weight changes (i.e., $\Delta\mathbf{W}=\mathbf{B}\mathbf{A}$).
This method reduces trainable parameters and mitigates heavy memory
consumption associated with full delta matrices by sequentially multiplying
$\mathbf{A}$ and $\mathbf{B}$ with the activation. Despite its success, the
intrinsic low-rank characteristic may limit its performance. Although several
variants have been proposed to address this issue, they often overlook the
crucial computational and memory efficiency brought by LoRA. In this paper, we
propose Circular Convolution Adaptation (C3A), which not only achieves high-
rank adaptation with enhanced performance but also excels in both
computational power and memory utilization. Extensive experiments demonstrate
that C3A consistently outperforms LoRA and its variants across various fine-
tuning tasks.

##  1 Introduction

In recent years, Large Foundation Models (LFMs) have surged in popularity due
to their remarkable performance across various tasks in natural language
processing Brown et al. (2020); Touvron et al. (2023), computer vision Radford
et al. (2021); Kirillov et al. (2023), and other domains Li et al. (2024).
Characterized by their vast number of parameters and substantial computational
demands, these models have set new benchmarks in accuracy and versatility.
However, their size and complexity pose significant challenges for efficient
fine-tuning, particularly in resource-constrained environments Malladi et al.
(2023); Zhang et al. (2024). To address this issue, Parameter-efficient fine-
tuning (PEFT) techniques Mangrulkar et al. (2022), such as Low-Rank Adaptation
(LoRA) Hu et al. (2021), have emerged as practical solutions.

LoRA reduces the number of trainable parameters by employing low-rank matrices
to approximate weight changes, making fine-tuning more feasible and accessible
without compromising the model’s performance. Concretely, LoRA can be
expressed in math:

$\displaystyle\mathbf{Wx}=(\mathbf{W}_{0}+\Delta\mathbf{W})\mathbf{x}=\mathbf{W}_{0}\mathbf{x}+\mathbf{B(Ax)},$

where $\mathbf{W},\mathbf{W}_{0},\Delta\mathbf{W}\in\mathbb{R}^{d\times d}$
are weight matrices, $\mathbf{B}\in\mathbb{R}^{d\times
r},\mathbf{A}\in\mathbb{R}^{r\times d}$ are low-rank matrices to construct
$\Delta\mathbf{W}$ and $\mathbf{x}\in\mathbb{R}^{d}$ is the activations. The
number of trainable parameters is $2rd$, prompting the choice $r\ll d$ (e.g.,
$r=8$ for $d=1024$) for achieving high parameter efficiency. However, as
discussed in Zeng & Lee (2023), LoRA’s capacity to represent a target model is
strictly limited by $r$. To address this trade-off between performance and
efficiency, Kopiczko et al. (2023) proposed Vector Random Matrix Adaptation
(VeRA), which achieves competitive performance with significantly fewer
trainable parameters through fixed random-matrix projections. Despite its low
parameter count, VeRA requires substantial computational power and memory due
to the large size of the random matrices used in projection. This raises the
following open research question in the context of PEFT:

How can high-rank adaptation be achieved without incurring significant costs
in terms of computational power, memory usage, and trainable parameters?

As discussed in Dosovitskiy et al. (2020), dense linear layers lack strong
inductive biases, limiting transformers’ ability to compete with Convolutional
Neural Networks (CNNs) when data is scarce. Given the constraints of a small
training set for the downstream task, we posit that incorporating a stronger
inductive bias could enhance adaptation. In light of this, we propose Circular
Convolution Adaptation (C3A). Circular convolution is famous in signal
processing Li et al. (2020) and cryptography Dworkin et al. (2001). This
operator can be equivalently represented as multiplication by a circulant
matrix, offering flexibility in rank unconstrained by the number of trainable
parameters. Moreover, leveraging the Fast Fourier Transform (FFT), C3A ensures
better computational and memory efficiency than directly multiplying a delta
matrix Bamieh (2018), rendering it competitive with LoRA from the efficiency
aspect. We summarize our contribution as below:

① We introduce C3A, a novel approach for PEFT using convolution. This method
leverages circular convolution kernels and their equivalent circulant matrices
to provide a flexible rank, which is free of linear constraint by the number
of trainable parameters, for the delta matrix.

② Leveraging the elegant diagonalization of the circulant matrix, we implement
both the forward pass and backpropagation using FFT. With the incorporation of
FFT, the computation and memory efficiency of C3A excels. C3A strikes a unique
balance between performance an efficiency.

③ To offer greater flexibility in controlling the number of trainable
parameters, we extend C3A by incorporating block-circular convolution, which
results in block-circulant matrices. This extension allows C3A to achieve
fully customizable parameter counts as well as adaptable rank configurations.

④ We validate C3A through comprehensive fine-tuning experiments across diverse
tasks including natural language understanding, instruction tuning, image
classification, and text-to-image generation. Our experiments demonstrate
C3A’s outstanding accuracy and memory merits compared to existing state-of-
the-art methods.

##  2 Related Work

###  2.1 Parameter-Efficient Fine-Tuning

Research on Parameter-Efficient Fine-Tuning (PEFT) has generally progressed
along three main directions. The first direction involves partially updating
the pre-trained neural network (e.g., the body Oh et al. (2020) or the biases
Zaken et al. (2021)). Initial methods relied on hand-crafted heuristics Han et
al. (2015); Oh et al. (2020) to identify which parameters are crucial and
should be fine-tuned. More advanced approaches employ optimization techniques
Guo et al. (2020); Xu et al. (2021); Fu et al. (2023). For example, Guo et al.
Guo et al. (2020) reformulated the discrete optimization problem into a
continuous one using Bernoulli masks and the Gumbel-softmax approximation.

The second direction emerged to maintain the integrity of the pre-trained
model and enable a high degree of parameter sharing through adapter-based
methods He et al. (2021); Rebuffi et al. (2017); Rücklé et al. (2020); Liu et
al. (2022). These works focus on integrating additional modules, termed
adapters, to fit the downstream task, effectively decoupling the pre-trained
model parameters from those specific to the downstream task. Prompt Tuning
Brown et al. (2020); Gao et al. (2020); Chen et al. (2023) and Prefix Tuning
Li & Liang (2021) also fall into this category, despite ignoring potential
semantic meanings.

The final direction is characterized by delta-weight-based methods, such as
Low-Rank Adaptation (LoRA) Hu et al. (2021) and Orthogonal Fine-tuning (OFT)
Qiu et al. (2023). These methods bridge the gap between the pre-trained model
and the downstream task by adaptive delta weights, which is learned and stored
separately while used in combination with the pre-trained weights. This unique
design enables disentanglement and parameter sharing while preserving the
ability of integrating without additional inference cost. LoRA models the
delta-weights by an additive matrix while OFT does it by a multiplicative one.
To further improve either parameter efficiency or performance, many variants
has been proposed for both of the methods Kopiczko et al. (2023); Liu et al.
(2024; 2023); Yuan et al. (2024). However, these methods can hardly achieve
high parameter efficiency and performance without incurring heavy computation
and memory usage.

###  2.2 Circular Convolution

Circular convolution has been extensively studied in signal processing Rabiner
et al. (1978); McGillem & Cooper (1984); Li et al. (2020) and cryptography
Dworkin et al. (2001); Gong et al. (2024). Owing to its computational
advantages, circular convolution has also been explored in machine learning
for generating long embeddings of high-dimensional data Yu et al. (2014) and
compressing heavily parameterized layers Cheng et al. (2015); Ding et al.
(2017). Remarkably, it achieves these efficiencies without significant
performance degradation, which makes it a promising technique for fine-tuning
applications.

Despite its success in small neural networks such as LeNet Cheng et al.
(2015), circular convolution has not demonstrated lossless performance in
modern large foundational models (LFMs) or even in their base architecture,
the transformer. This limitation may be attributed to the conflict between its
high intrinsic bias (i.e., the circulant pattern) and the vast amount of data
required for training LFMs. Conversely, when fine-tuning LFMs, it is often
impractical to collect as much data as needed for training from scratch. In
such scenarios, the intrinsic bias of circular convolution could potentially
serve as a regularization mechanism, thereby benefiting the fine-tuning
process.

##  3 Method

In this section, we present C3A (see an overview in Figure 1), a novel PEFT
approach based on the circular convolution. C3A follows LoRA’s setting of
learning an additive linear operation over the original dense linear
transformation. However, instead of using low-rank decomposition and the
matrix multiplication operator, C3A resorts to circular convolution as this
additive linear operation. Section 3.1 introduces the notations we use.
Section 3.2 discusses the circular convolution operator, its equivalent
circulant matrix, and its calculation in the frequency domain. Section 3.3
details an efficient method for backpropagation. Section 3.4 describes block-
wise convolution for controlling the number of trainable parameters. Finally,
Section 3.5 analyzes the computational complexity.

Figure 1: Overview of LoRA (A) and our C3A (B,C) method. In LoRA, only low-
rank matrices $\mathbf{A}$ and $\mathbf{B}$ are trained and the delta weight
is represented by their product (i.e., $\Delta\mathbf{W}=\mathbf{BA}$). The
total trainable parameter number is $2rd$, which is assosiated with the rank
of the delta weight. In C3A, circular convolution kernels $\mathbf{w}$ are
tuned to adapt to the downstream task and the delta weight is represented by
the (block-)circular matrix they construct (i.e.,
$\Delta\mathbf{W}=\mathcal{C}_{b}(\Delta\mathbf{w})$). The total trainable
parameter number is $bd$, which disentangles with the rank of the delta
weight.

###  3.1 Notations

We denote circular convolution operator as $\star$. The adapted weight matrix,
the original weight matrix, and the delta matrix are denoted by $\mathbf{W}$,
$\mathbf{W}_{0}$, and $\Delta\mathbf{W}$, respectively. The activation vector
of the previous layer is denoted by $\mathbf{x}$. The post-transformation
vector is $\mathbf{z}$, where $\mathbf{z}=\mathbf{W}\mathbf{x}$, and the
incremental part is denoted by $\Delta\mathbf{z}$, where
$\Delta\mathbf{z}=\Delta\mathbf{W}\mathbf{x}$. The length of the activation
vector is $d$. The matrices $\mathbf{A}$ and $\mathbf{B}$ are low-rank
matrices introduced by LoRA to represent $\Delta\mathbf{W}$, with $r$ being
their rank. $r_{v}$ is used to specify the random projection matrix rank used
in VeRA. The circular convolution kernel of C3A is denoted by
$\Delta\mathbf{w}$. The loss function is represented by $\mathcal{L}$. The
Fast Fourier Transform and its inverse are denoted by $\mathrm{FFT}$ and
$\mathrm{iFFT}$, respectively. The Hadamard product is denoted by $\circ$. In
terms of dimensions, we have
$\mathbf{W},\mathbf{W}_{0},\Delta\mathbf{W}\in\mathbb{R}^{d\times d}$, and
$\mathbf{x},\mathbf{z},\Delta\mathbf{z},\Delta\mathbf{w}\in\mathbb{R}^{d}$.
The matrices $\mathbf{B}\in\mathbb{R}^{d\times r}$ and
$\mathbf{A}\in\mathbb{R}^{r\times d}$ are controlled by the rank $r$.

###  3.2 Circular Convolution

The circular convolution operator is defined as
$\Delta\mathbf{z}=\Delta\mathbf{w}\star\mathbf{x}=\mathcal{C}(\Delta\mathbf{w})\mathbf{x}$,
where $\mathcal{C}(\Delta\mathbf{w})$ is the circulant matrix of
$\Delta\mathbf{w}$. The first row of $\mathcal{C}(\Delta\mathbf{w})$ is
$\Delta\mathbf{w}$ and each row equals the row above it periodically shifted
to right by one element. Concretely,

$\displaystyle\mathcal{C}(\Delta\mathbf{w})=\begin{bmatrix}\Delta w_{1}&\Delta
w_{2}&\cdots&\Delta w_{d-1}&\Delta w_{d}\\\ \Delta w_{d}&\Delta
w_{1}&\cdots&\Delta w_{d-2}&\Delta w_{d-1}\\\
\cdots&\cdots&\cdots&\cdots&\cdots\\\ \Delta w_{3}&\Delta w_{4}&\cdots&\Delta
w_{1}&\Delta w_{2}\\\ \Delta w_{2}&\Delta w_{3}&\cdots&\Delta w_{d}&\Delta
w_{1}\\\ \end{bmatrix}\in\mathbb{R}^{d\times d}.$

Theoretically, the rank of $\mathcal{C}(\Delta\mathbf{w})$ is
$d-\mathrm{Deg}(\mathrm{gcd}(f(x),x^{d}-1))$ Ingleton (1956), where
$\mathrm{Deg}(\cdot)$ represents the degree of a polynomial, $f(x)$ is the
associated polynomial of $\Delta\mathbf{w}$ (i.e., $f(x)=\sum_{i=1}^{d}\Delta
w_{i}x^{i-1}$), and $\mathrm{gcd}(\cdot)$ means the greatest common divisor.
By learning $\Delta\mathbf{w}$ in the $\mathbb{R}^{n}$ oracle, C3A
automatically achieves dynamic rank selection that is not linearly bounded by
the number of learnable parameters as LoRA.

To achieve efficiency, we leverage the beautiful circulant structure of
$\mathcal{C}(\Delta\mathbf{w})$, which makes it diagonalizable by the Fourier
basis ($\mathbf{F}$) . In math, it can be described as
$\mathcal{C}(\Delta\mathbf{w})=\mathbf{F}\frac{\Lambda}{d}\mathbf{F}^{-1}$
Golub & Van Loan (1996), where $\Lambda$ is its eigenvalues and can be
calculated by a Fourier transform of the first row (i.e.,
$\Lambda=\mathrm{diag}(\mathbf{F}\Delta\mathbf{w})$). Therefore, we can
calculate $\Delta\mathbf{w}\star\mathbf{x}$ as

$\displaystyle\begin{split}\Delta\mathbf{w}\star\mathbf{x}&=\mathbf{F}\mathrm{diag}(\frac{\mathbf{F}\Delta\mathbf{w}}{d})\mathbf{F}^{-1}\mathbf{x}\\\
&=\mathrm{FFT}(\mathrm{FFT}(\Delta\mathbf{w})\circ\mathrm{iFFT}(\mathbf{x})).\end{split}$
(1)

###  3.3 Backpropagation

To implement backpropagation efficiently, we require the analytical
derivatives of the loss function $\mathcal{L}$, with respect to
$\Delta\mathbf{w}$ and $\mathbf{x}$. Utilizing the chain rule, these
derivatives can be expressed as:

$\displaystyle\frac{\partial\mathcal{L}}{\partial\mathbf{x}}=\frac{\partial\Delta\mathbf{z}}{\partial\mathbf{x}}\frac{\partial\mathcal{L}}{\partial\Delta\mathbf{z}},\qquad\frac{\partial\mathcal{L}}{\partial\Delta\mathbf{w}}=\frac{\partial\Delta\mathbf{z}}{\partial\Delta\mathbf{w}}\frac{\partial\mathcal{L}}{\partial\Delta\mathbf{z}}.$
(2)

Since $\Delta\mathbf{z}=\mathcal{C}(\Delta\mathbf{w})\mathbf{x}$, it follows
that
$\frac{\partial\Delta\mathbf{z}}{\partial\mathbf{x}}=\mathcal{C}(\Delta\mathbf{w})$.
For $\frac{\partial\Delta\mathbf{z}}{\partial\Delta\mathbf{w}}$, we note that
the circular convolution operation is commutative (i.e.,
$\Delta\mathbf{w}\star\mathbf{x}=\mathbf{x}\star\Delta\mathbf{w}$), thus
$\frac{\partial\Delta\mathbf{z}}{\partial\Delta\mathbf{w}}=\mathcal{C}(\mathbf{x})$.
Substituting these results into Equation 2, we obtain:

$\displaystyle\frac{\partial\mathcal{L}}{\partial\mathbf{x}}=\mathcal{C}(\Delta\mathbf{w})\frac{\partial\mathcal{L}}{\partial\Delta\mathbf{z}},\qquad\frac{\partial\mathcal{L}}{\partial\Delta\mathbf{w}}=\mathcal{C}(\mathbf{x})\frac{\partial\mathcal{L}}{\partial\Delta\mathbf{z}}.$

These expressions can also be interpreted as circular convolutions:

$\displaystyle\frac{\partial\mathcal{L}}{\partial\mathbf{x}}=\Delta\mathbf{w}\star\frac{\partial\mathcal{L}}{\partial\Delta\mathbf{z}},\qquad\frac{\partial\mathcal{L}}{\partial\Delta\mathbf{w}}=\mathbf{x}\star\frac{\partial\mathcal{L}}{\partial\Delta\mathbf{z}}.$

By explicitly implementing this derivative calculation using Equation 1,
backpropagation can also enjoy the computational efficiency provided by the
FFT algorithm.

###  3.4 Block-Circular Convolution

Despite the elegance and efficiency of the circular convolution operator, it
is worth noting that we can not alter the number of learnable parameters due
to the requirement of the convolution kernel being the same size as the
activation vector. To enable this flexibility in case of difficult downstream
tasks, we resort to the block-circular convolution. We can simply split the
activation vector $\mathbf{x}$ (and the post-transformation vector
$\mathbf{z}$) into blocks, and assign each pair of blocks with different
convolution kernels. To be concrete,

$\displaystyle\mathbf{x}=\begin{bmatrix}\mathbf{x}_{1}&\mathbf{x}_{2}&\cdots&\mathbf{x}_{b}\end{bmatrix}\qquad\Delta\mathbf{z}=\begin{bmatrix}\Delta\mathbf{z}_{1}&\Delta\mathbf{z}_{2}&\cdots&\Delta\mathbf{z}_{b}\end{bmatrix},$

where $b$ is the block number. We will need $b^{2}$ convolution kernels to
densely connect these blocks, which can be expressed as

$\displaystyle\Delta\mathbf{z}_{i}=\sum_{j=1}^{b}\Delta\mathbf{w}_{ij}\star\mathbf{x}_{j},i\in\\{1,2,\cdots,b\\},$

and can also be represented by a block-circular matrix

$\displaystyle\Delta\mathbf{z}=\mathcal{C}_{b}(\Delta\mathbf{w})\mathbf{x}\qquad\mathcal{C}_{b}(\Delta\mathbf{w})=\begin{bmatrix}\mathcal{C}(\Delta\mathbf{w}_{11})&\mathcal{C}(\Delta\mathbf{w}_{12})&\cdots&\mathcal{C}(\Delta\mathbf{w}_{1b})\\\
\mathcal{C}(\Delta\mathbf{w}_{21})&\mathcal{C}(\Delta\mathbf{w}_{22})&\cdots&\mathcal{C}(\Delta\mathbf{w}_{2b})\\\
\cdots&\cdots&\cdots&\cdots\\\
\mathcal{C}(\Delta\mathbf{w}_{b1})&\mathcal{C}(\Delta\mathbf{w}_{b2})&\cdots&\mathcal{C}(\Delta\mathbf{w}_{bb})\end{bmatrix}.$

In this case, $\Delta\mathbf{w}_{ij}\in\mathbb{R}^{\frac{d}{b}}$ (it is
required that $d$ is divisible by $b$) and we have $b^{2}\frac{d}{b}=bd$
learnable parameters. $b$ acts as a hyperparameter controlling the number of
learnable parameters here, just like $r$ for LoRA. However, we highlight that
while $r$ controls the rank of the delta matrix and learnable parameter number
at the same time, $b$ only affects the latter. Disentangling these attributes
enables higher adaptation flexibility and potential better results.

###  3.5 Complexity Analysis

We compare time compelxity and space complexity of LoRA, VeRA and C3A in Table
1. Detailed analysis follows.

Table 1: Time and space complexity comparison of LoRA, VeRA and C3A. We split
the space complexity into Parameter number and Other auxiliary tensors to help
better understand the differences. We highlight that in practice, to achieve
similar performance, $b\leq r\ll r_{v}$.

Method | Time | # Parameter | # Other | # Total
---|---|---|---|---
LoRA | $\mathcal{O}(rd)$ | $2rd$ | $r$ | $2rd+r$
VeRA | $\mathcal{O}(r_{v}d)$ | $r_{v}+d$ | $2r_{v}d+r_{v}+d$ | $2(r_{v}d+r_{v}+d)$
C3A | $\mathcal{O}(bd\log\frac{d}{b})$ | $bd$ | $0$ | $bd$

####  3.5.1 Time Complexity

LoRA incorporates low-rank matrices $\mathbf{A}$ and $\mathbf{B}$, which are
sequentially multiplied with the activation vector, resulting in a time
complexity of $\mathcal{O}(rd)$. Typically, $r\ll d$. Due to the
$\mathcal{O}(d\log d)$ complexity of the FFT algorithm used in Equation 1, C3A
achieves a time complexity of $\mathcal{O}(bd\log\frac{d}{b})$. Given that
$b\ll d$, C3A is comparable to LoRA in terms of time complexity. Conversely,
VeRA, despite its high-rank nature and minimal number of trainable parameters,
incurs a prohibitive time complexity of $\mathcal{O}(r_{v}d)$, where $r_{v}$
can exceed $d$. Therefore, striking a balance between high rank and
computational efficiency is never a low-hanging fruit.

####  3.5.2 Space Complexity

We analyze the space complexity of LoRA, VeRA, and C3A during training. The
differences among these methods primarily arise from the trainable parameter
size and the auxiliary tensors required for the forward pass and
backpropagation. For LoRA, an intermediate tensor of size $r$ is introduced
(i.e., $\mathbf{Ax}\in\mathbb{R}^{r}$). In contrast, VeRA necessitates two
random projection matrices, with a total size of $2r_{v}d$. Additionally, VeRA
introduces two intermediate tensors of sizes $d$ and $r_{v}$. Thanks to the
efficient backpropagation algorithm detailed in Section 3.3, C3A does not
require any additional intermediate tensors. This optimization significantly
reduces its space complexity compared to LoRA and VeRA.

##  4 Experiment

In this section, we comprehensively evaluate C3A in both natural language
processing (NLP) and computer vision (CV). For NLP, we first demonstrate the
effectiveness of C3A using the GLUE benchmark with RoBERTa-Base and RoBERTa-
Large models. Additionally, we apply C3A to fine-tuning the LLaMA family
models, which stands as a representative of prevalent large language models
(LLMs). For CV, we examine two practical scenarios: the traditional
classification task using Vision Transformers (ViTs) and the more intricate
text-to-image generation task based on diffusion models.

###  4.1 Natural Language Understanding

Baselines. We compare our C3A with several representative PEFT methods,
including BitFit Zaken et al. (2021), (IA)3 Liu et al. (2022), LoRA Hu et al.
(2021), VeRA Kopiczko et al. (2023), and BOFT Liu et al. (2023). BitFit
selectively fine-tunes existing parameters, specifically the biases. (IA)3 is
the state-of-the-art method that adds extra adapters. LoRA is a widely known
PEFT method that employs low-rank decomposition to compress additive delta
matrices. VeRA is a recent approach that focuses on further reducing trainable
parameters for LoRA. BOFT is another innovative method in PEFT research,
compressing multiplicative delta matrices using orthogonal decomposition and
butterfly factorization.

Settings. We evaluate our proposed C3A on the General Language Understanding
Evaluation (GLUE) benchmark Wang et al. (2018), which encompasses a wide range
of natural language understanding (NLU) tasks, including single-sentence
classification, similarity and paraphrase, and natural language inference. To
enhance practicality, we split these datasets following the train-validation-
test approach. The best-performing model is selected based on validation set
performance across the fine-tuning epochs, and the reported performance
corresponds to its performance on the test set. For this evaluation, we fine-
tune the pre-trained RoBERTa-Base and RoBERTa-Large models Liu et al. (2019).
For the unique hyperparameters of different baselines, we adopt the values
suggested in the original papers (e.g., VeRA’s $r$ and BOFT’s $b$ and $m$).
The number of trainable parameters excludes the classification head, as each
method uses the same one. The shared hyperparameters (i.e., the learning rate
for classification head and the learning rate for other trainable parameters)
are found by hyperparameter search. For the memory cost, to ensure fairness
and consistency, we fix the length of input data to 256 tokens and use a
batchsize of 64.

Results. Results are summarized in Table 2. In general, C3Ab=1 achieves better
or on-par performance compared to baseline methods with extremely small
trainable parameters. When a larger number of trainable parameters is allowed,
C3Ab=6 and C3Ab=8 significantly outperform the baselines. Furthermore,
compared to (IA)3, LoRA, VeRA, and BOFT, despite training a similar number of
or more parameters, C3A stands out for its excellent memory efficiency. The
only method with better memory efficiency is BitFit, which can be seen as an
upper bound since it does not introduce any new parameters.

Table 2: Performance of different PEFT methods on the GLUE benchmark. We fine-
tune pre-trained RoBERTa-Base and -Large models on 6 datasets. We report the
Matthew’s correlation coefficient (MCC) for CoLA, Pearson correlation
coefficient (PCC) for STS-B, and accuracy (Acc.) for all remaining tasks. For
all 6 datasets, a higher metric indicates better performance. “Avg.” denotes
the average accuracy of each method across all datasets. The best results for
each dataset are highlighted in bold. # Trainable parameters does not include
the classification head since each method uses the same head. Memory Cost is
measured on fixed length (i.e., $256$) data with a batchsize of $64$.

| Methods |  # Trainable Parameters |  Memory Cost (GB) | SST-2 | MRPC | CoLA | QNLI | RTE | STS-B | Avg.
---|---|---|---|---|---|---|---|---|---|---
Base | BitFit | 102K | 12.60 | 93.30$\pm$0.30 | 85.80$\pm$0.21 | 59.21$\pm$1.74 | 91.96$\pm$0.18 | 73.07$\pm$1.34 | 90.18$\pm$0.17 | 82.25
(IA)3 | 111K | 19.86 | 92.98$\pm$0.34 | 85.86$\pm$0.59 | 60.49$\pm$1.09 | 91.56$\pm$0.17 | 69.10$\pm$1.18 | 90.06$\pm$0.21 | 81.67
LoRAr=8 | 295K | 13.75 | 94.50$\pm$0.41 | 85.68$\pm$0.74 | 60.95$\pm$1.57 | 92.54$\pm$0.20 | 76.68$\pm$1.42 | 89.76$\pm$0.39 | 83.35
VeRAr=1024 | 43K | 15.51 | 93.97$\pm$0.17 | 86.23$\pm$0.41 | 62.24$\pm$1.91 | 91.85$\pm$0.17 | 75.74$\pm$1.56 | 90.27$\pm$0.25 | 83.38
BOFT${}^{m=2}_{b=8}$ | 166K | 13.11 | 93.23$\pm$0.50 | 84.37$\pm$0.54 | 59.50$\pm$1.25 | 91.69$\pm$0.12 | 74.22$\pm$0.84 | 89.63$\pm$0.37 | 82.11
C3Ab=1 | 18K | 12.83 | 93.42$\pm$0.26 | 86.33$\pm$0.32 | 61.83$\pm$0.96 | 91.83$\pm$0.04 | 76.17$\pm$1.39 | 90.46$\pm$0.29 | 83.34
C3Ab=6 | 111K | 12.72 | 94.20$\pm$0.16 | 86.67$\pm$0.54 | 62.48$\pm$1.20 | 92.32$\pm$0.25 | 77.18$\pm$1.41 | 90.16$\pm$0.42 | 83.84
Large | BitFit | 271K | 30.65 | 95.09$\pm$0.27 | 88.10$\pm$0.76 | 65.40$\pm$0.76 | 94.06$\pm$0.14 | 82.60$\pm$1.15 | 91.73$\pm$0.20 | 86.16
(IA)3 | 295K | 48.81 | 95.32$\pm$0.20 | 87.06$\pm$0.57 | 66.52$\pm$1.10 | 94.18$\pm$0.15 | 84.33$\pm$2.38 | 91.58$\pm$0.39 | 86.50
LoRAr=8 | 786K | 34.12 | 95.53$\pm$0.35 | 86.12$\pm$3.22 | 65.16$\pm$0.76 | 93.73$\pm$0.30 | 83.75$\pm$0.51 | 91.46$\pm$0.21 | 85.96
VeRAr=256 | 61K | 34.16 | 95.83$\pm$0.43 | 87.72$\pm$0.55 | 63.66$\pm$1.45 | 94.11$\pm$0.20 | 83.03$\pm$1.65 | 91.12$\pm$0.37 | 85.91
BOFT${}^{m=2}_{b=8}$ | 442K | 33.08 | 95.76$\pm$0.41 | 88.28$\pm$0.33 | 64.72$\pm$2.37 | 93.89$\pm$0.14 | 82.82$\pm$1.40 | 91.03$\pm$0.32 | 86.08
C3Ab=1 | 49K | 31.83 | 95.78$\pm$0.05 | 88.02$\pm$0.62 | 66.59$\pm$1.20 | 94.22$\pm$0.25 | 82.89$\pm$0.67 | 91.86$\pm$0.14 | 86.56
C3Ab=8 | 393K | 31.79 | 95.78$\pm$0.15 | 88.09$\pm$0.47 | 67.18$\pm$1.92 | 94.26$\pm$0.19 | 84.62$\pm$1.36 | 91.81$\pm$0.36 | 86.96

##  5 Conclusion

In this paper, we introduce C3A, an innovative approach to Parameter-Efficient
Fine-Tuning (PEFT). Unlike LoRA, which utilizes low-rank decomposition, C3A
employs circular convolution and its associated circulant matrix to model the
delta weight matrix. This approach aims to decouple the rank of the delta
weight matrix from the number of learnable parameters, allowing for high-rank
adaptation while maintaining a limited parameter size. By leveraging the Fast
Fourier Transform (FFT) in both the forward and backward passes, C3A achieves
significant computational and memory efficiency. Consequently, C3A presents
itself as a compelling alternative to LoRA for model fine-tuning.

## References

  * Bamieh (2018) Bassam Bamieh.  Discovering transforms: A tutorial on circulant matrices, circular convolution, and the discrete fourier transform.  _arXiv preprint arXiv:1805.05533_ , 2018.
  * Brown et al. (2020) Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al.  Language models are few-shot learners.  _Advances in neural information processing systems_ , 33:1877–1901, 2020.
  * Chen et al. (2023) Aochuan Chen, Yuguang Yao, Pin-Yu Chen, Yihua Zhang, and Sijia Liu.  Understanding and improving visual prompting: A label-mapping perspective.  In _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , pp. 19133–19143, 2023.
  * Cheng et al. (2015) Yu Cheng, Felix X Yu, Rogerio S Feris, Sanjiv Kumar, Alok Choudhary, and Shi-Fu Chang.  An exploration of parameter redundancy in deep networks with circulant projections.  In _Proceedings of the IEEE international conference on computer vision_ , pp. 2857–2865, 2015.
  * Ding et al. (2017) Caiwen Ding, Siyu Liao, Yanzhi Wang, Zhe Li, Ning Liu, Youwei Zhuo, Chao Wang, Xuehai Qian, Yu Bai, Geng Yuan, Xiaolong Ma, Yipeng Zhang, Jian Tang, Qinru Qiu, Xue Lin, and Bo Yuan.  Circnn: accelerating and compressing deep neural networks using block-circulant weight matrices.  In _Proceedings of the 50th Annual IEEE/ACM International Symposium on Microarchitecture_ , MICRO-50 ’17, pp. 395–408, New York, NY, USA, 2017. Association for Computing Machinery.  ISBN 9781450349529.  doi: 10.1145/3123939.3124552.  URL https://doi.org/10.1145/3123939.3124552.
  * Dosovitskiy et al. (2020) Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al.  An image is worth 16x16 words: Transformers for image recognition at scale.  _arXiv preprint arXiv:2010.11929_ , 2020.
  * Dworkin et al. (2001) Morris Dworkin, Elaine Barker, James Nechvatal, James Foti, Lawrence Bassham, E. Roback, and James Dray.  Advanced encryption standard (aes), 2001-11-26 2001.
  * Fu et al. (2023) Zihao Fu, Haoran Yang, Anthony Man-Cho So, Wai Lam, Lidong Bing, and Nigel Collier.  On the effectiveness of parameter-efficient fine-tuning.  In _Proceedings of the AAAI conference on artificial intelligence_ , volume 37, pp. 12799–12807, 2023.
  * Gao et al. (2020) Tianyu Gao, Adam Fisch, and Danqi Chen.  Making pre-trained language models better few-shot learners.  _arXiv preprint arXiv:2012.15723_ , 2020.
  * Golub & Van Loan (1996) Gene H. Golub and Charles F. Van Loan.  _Matrix computations (3rd ed.)_.  Johns Hopkins University Press, USA, 1996.  ISBN 0801854148.
  * Gong et al. (2024) Yanwei Gong, Xiaolin Chang, Jelena Mišić, Vojislav B Mišić, Jianhua Wang, and Haoran Zhu.  Practical solutions in fully homomorphic encryption: a survey analyzing existing acceleration methods.  _Cybersecurity_ , 7(1):5, 2024.
  * Guo et al. (2020) Demi Guo, Alexander M Rush, and Yoon Kim.  Parameter-efficient transfer learning with diff pruning.  _arXiv preprint arXiv:2012.07463_ , 2020.
  * Han et al. (2015) Song Han, Huizi Mao, and William J Dally.  Deep compression: Compressing deep neural networks with pruning, trained quantization and huffman coding.  _arXiv preprint arXiv:1510.00149_ , 2015.
  * He et al. (2021) Junxian He, Chunting Zhou, Xuezhe Ma, Taylor Berg-Kirkpatrick, and Graham Neubig.  Towards a unified view of parameter-efficient transfer learning.  _arXiv preprint arXiv:2110.04366_ , 2021.
  * Hu et al. (2021) Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen.  Lora: Low-rank adaptation of large language models.  _arXiv preprint arXiv:2106.09685_ , 2021.
  * Ingleton (1956) A. W. Ingleton.  The rank of circulant matrices.  _Journal of the London Mathematical Society_ , s1-31(4):445–460, 1956.  doi: https://doi.org/10.1112/jlms/s1-31.4.445.  URL https://londmathsoc.onlinelibrary.wiley.com/doi/abs/10.1112/jlms/s1-31.4.445.
  * Kirillov et al. (2023) Alexander Kirillov, Eric Mintun, Nikhila Ravi, Hanzi Mao, Chloe Rolland, Laura Gustafson, Tete Xiao, Spencer Whitehead, Alexander C Berg, Wan-Yen Lo, et al.  Segment anything.  In _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , pp. 4015–4026, 2023.
  * Kopiczko et al. (2023) Dawid Jan Kopiczko, Tijmen Blankevoort, and Yuki Markus Asano.  Vera: Vector-based random matrix adaptation.  _arXiv preprint arXiv:2310.11454_ , 2023.
  * Li et al. (2020) Changli Li, Hon Keung Kwan, and Xinxin Qin.  Revisiting linear convolution, circular convolution and their related methods.  _2020 13th International Congress on Image and Signal Processing, BioMedical Engineering and Informatics (CISP-BMEI)_ , pp. 1124–1131, 2020.  URL https://api.semanticscholar.org/CorpusID:227220098.
  * Li & Liang (2021) Xiang Lisa Li and Percy Liang.  Prefix-tuning: Optimizing continuous prompts for generation.  _arXiv preprint arXiv:2101.00190_ , 2021.
  * Li et al. (2024) Yuhan Li, Peisong Wang, Zhixun Li, Jeffrey Xu Yu, and Jia Li.  Zerog: Investigating cross-dataset zero-shot transferability in graphs.  _arXiv preprint arXiv:2402.11235_ , 2024.
  * Liu et al. (2022) Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A Raffel.  Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning.  _Advances in Neural Information Processing Systems_ , 35:1950–1965, 2022.
  * Liu et al. (2024) Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang-Ting Cheng, and Min-Hung Chen.  Dora: Weight-decomposed low-rank adaptation.  _arXiv preprint arXiv:2402.09353_ , 2024.
  * Liu et al. (2023) Weiyang Liu, Zeju Qiu, Yao Feng, Yuliang Xiu, Yuxuan Xue, Longhui Yu, Haiwen Feng, Zhen Liu, Juyeon Heo, Songyou Peng, et al.  Parameter-efficient orthogonal finetuning via butterfly factorization.  _arXiv preprint arXiv:2311.06243_ , 2023.
  * Liu et al. (2019) Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov.  Roberta: A robustly optimized bert pretraining approach.  _arXiv preprint arXiv:1907.11692_ , 2019.
  * Malladi et al. (2023) Sadhika Malladi, Tianyu Gao, Eshaan Nichani, Alex Damian, Jason D Lee, Danqi Chen, and Sanjeev Arora.  Fine-tuning language models with just forward passes.  _Advances in Neural Information Processing Systems_ , 36:53038–53075, 2023.
  * Mangrulkar et al. (2022) Sourab Mangrulkar, Sylvain Gugger, Lysandre Debut, Younes Belkada, Sayak Paul, and Benjamin Bossan.  Peft: State-of-the-art parameter-efficient fine-tuning methods.  https://github.com/huggingface/peft, 2022.
  * McGillem & Cooper (1984) Clare D. McGillem and George R. Cooper.  Continuous and discrete signal and system analysis.  1984\.  URL https://api.semanticscholar.org/CorpusID:117907785.
  * Oh et al. (2020) Jaehoon Oh, Hyungjun Yoo, ChangHwan Kim, and Se-Young Yun.  Boil: Towards representation change for few-shot learning.  _arXiv preprint arXiv:2008.08882_ , 2020.
  * Qiu et al. (2023) Zeju Qiu, Weiyang Liu, Haiwen Feng, Yuxuan Xue, Yao Feng, Zhen Liu, Dan Zhang, Adrian Weller, and Bernhard Schölkopf.  Controlling text-to-image diffusion by orthogonal finetuning.  _Advances in Neural Information Processing Systems_ , 36:79320–79362, 2023.
  * Rabiner et al. (1978) L. R. Rabiner, B. Gold, and C. K. Yuen.  Theory and application of digital signal processing.  _IEEE Transactions on Systems, Man, and Cybernetics_ , 8(2):146–146, 1978.  doi: 10.1109/TSMC.1978.4309918.
  * Radford et al. (2021) Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.  Learning transferable visual models from natural language supervision.  In _International conference on machine learning_ , pp. 8748–8763. PMLR, 2021.
  * Rebuffi et al. (2017) Sylvestre-Alvise Rebuffi, Hakan Bilen, and Andrea Vedaldi.  Learning multiple visual domains with residual adapters.  _Advances in neural information processing systems_ , 30, 2017.
  * Rücklé et al. (2020) Andreas Rücklé, Gregor Geigle, Max Glockner, Tilman Beck, Jonas Pfeiffer, Nils Reimers, and Iryna Gurevych.  Adapterdrop: On the efficiency of adapters in transformers.  _arXiv preprint arXiv:2010.11918_ , 2020.
  * Touvron et al. (2023) Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, et al.  Llama: Open and efficient foundation language models.  _arXiv preprint arXiv:2302.13971_ , 2023.
  * Wang et al. (2018) Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman.  Glue: A multi-task benchmark and analysis platform for natural language understanding.  _arXiv preprint arXiv:1804.07461_ , 2018.
  * Xu et al. (2021) Runxin Xu, Fuli Luo, Zhiyuan Zhang, Chuanqi Tan, Baobao Chang, Songfang Huang, and Fei Huang.  Raise a child in large language model: Towards effective and generalizable fine-tuning.  _arXiv preprint arXiv:2109.05687_ , 2021.
  * Yu et al. (2014) Felix Yu, Sanjiv Kumar, Yunchao Gong, and Shih-Fu Chang.  Circulant binary embedding.  In _International conference on machine learning_ , pp. 946–954. PMLR, 2014.
  * Yuan et al. (2024) Shen Yuan, Haotian Liu, and Hongteng Xu.  Bridging the gap between low-rank and orthogonal adaptation via householder reflection adaptation.  _arXiv preprint arXiv:2405.17484_ , 2024.
  * Zaken et al. (2021) Elad Ben Zaken, Shauli Ravfogel, and Yoav Goldberg.  Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models.  _arXiv preprint arXiv:2106.10199_ , 2021.
  * Zeng & Lee (2023) Yuchen Zeng and Kangwook Lee.  The expressive power of low-rank adaptation.  _arXiv preprint arXiv:2310.17513_ , 2023.
  * Zhang et al. (2024) Yushun Zhang, Congliang Chen, Ziniu Li, Tian Ding, Chenwei Wu, Yinyu Ye, Zhi-Quan Luo, and Ruoyu Sun.  Adam-mini: Use fewer learning rates to gain more.  _arXiv preprint arXiv:2406.16793_ , 2024.

##  Appendix A Appendix


# Renormalons beyond the Borel plane

E. Cavalcanti<EMAIL_ADDRESS>Centro Brasileiro de Pesquisas Físicas/MCTI,
22290-180 Rio de Janeiro, RJ, Brazil

###### Abstract

The renormalon singularities are a known source of the divergent behavior of
asymptotic perturbative series from field theoretical models. These
singularities live in the Borel plane and are responsible for ambiguities in
the physical plane. We propose that field theories can have renormalons beyond
the usual first Borel plane. We show an example with a scalar field theory
where, considering a chain of cat’s eyes diagrams, the model develops a
Gevrey-3 asymptotic series.

##  I Introduction

The renormalon phenomenon is known since the ‘70s; it was initially proposed
by ’t Hooft, who noted it as one source of divergence for the perturbative
asymptotic series of quantum field theories. The renormalon shows up as a
singularity in the Borel plane. Initially, the exact definition of the
renormalon singularity was a matter of discussion, with some debate whether it
was a simple pole, a $(n)$-pole, or rather a branch cut on the Borel plane
Beneke (1999). Although new perspectives have draw attention in the past two
decades Loewe and Valenzuela (2000); Kneur and Reynaud (2003); Guralnik and
Guralnik (2010); Unsal and Yaffe (2008); Argyres and Unsal (2012a, b); Dunne
and Unsal (2012); Dunne and Ünsal (2013); Poppitz  _et al._ (2013); Cherman
_et al._ (2014); Shifman (2015a); Dabrowski and Dunne (2013); Cherman  _et
al._ (2015); Shifman (2015b); Anber and Sulejmanpasic (2015); Sulejmanpasic
and Ünsal (2018); Sulejmanpasic (2017); Dunne and Ünsal (2016); Buividovich
and Valgushev (2017); Caprini  _et al._ (2017); Aniceto  _et al._ (2019);
Cherman  _et al._ (2019); Aniceto  _et al._ (2019); Cherman  _et al._ (2019);
Maiezza and Vasquez (2018); Cavalcanti  _et al._ (2018); Mariño and Reis
(2019, 2020a, 2020b); Correa  _et al._ (2019); Ishikawa  _et al._ (2020a, b);
Ashie  _et al._ (2020a); Correa  _et al._ (2019); Borinsky and Dunne (2020);
Cavalcanti (2020); Dondi  _et al._ (2020); Ashie  _et al._ (2020b); Borinsky
and Dunne (2020); Ünsal (2020); Costin and Dunne (2020), the seminal report
from Beneke, see ref. Beneke (1999), is still a useful reference on state of
the art regarding the formal discussion.

Nowadays, the renormalon problem is understood as a component of the broader
context of the resurgence program Anber and Sulejmanpasic (2015); Dunne and
Ünsal (2016); Maiezza and Vasquez (2018); Mariño and Reis (2019, 2020a,
2020b); Ishikawa  _et al._ (2020a); Dondi  _et al._ (2020). The singularity is
an indication that the original perturbative series requires a transseries
contribution. Also, the renormalon singularity, which arose as a formal aspect
from quantum field theory, is seen as a useful guide to phenomenological
predictions Cvetič (2020); Hadjimichef  _et al._ (2020); Takaura (2019);
Hayashi and Sumino (2019); Corcella (2019); Maiezza and Vasquez (2019); Nason
(2019a); Cvetič (2019); Ortega  _et al._ (2018); Kataev and Molokoedov
(2018a); Mateu  _et al._ (2019); Boito  _et al._ (2019); Ferrario Ravasio  _et
al._ (2019); Brambilla  _et al._ (2018a); Braun  _et al._ (2019); Kataev and
Molokoedov (2018b); Bell  _et al._ (2019); Takaura  _et al._ (2019a, b);
Suzuki and Takaura (2019); Del Debbio  _et al._ (2018); Kataev and Molokoedov
(2018c); Peset  _et al._ (2018); Bazavov  _et al._ (2018); Brambilla  _et al._
(2018b); Nason (2019b); Ortega and Mateu (2018); Mateu and Ortega (2018);
Hoang  _et al._ (2017); Ahmadov  _et al._ (2017); Steinhauser (2016); Jamin
and Miravitllas (2016); Ayala  _et al._ (2016); Mishima  _et al._ (2017).

It is usual in the literature to consider a large-$N$ expansion when one is
interested in exploring the existence of renormalons inside the theory. That
occurs because, in this limit, the relevant contributions are planar diagrams,
more commonly chains of bubble diagrams. For example, for quantum
chromodynamics (QCD) this produces the usual Adler function. However, the
large-$N$ expansion hides other singularities, as the instanton problem,
because the poles get far away from the origin in this approximation. Due to
historical reasons – as pointed by ref. Beneke (1999) – it is very common to
relate renormalons and bubble-chain diagrams, although the renormalon
singularity might appear at different subsets of Feynman diagrams. Once we
consider a quantum field theory in general (outside the large $N$
approximation), we have no guarantee that the usual planar diagrams are the
most relevant ones, and other scenarios might appear.

Another common relationship is that the renormalon singularities live in the
Borel plane. To our knowledge, every report so far is still considering that
renormalons are singularities that lives in the first Borel plane and produce
a Gevre-1 asymptotic series Marino and Reis (2020); Mariño and Reis (2020b,
a); Ishikawa  _et al._ (2020a, b); Cavalcanti  _et al._ (2018); Cavalcanti
(2020); Ashie  _et al._ (2020b). Here we propose a new perspective, where we
suppose a generic chain and show that new kinds of singularities occur that
might live in a $q$-Borel plane instead of the typical Gevrey-1 case of the
Borel plane. Therefore, we propose an extension on the definition of the
renormalon singularity, asserting that they live in a Gevrey-$q$ plane, thus
extending the Gevrey-order of the perturbative series.

We aim to identify scenarios where a renormalon singularity might occur.
Notice that we do not intend to discuss at which order in some perturbative
expansions these contributions appear. The relevant point here is that the
contributions considered are subsets of the full perturbative expansion and
shall appear. Following this chain of thought, we also do not apply any
procedure to circumvent the renormalon problem and give the nonperturbative
solution, as we are mostly interested in studying the existence or not of
renormalons singularities.

As our main interest lies in the formal aspect of quantum field theory, it
suffices to consider as a toy model a massless scalar field theory. This model
is known to possess ultraviolet (UV) renormalons produced by a sum of bubble-
chain diagrams, and – more importantly – we show that this model reveals one
novel scenario with cat’s eye chain diagrams. The sum over these chains
produces a Gevrey-3 series, therefore extending the definition of the
renormalon singularity beyond the usual Borel plane.

##  II $q$-Borel series

In many cases, perturbative series diverge and only makes sense as an
asymptotic series. A rather simple example of divergence occurs when the
perturbative series has a factorial growth, as

$A\sim\sum_{k=0}^{\infty}(k!)^{q}(s)^{k}g^{k},$ (1)

where $g$ is the expansion parameter (the coupling constant), $q$ is the power
of the factorial growth, and $s$ is the sign ($s=\pm 1$) of the coefficients.

A well-stablished procedure is to consider an extension of the Borel sum
Bender and Orszag (2013). The usual Borel sum occurs for $q=1$ and relates to
the conventional asymptotic power series (in the sense of Poincaré). In this
extension we consider $q$ factorials to control the growth, which is well
established in the studies of $q$-summability and Gevrey-$q$ series. First we
transport the usual sum to a summation in the $q$-Borel plane

$A\rightarrow B_{q}[A](u)\sim\sum_{k=0}^{\infty}(su)^{k}=\frac{1}{1-su}.$ (2)

Then, if this new infinite sum is summable in the $q$-Borel plane we proceed
to implement a $q$-Borel inverse transform

$\widetilde{A}[g]=\int_{0}^{\infty}du_{1}\ldots
du_{q}e^{-(u_{1}+\cdots+u_{q})}B_{q}[A](gu_{1}\ldots u_{q}).$ (3)

In the following, we denote the physical plane as 0-Borel (Gevrey-0), the
usual Borel plane as 1-Borel (Gevrey-1), and so on.

Let us consider, as an example, $q=1$ and $q=2$ in the most simple scenario.
For $s=-1$

$\displaystyle-\frac{e^{\frac{1}{g}}}{g}\text{Ei}\left(-\frac{1}{g}\right),$
$\displaystyle\quad q=1;$
$\displaystyle\frac{\pi}{g}\text{G}^{1,1}_{3,1}\left\lparen\,-\frac{1}{g}\middle|\begin{smallmatrix}0,\frac{1}{2}\\\
0,0,0,\frac{1}{2}\end{smallmatrix}\right\rparen,$ $\displaystyle\quad q=2;$

where Ei is the exponential integral function and $\text{G}^{mn}_{pq}$ is the
Meijer-$G$ function. Therefore, we can relate the asymptotic sum to a well-
defined function through $q$-summability.

On the other hand, if we take $s=1$, we find a pole in the positive real axis
of the $q$-Borel plane. This singularity introduces an ambiguity due to the
choice of the integration path.

Few known physical models produce a Gevrey-$q$ series and discuss
$q$-summability. For example, a scenario with Gevrey-2 is the sextic
anharmonic oscillator, while for Gevrey-3 there is the octic anharmonic
oscillator Shalaby (2020); Weniger (1996).

In sec. III, we propose with some degree of generality that there can be
Gevrey-$q$ series in field theoretical models. This is shown both for UV and
IR renormalons. In sec. IV, we take a toy model and show explicitly the
existence of a Gevrey-3 series when considering the subset of cat’s eye chain
diagrams.

##  III Renormalons from chains

Renormalons are singularities that arise due to low/large momenta of
integration (IR/UV renormalons) in the summing of a particular subset of
Feynman diagrams. The usual understanding is that they live in the first Borel
plane. In this section, we argue that they can occur in the $q$-Borel plane,
meaning that we are not considering the Gevrey-1 asymptotic series as usual,
but that the asymptotic series can be of the Gevrey-$q$ type.

In the following we consider the sum over a subset of Feynman diagrams that
produce a “chain”, $R_{k}(\alpha)$ (fig. 2). The chain is built by the
successive introduction of “chain-links”, $g(\ell)$ (fig. 1). The chain link
is some simple structure as a bubble diagram, a sunset diagram, a cat’s eye
diagram or other possibilities. Formally, we can write the sum over all chain
diagrams of some particular chain link as

$R(\alpha)=\sum_{k}\int d\ell f(\ell)\left[\alpha g(\ell)\right]^{k},$ (4)

where $\alpha$ is related to the coupling of the theory, and $f(\ell)$
contains all additional contribution for the chain diagram. The momentum
$\ell$ is integrated over the whole chain.

This representation is very general and contains all possibilities for a one-
chain scenario. We remark that we do not deal here with the multi-chain
scenario.

To determine the structure of $R(\alpha)$ we are mainly interested in the
asymptotic behavior with respect fo the internal momentum $\ell$ (this
translates to consider the large $k$ behavior. The behavior of both $f(\ell)$
and $g(\ell)$ depends on the particular scenario under interest. However, it
is known from the expression of Feynman amplitudes in the complete Mellin
representation Bergere  _et al._ (1978); Smirnov (1995); Linhares  _et al._
(2008), that the asymptotic expansion with respect to the external momentum
behaves as

$I_{G}\sim\ell^{p}\ln^{q}\ell,$ (5)

where $p\in\mathbb{Z}$, $q\in\mathbb{Z}^{+}$.

Therefore, we can say without loss of generality111In a recent article, ref.
Mariño and Reis (2020b) considered a scenario that behaves as $\ln x/(x+1)$.
That produces renormalon poles in scenarios prohibited here. However, this
expression is “unstable”, a small perturbation in the asymptotic approximation
($\ln x\rightarrow\ln 1+x$) destroys the renormalon pole. that both $f(\ell)$
and $g(\ell)$ behaves asymptotically as a transmonomial222The transmonomial is
an extension of the usual monomial structure and is related to the definition
of a transseries. $\ell^{p}\ln^{q}\ell$.

###  III.1 UV renormalons

We have to split the investigation into the IR and UV scales, as the
asympototic behavior is different in each case. First, let us consider the UV
scale

$\displaystyle f(\ell)$ $\displaystyle=\ell^{-a}\ln^{b}\ell,\quad
g(\ell)=\ell^{-c}\ln^{d}\ell,$ (6a) $\displaystyle R_{UV}(\alpha)$
$\displaystyle\sim\sum_{k}\int_{1}^{\infty}d\ell f(\ell)\left[\alpha
g(\ell)\right]^{k}$
$\displaystyle=\sum_{k}\alpha^{k}\int_{1}^{\infty}d\ell\;\ell^{-a-ck}\ln^{b+dk}\ell,$
(6b)

here $a\geq 0,c\geq 0$ such that the integration does not require any new
subtraction and is well behaved for $\ell\rightarrow\infty$.

Making the change of variables $\ell=e^{t}$ and identifying the gamma
function,

$R_{UV}(\alpha)\sim\sum_{k}\alpha^{k}\frac{\Gamma(dk+b+1)}{(a+ck-1)^{dk+b+1}}.$
(7)

We obtain the large $k$ behavior using the Stirling approximation for the
gamma function,

$R_{UV}(\alpha)\sim\sum_{k}\alpha^{k}\left(\frac{dk+b}{e(a+ck-1)}\right)^{dk+b}\frac{\sqrt{2\pi(dk+b)}}{a+ck-1}.$

In the scenario with $c\neq 0$ the $k^{k}$ behaviour related to the factorial
growth is cancelled out and the function is well defined,

$R_{UV}(\alpha)\sim\sum_{k}\alpha^{k}\left(\frac{d}{ec}\right)^{dk+b}\frac{\sqrt{2\pi
d}}{c\sqrt{k}}\\\ =\frac{\sqrt{2\pi
d}}{c}\left(\frac{d}{ec}\right)^{b}\text{Li}_{1/2}\left(\left(\frac{d}{ec}\right)^{d}\alpha\right).$

The relevant scenario occurs only if $c=0$ and $d>1$, at large $k$ we obtain

$R_{UV}(\alpha)\sim\frac{1}{(2\pi)^{\frac{d-1}{2}}}\frac{d^{b+\frac{1}{2}}}{(a-1)^{b+1}e^{b}}\times\\\
\sum_{k}\frac{k^{b}k!^{d}}{(2\pi
k)^{\frac{d-1}{2}}}\left[\alpha\left(\frac{d}{a-1}\right)^{d}\right]^{k}.$ (8)

This expression reveals that in the UV regime the sum over the set of chain
diagrams behaves as a Gevrey-$d$ series. The special case of $d=1$ produces
back the usual Grevrey-1 scenario of the Borel sum. In this approximation the
sum of chains is $d$-Borel summable and produces a polylogarithm of order
$(d-1)/2-b$,

$R_{UV}(\alpha)\rightarrow
B[R_{UV}](u)\sim\frac{1}{(2\pi)^{\frac{d-1}{2}}}\frac{d^{b+\frac{1}{2}}}{(a-1)^{b+1}e^{b}}\times\\\
\sum_{k}\left[\left(\frac{d}{a-1}\right)^{d}\text{sign}(\alpha)u\right]^{k}\frac{k^{b}}{k^{\frac{d-1}{2}}}\\\
=\frac{d^{b+\frac{1}{2}}\text{Li}_{\frac{d-1}{2}-b}\left[\left(\frac{d}{a-1}\right)^{d}\text{sign}(\alpha)u\right]}{(2\pi)^{\frac{d-1}{2}}(a-1)^{b+1}e^{b}}.$
(9)

The polylogarithm function of order $\nu$, $\text{Li}_{\nu}(x)$, has a branch
point at $x=1$ if $\nu>0$, or a pole of order $1-\nu$ if $\nu\leq 0$.
Therefore, if $\frac{d-1}{2}-b>0$ we have a branch point at
$u=u_{0}=\text{sign}(\alpha)\left(\frac{a-1}{d}\right)^{d}$. And, if
$\frac{d-1}{2}-b\leq 0$ this point is a pole of order $1+b+\frac{1-d}{2}$.

For example, at perturbative QCD, see ref. Beneke (1999), $\alpha=-|\alpha|$
and in UV region (for the chain of fermionic bubbles at $1/N_{f}$, where
$N_{f}$ is the number of flavors) we have $a=2,b=1,c=0,d=1$ This means that
$\nu=1$ and we obtain a Gevrey-1 UV renormalon located at $u=-1$ as a double
pole (as expected).

###  III.2 IR renormalons

To take into account the IR scale we use a slightly different choice of
$f(\ell)$ and $g(\ell)$

$\displaystyle f(\ell)$ $\displaystyle=\ell^{a}\ln^{b}\ell,\quad
g(\ell)=\ell^{c}\ln^{d}\ell,$ (10a) $\displaystyle R_{IR}(\alpha)$
$\displaystyle\sim\sum_{k}\int_{0}^{1}d\ell\;f(\ell)\left[\alpha
g(\ell)\right]^{k}$
$\displaystyle=\sum_{k}\alpha^{k}\int_{0}^{1}d\ell\;\ell^{a+ck}\ln^{b+dk}\ell.$
(10b)

Note that the power dependence with $\ell$ is in the numerator instead of the
denominator as in the UV scale. This choice is to guarantee that $R_{IR}$ is
well behaved in the IR scale and does not require any new subtraction.

We can employ the substitution $\ell=1/v$,

$R_{IR}(\alpha)\sim\sum_{k}\alpha^{k}\int_{1}^{\infty}dv\;\frac{1}{v^{2}}v^{-a-ck}(-\ln
v)^{b+dk}\\\
=\sum_{k}(-1)^{b}\left((-1)^{d}\alpha\right)^{k}\int_{1}^{\infty}dv\;\frac{1}{v^{2}}v^{-a-ck}(\ln
v)^{b+dk},$

the integral assumes the same form as $R_{UV}$, see eq. (6b), with the little
shift $a\rightarrow a+2$. Following the same steps as before we obtain that –
requiring $c=0$ and $d>1$ – in the IR regime the sum of chains also behaves as
a Gevrey-$d$ series. At large $k$ it produces

$R_{IR}(\alpha)\sim\frac{1}{(2\pi)^{\frac{d-1}{2}}}\frac{(-1)^{b}d^{b+\frac{1}{2}}}{(a+1)^{b+1}e^{b}}\times\\\
\sum_{k}\frac{k^{b}k!^{d}}{(2\pi
k)^{\frac{d-1}{2}}}\left[\alpha\left(-\frac{d}{a+1}\right)^{d}\right]^{k},$
(11)

which is $d$-Borel summable, producing a polylogarithm of order $(d-1)/2-b$
with a singularity at

$u=\left(-\text{sign}(\alpha)\frac{a+1}{d}\right)^{d},$

once again this singularity is a pole of order $1+b+(1-d)/2$ in the scenario
where $(d-1)/2-b\leq 0$, or a branch point if $(d-1)/2-b>0$.

At perturbative QCD, see ref. Beneke (1999), $\alpha=-|\alpha|$ and in IR
region we have $a=1,b=0,c=0,d=1$ for the chain of fermionic bubbles at
$1/N_{f}$, where $N_{f}$ is the number of flavors. Also $\nu=1$, so $u=2$ is
the first IR renormalon and is a simple pole (as expected).

###  III.3 Discussion

In the last sections, we indicate that the existence of renormalons is
directly related to the logarithm behavior of the chain links. This
relationship is commonly assumed, although not explicitly formulated.
Therefore we propose to conjecture that:

  * •

the sum of a set of chains can have a renormalon only if the chain-link
behaves as $\ln\ell$.

About the renormalon singularity, we also claim that:

  * •

The renormalon “lives” in the $d$-Borel plane, where $\ln^{d}\ell$ is the
behavior of each “insertion”/chain-link.

  * •

The renormalon is a pole of order $1+b+(1-d)/2$, where $b$ is the logarithm
contribution, $\ln^{b}\ell$, outside the insertions. It can also be a branch
point if $b+(1-d)/2<0$.

  * •

The location of the UV/IR pole depends only on $a$ and $d$, where $a$ is the
power contribution ($\ell^{a}$ for IR renormalons, $\ell^{-a}$ for UV
renormalons) outside the insertions.

With this, it becomes clear that renormalons do not need to be singularities
in the first Borel plane; this is just a particular case.

In what follows, we exhibit a simple example where a Gevrey-3 renormalon
appears.

##  IV Cat’s eye chain

Let us consider a flavorless and massless scalar field theory with quartic
interaction $\lambda\phi^{4}/4!$. We propose to build a chain diagram where
each chain link is given by a cat’s eye diagram. Each insertion is

$G(\ell)=(-g)^{3}I(\ell),$ (12)

Figure 1: Eye-cat–link diagram

where $I(\ell)$ is the cat’s eye diagram, see fig. 1, whose amplitude is given
by

$I(\ell)=\int\frac{d^{D}q_{1}}{(2\pi)^{D}}\int\frac{d^{D}q_{2}}{(2\pi)^{D}}\int\frac{d^{D}q_{3}}{(2\pi)^{D}}\frac{1}{(q_{1}+\ell)^{2}}\frac{1}{q_{1}^{2}}\times\\\
\frac{1}{(q_{2}+\ell)^{2}}\frac{1}{q_{2}^{2}}\frac{1}{(q_{3}+q_{1}-q_{2})^{2}}\frac{1}{q_{3}^{2}}.$
(13)

After some manipulations, see app. A, we can obtain that the solution of cat’s
eye diagram at $D=4-2\varepsilon$ is

$I(\ell)=\frac{\Gamma\left[3\varepsilon\right]}{(4\pi)^{6-3\varepsilon}\ell^{6\varepsilon}}\Bigg{(}-c_{1}+\frac{1}{8\varepsilon^{2}}+\frac{\pi^{2}}{12}+\frac{\ln
2-\gamma}{4\varepsilon}\\\ +\frac{\gamma^{2}}{2}-\gamma\ln 2\Bigg{)},$ (14a)
with $c_{1}=0.822\,467\,029\,8\ldots$ (14b)

Or, after applying a minimal subtraction scheme and keeping only the finite
components

$\widehat{I}(\ell)=a_{0}+a_{1}\ln\ell^{2}+a_{2}\ln^{2}\ell^{2}+a_{3}\ln^{3}\ell^{2},$
(15a) with constants $\displaystyle a_{3}=-4.761\,481\,354\times 10^{-8},$
(15b) $\displaystyle a_{2}=2.901\,308\,090\times 10^{-7},$ (15c)
$\displaystyle a_{1}=-7.640\,504\,523\times 10^{-7},$ (15d) $\displaystyle
a_{0}=6.440\,917\,284\times 10^{-7}.$ (15e)

We can already expect that this will generate a Gevrey-3 series due to the
$\ln^{3}$ behavior.

Figure 2: Chain of cat’s eye organized as a correction to the mass.

We produce the chain diagram by adding the chain-links. At fig. 2 we show the
scenario with $k$ chain links. The amplitude, before regularization, is

$R_{k}(p)=-gS_{k}\int\frac{d^{D}\ell}{(2\pi)^{D}}\frac{(-g^{3}{I}(\ell))^{k}}{(p-\ell)^{2}},$
(16)

where $S_{k}=(k+1)2^{k}$ is the symmetry factor for the diagram with $k$ eye-
cat–links. We calculate the symmetry factor using a general expression for
symmetry factors of the scalar theory, by ref. Palmer and Carrington (2002).

At $D=4$ the function $I$ behaves asymptotically as a logarithm and by power
counting $R_{k}$ has a quadratic divergence. We can make a subtraction as a
BPHZ procedure ($R_{k}(p)-R_{k}(0)-p^{2}\partial_{p^{2}}R_{k}(p)|_{0}$), which
produces at low $p$

$\widehat{R}_{k}(p)\sim-\frac{gp^{4}}{8\pi^{2}}(k+1)(-2g^{3})^{k}\int_{0}^{\infty}\frac{d\ell}{\ell^{3}}\;\widehat{I}^{k}(\ell).$
(17)

To investigate the asymptotic UV behavior, we can consider $\ell\in[1,\infty[$
and make the change of variables $\ell=e^{t}$ such that,

$\widehat{R}_{k}(p)\sim-\frac{gp^{4}}{8\pi^{2}}(k+1)(-2g^{3})^{k}\times\\\
\int_{0}^{\infty}dt\;e^{-2t}(a_{0}+2a_{1}t+4a_{2}t^{2}+8a_{3}t^{3})^{k}.$ (18)

As $k\in\mathbb{N}$ we can use the multinomial representation and integrate
over $t$,

$\widehat{R}_{k}(p)\sim-\frac{gp^{4}}{16\pi^{2}}(k+1)(-2g^{3})^{k}\times\\\
\sum_{j_{0}+j_{1}+j_{2}+j_{3}=k}a_{0}^{j_{0}}a_{1}^{j_{1}}a_{2}^{j_{2}}a_{3}^{j_{3}}\frac{k!\Gamma\left(1+j_{1}+2j_{2}+3j_{3}\right)}{j_{0}!j_{1}!j_{2}!j_{3}!}.$
(19)

The dominant contribution comes from $\mathbf{j}=(0,0,0,k)$ (accuracy of
$70\%$, it can be checked numerically for large value of $k$). The second
contribution comes from $\mathbf{j}=(0,0,1,k-1)$ (with an accuracy of more
than $90\%$). That way, we can write

$\widehat{R}_{k}(p)\sim-\frac{gp^{4}}{16\pi^{2}}\left(1+\frac{a_{2}}{3a_{3}}\right)(-2g^{3}a_{3})^{k}\times\\\
(k+1)\Gamma\left(1+3k\right),$ (20)

And it is clear that these coefficients produce a divergent series. Taking the
Stirling approximation for large values of $k$ the sum over all cat’s eye
chains produce

$R\sim\sum_{k=1}^{\infty}\widehat{R}_{k}(p)\sim\kappa\sum_{k=1}^{\infty}\left(\frac{\widetilde{g}}{e^{3}}\right)^{k}k^{3k+\frac{3}{2}},$
(21a) with $\kappa$ and $\widetilde{g}$ given by $\displaystyle\kappa$
$\displaystyle=-\sqrt{6\pi}\frac{gp^{4}}{16\pi^{2}}\left(1+\frac{a_{2}}{3a_{3}}\right),$
(21b) $\displaystyle\widetilde{g}$ $\displaystyle=-2\times 3^{3}g^{3}a_{3}.$
(21c)

This expression for $R$ is not summable in the first Borel plane. The Borel
transform introduces a $k!$ damping that behaves just as $k^{k}$. We can
control the divergent behavior only with a 3-Borel transform, that introduces
a $k!^{3}$, meaning a $k^{3k}$ damping, sufficient to obtain a summable
expression. Therefore, this is a Gevrey-3 asymptotic series.

If we transport eq. (20) into the 3-Borel plane we obtain a hypergeometric
function,

$G_{R}\sim-\frac{gp^{4}}{16\pi^{2}}\left(1+\frac{a_{2}}{3a_{3}}\right)\Bigg{[}-1+{}_{2}F_{1}\left(\frac{1}{3},\frac{2}{3};1;\widetilde{g}u\right)\\\
+\frac{2\widetilde{g}u}{9}{}_{2}F_{1}\left(\frac{4}{3},\frac{5}{3};2;\widetilde{g}u\right)\Bigg{]},$
(22)

it has a branch point at $u=1/\widetilde{g}$. This cut lies in the integration
path of the 3-Borel plane, therefore we have a singularity that blocks the
inverse Borel transform.

If we try to get back into the physical plane we obtain an imaginary ambiguity
due to the choice of the integration path from the 3-Borel to the 2-Borel
plane. However, there are no new poles in the 2-Borel plane, neither in the
1-Borel plane. Meaning that the only ambiguity appears when passing the
Gevrey-3 series to the 2-Borel plane.

The fact that the only singularity comes from the 3-Borel plane is a
consequence of the previous approximation to consider the most dominant
divergent behavior. In the full scenario, one would expect renormalon
singularities at Gevrey-1,2 and 3. Note, for example, that at eq. (19) one of
the contributions is $j_{0}=j_{1}=j_{3}=0,j_{2}=k$ which produces a Gevrey-2
series,

$\sum_{k=1}^{\infty}(k+1)(-2g^{3}a_{2})^{k}\Gamma(1+2k),$

This new series has a branch point in the 2-Borel plane located at
$u=-1/(8a_{2}g^{3})<0$, which does not lie in the integration path. If we keep
investigating, we find other singularities produced by eq. (19). In fact, the
sum over the cat’s eye chain diagrams produces more then one asymptotic
series, each with a different Gevrey order. However, the relevant behavior is
already revealed: the highest degree is Gevrey-3 and that there is a
renormalon singularity that lives in the 3-Borel plane.

##  V Conclusion

We exhibited that the renormalon problem goes beyond the first Borel plane. It
seems that the renormalon singularities are in some different category when
compared to the instanton-like singularities, only known to occur at the
Gevrey-1 level. Although we did not discuss a realistic theory nor considered
the phenomenological influence, it is beyond doubt that we have a new
singularity that lives in the 3-Borel plane. We remark that our report poses a
new difficulty for the program to cure all renormalon ambiguities.
Furthermore, there are some aspects open to future investigations. We need to
determine how the Gevrey-$q$ renormalon impacts the asymptotic series or the
resurgence program. Also, we must check whether a Gevrey-$d$ series (with
$d>1$) also appear for realistic theories.

###### Acknowledgements.

The author thanks the Brazilian agency Conselho Nacional de Desenvolvimento
Científico e Tecnológico (CNPq) for financial support.

##  Appendix A Cat’s eye diagram

In this section, we consider for completeness a step-by-step computation of
the cat’s eye diagram, as this evaluation is not easily found in textbooks.

Let us start from the original amplitude,

$I(\ell)=\int\frac{d^{D}q_{1}}{(2\pi)^{D}}\int\frac{d^{D}q_{2}}{(2\pi)^{D}}\int\frac{d^{D}q_{3}}{(2\pi)^{D}}\frac{1}{(q_{1}+\ell)^{2}}\frac{1}{q_{1}^{2}}\times\\\
\frac{1}{(q_{2}+\ell)^{2}}\frac{1}{q_{2}^{2}}\frac{1}{(q_{3}+q_{1}-q_{2})^{2}}\frac{1}{q_{3}^{2}},$
(23)

and then introduce Schwinger parameters $\alpha_{i}$ for each propagator,

$I(\ell)=\int\frac{d^{D}q_{1}}{(2\pi)^{D}}\int\frac{d^{D}q_{2}}{(2\pi)^{D}}\int\frac{d^{D}q_{3}}{(2\pi)^{D}}\int_{0}^{\infty}\prod_{i=1}^{6}d\alpha_{i}\\\
e^{-\alpha_{1}(q_{1}+\ell)^{2}-\alpha_{2}q_{1}^{2}}e^{-\alpha_{3}(q_{2}+\ell)^{2}-\alpha_{4}q_{2}^{2}}e^{-\alpha_{5}(q_{3}+q_{1}-q_{2})^{2}-\alpha_{6}q_{3}^{2}}.$
(24)

After some algebraic manipulations on these expressions, we obtain that

$I(\ell)=\int\frac{d^{D}q_{1}}{(2\pi)^{D}}\int\frac{d^{D}q_{2}}{(2\pi)^{D}}\int\frac{d^{D}q_{3}}{(2\pi)^{D}}\int_{0}^{\infty}\prod_{i=1}^{6}d\alpha_{i}\\\
e^{-\alpha_{125}\left(q_{1}+\frac{\ell\alpha_{1}+q_{3}\alpha_{5}}{\alpha_{125}}\right)^{2}}e^{-\alpha_{346}\left(q_{2}+\frac{\ell\alpha_{3}+q_{3}\alpha_{6}}{\alpha_{346}}\right)^{2}}\times\\\
e^{-\frac{U}{\alpha_{125}\alpha_{346}}\left[q_{3}-\ell\frac{\alpha_{1}\alpha_{5}\alpha_{346}+\alpha_{3}\alpha_{6}\alpha_{125}}{U}\right]^{2}}e^{-\frac{V\ell^{2}}{U}},$
(25)

where

$\displaystyle\alpha_{i_{1}i_{2}\ldots i_{k}}$
$\displaystyle=\alpha_{i_{1}}+\alpha_{i_{2}}+\cdots+\alpha_{i_{k}},$ (26a)
$\displaystyle U$
$\displaystyle=\alpha_{12}\alpha_{5}\alpha_{346}+\alpha_{34}\alpha_{6}\alpha_{125},$
(26b) $\displaystyle V$
$\displaystyle=\alpha_{1}\alpha_{2}(\alpha_{34}\alpha_{56}+\alpha_{5}\alpha_{6})+\alpha_{3}\alpha_{4}(\alpha_{12}\alpha_{56}+\alpha_{5}\alpha_{6})$
$\displaystyle+\alpha_{5}\alpha_{6}(\alpha_{1}\alpha_{4}+\alpha_{2}\alpha_{3}).$
(26c)

We can make a shift in the momenta,

$I(\ell)=\int\frac{d^{D}q_{1}}{(2\pi)^{D}}\int\frac{d^{D}q_{2}}{(2\pi)^{D}}\int\frac{d^{D}q_{3}}{(2\pi)^{D}}\int_{0}^{\infty}\prod_{i=1}^{6}d\alpha_{i}\\\
e^{-\alpha_{125}q_{1}^{2}}e^{-\alpha_{346}q_{2}^{2}}e^{-\frac{U}{\alpha_{125}\alpha_{346}}q_{3}^{2}}e^{-\frac{V\ell^{2}}{U}},$
(27)

and then compute the integration of the internal momenta

$I(\ell)=\frac{1}{(4\pi)^{\frac{3D}{2}}}\int_{0}^{\infty}\prod_{i=1}^{6}d\alpha_{i}\frac{e^{-\ell^{2}\frac{V}{U}}}{U^{\frac{D}{2}}}.$
(28)

Let us reparametrize defining the new sectors

$\displaystyle\alpha_{1}$ $\displaystyle=st_{1}\cdots t_{5},$
$\displaystyle\alpha_{2}$ $\displaystyle=st_{1}\cdots t_{4}(1-t_{5}),$
$\displaystyle\alpha_{3}$ $\displaystyle=st_{1}t_{2}t_{3}(1-t_{4}),$
$\displaystyle\alpha_{4}$ $\displaystyle=st_{1}t_{2}(1-t_{3}),$
$\displaystyle\alpha_{5}$ $\displaystyle=st_{1}(1-t_{2}),$
$\displaystyle\alpha_{6}$ $\displaystyle=s(1-t_{1}),$

such that $\alpha_{1}+\ldots+\alpha_{6}=s$. This change of variables produce
the Jacobian

$d\alpha_{1}\cdots
d\alpha_{6}=s^{5}t_{1}^{4}t_{2}^{3}t_{3}^{2}t_{4}dsdt_{1}\cdots dt_{5}.$

and after some manipulation we obtain that

$\displaystyle\alpha_{12}$ $\displaystyle=st_{1}\cdots t_{4},$ (29a)
$\displaystyle\alpha_{34}$ $\displaystyle=st_{1}t_{2}(1-t_{3}t_{4}),$ (29b)
$\displaystyle\alpha_{125}$ $\displaystyle=st_{1}(1-t_{2}+t_{2}t_{3}t_{4}),$
(29c) $\displaystyle\alpha_{346}$
$\displaystyle=s\left(1-t_{1}+t_{1}t_{2}(1-t_{3}t_{4})\right),$ (29d)
$\displaystyle U$ $\displaystyle=s^{3}t_{1}^{2}t_{2}\overline{U},$ (29e)
$\displaystyle V$ $\displaystyle=s^{4}t_{1}^{3}t_{2}^{2}t_{3}\overline{V},$
(29f)

with $\overline{U}$ and $\overline{V}$ given by

$\overline{U}=(1-t_{2})t_{3}t_{4}(1-t_{1}+t_{1}t_{2}(1-t_{3}t_{4}))\\\
+(1-t_{2}+t_{2}t_{3}t_{4})(1-t_{3}t_{4})(1-t_{1}),$ (30)
$\overline{V}=(1-t_{1})(1-t_{2})(1-t_{4}+t_{4}t_{5})(1-t_{3}+t_{3}t_{4}(1-t_{5}))\\\
+t_{2}t_{3}t_{4}(1-t_{1}t_{2})\left((1-t_{3})(1-t_{4})+(1-t_{3}t_{4})(1-t_{5})t_{4}t_{5}\right)$
(31)

Substituting this back into the integral $I$, eq. (28), we obtain

$I(\ell)=\frac{1}{(4\pi)^{\frac{3D}{2}}}\int_{0}^{1}\prod_{i=1}^{5}dt_{i}\frac{t_{1}^{4-D}t_{2}^{3-\frac{D}{2}}t_{3}^{2}t_{4}}{\overline{U}^{\frac{D}{2}}(\boldsymbol{t})}\\\
\int_{0}^{\infty}ds\;s^{\left(6-\frac{3D}{2}\right)-1}e^{-st_{1}t_{2}t_{3}\ell^{2}\frac{\overline{V}}{\overline{U}}}.$
(32)

This expression can be directly integrated over $s$,

$I(\ell)=\frac{\Gamma\left[6-\frac{3D}{2}\right]}{(4\pi)^{\frac{3D}{2}}}\int_{0}^{1}\prod_{i=1}^{5}dt_{i}\frac{t_{1}^{-2+\frac{D}{2}}t_{2}^{-3+D}t_{3}^{-4+\frac{3D}{2}}t_{4}}{\overline{U}^{2D-6}\overline{V}^{6-\frac{3D}{2}}\ell^{12-3D}},$
(33)

and taking that $D=4-2\varepsilon$,

$I(\ell)=\frac{\Gamma\left[3\varepsilon\right]}{(4\pi)^{6-3\varepsilon}\ell^{6\varepsilon}}\int_{0}^{1}\prod_{i=1}^{5}dt_{i}\frac{t_{1}^{-\varepsilon}t_{2}^{1-2\varepsilon}t_{3}^{2-3\varepsilon}t_{4}}{\overline{U}^{2-4\varepsilon}\overline{V}^{3\varepsilon}}.$
(34)

Although at this point we do not know the behavior of the remaining integral
with respect to $\varepsilon$, we can already see that the pure logarithmic
behavior of $\ell$ can only occur at $\varepsilon=0$, which reinforces the
perception that the existence of renormalons is an aspect related to the
dimension where the theory is renormalizable.

Let us take $t_{1}^{-\varepsilon}\overline{V}^{-3\varepsilon}$ as a
subdominant contribution ($1+\mathcal{O}(\varepsilon)$). And as the polynomial
$\overline{U}$ is independent of $t_{5}$ this variable can be integrated out.
Also we can rewrite $\overline{U}$ as

$\overline{U}=(1-t_{2}+t_{2}t_{3}t_{4}(1-t_{3}t_{4}))\\\
-t_{1}(1-t_{2}+t_{2}^{2}t_{3}t_{4}(1-t_{3}t_{4}))=A_{1}-t_{1}A_{2},$ (35)

to integrate over $t_{1}$

$\int_{0}^{1}dt_{1}\frac{1}{(A_{1}-t_{1}A_{2})^{2-4\varepsilon}}=\int_{A_{1}}^{A_{1}-A_{2}}\frac{dy}{-A_{2}}\frac{1}{y^{2-4\varepsilon}}\\\
=\frac{1}{1-4\varepsilon}\left[\frac{1}{(A_{1}-A_{2})^{1-4\varepsilon}A_{2}}-\frac{1}{A_{1}^{1-4\varepsilon}A_{2}}\right].$
(36)

At this point, eq. (34) becames

$I(\ell)=\frac{\Gamma\left[3\varepsilon\right](1+\mathcal{O}(\varepsilon))}{(4\pi)^{6-3\varepsilon}\ell^{6\varepsilon}}J_{1}-\frac{\Gamma\left[3\varepsilon\right](1+\mathcal{O}(\varepsilon))}{(4\pi)^{6-3\varepsilon}\ell^{6\varepsilon}}J_{2},$
(37)

where $J_{1}$ and $J_{2}$ are given by

$J_{1}=\int_{0}^{1}dt_{2}dt_{3}dt_{4}\frac{t_{2}^{1-2\varepsilon}t_{3}^{2-3\varepsilon}t_{4}}{1-t_{2}+t_{2}^{2}t_{3}t_{4}(1-t_{3}t_{4})}\times\\\
\frac{1}{\left[t_{2}t_{3}t_{4}(1-t_{2})(1-t_{3}t_{4})\right]^{1-4\varepsilon}},$
(38)
$J_{2}=\int_{0}^{1}dt_{2}dt_{3}dt_{4}\frac{t_{2}^{1-2\varepsilon}t_{3}^{2-3\varepsilon}t_{4}}{1-t_{2}+t_{2}^{2}t_{3}t_{4}(1-t_{3}t_{4})}\times\\\
\frac{1}{\left[1-t_{2}+t_{2}t_{3}t_{4}(1-t_{3}t_{4})\right]^{1-4\varepsilon}}.$
(39)

The second integral, $J_{2}$, is finite and gives

$J_{2}=0.822\,467\,029\,8\ldots=c_{1}.$ (40)

The integral $J_{1}$ has a divergent contribution from the $t_{2}$
integration. We can reorganize it and compute the other integrals

$J_{1}=\int_{0}^{1}dt_{2}\frac{t_{2}^{2\varepsilon}}{(1-t_{2})^{1-4\varepsilon}}\int_{0}^{1}dt_{3}\int_{0}^{1}dt_{4}\times\\\
\frac{1}{1-t_{2}+t_{2}^{2}t_{3}t_{4}(1-t_{3}t_{4})}\frac{t_{3}}{(1-t_{3}t_{4})^{1-4\varepsilon}}\\\
=\int_{0}^{1}dt\,t^{2\varepsilon-1}(1-t)^{4\varepsilon-1}\frac{2\ln(1-t)}{t-2}.$

Note that we can rewrite it as a derivative of a function of $\mu$ and $\nu$
(and later identify $\mu=2\varepsilon$, $\nu=4\varepsilon$)

$\displaystyle
G(\mu,\nu)=-\int_{0}^{1}dt\,t^{\mu-1}(1-t)^{\nu-1}\left(1-\frac{t}{2}\right)^{-1},$
(41)
$\displaystyle\partial_{\nu}G(\mu,\nu)=\int_{0}^{1}dt\,t^{\mu-1}(1-t)^{\nu-1}\frac{2\ln(1-t)}{t-2}.$
(42)

Where is $G(\mu,\nu)$is known, see Eq. 3.197(3) from ref. Gradshteyn and
Ryzhik (2007),

$\displaystyle G(\mu,\nu)$
$\displaystyle=-B(\mu,\nu){}_{2}F_{1}\left(1,\mu;\mu+\nu;\frac{1}{2}\right)$
$\displaystyle=-\sum_{k=0}^{\infty}\frac{\Gamma(\mu+k)\Gamma(\nu)}{\Gamma(\mu+\nu+k)}\frac{1}{2^{k}},$
(43)

where $B(\mu,\nu)$ is the Beta function and ${}_{2}F_{1}$ is the
hypergeometric function. We use the representation as a infinite sum of gamma
functions, to make manipulations easier.

Making the derivative with respect to $\nu$ and substituting
$\mu=2\varepsilon,\nu=4\varepsilon$,

$J_{1}=G^{(0,1)}(2\varepsilon,4\varepsilon)=-\sum_{k=0}^{\infty}\frac{\Gamma(2\varepsilon+k)}{2^{k}\Gamma(6\varepsilon+k)}\Bigg{(}\Gamma^{\prime}(4\varepsilon)\\\
-\frac{\Gamma(4\varepsilon)\Gamma^{\prime}(6\varepsilon+k)}{\Gamma(6\varepsilon+k)}\Bigg{)}=\left(\frac{1}{16\varepsilon^{2}}+\frac{\pi^{2}}{6}\right)\\\
+\left(\frac{1}{16\varepsilon^{2}}-\frac{\pi^{2}}{12}-\frac{\gamma^{2}}{2}\right)\sum_{k=1}^{\infty}\frac{1}{2^{k}}+\left(\frac{1}{4\varepsilon}-\gamma\right)\sum_{k=1}^{\infty}\frac{\psi(k)}{2^{k}}\\\
=\frac{1}{8\varepsilon^{2}}+\frac{\pi^{2}}{12}+\frac{\ln
2-\gamma}{4\varepsilon}+\frac{\gamma^{2}}{2}-\gamma\ln 2.$ (44)

Finally, we obtain

$I(\ell)=\frac{\Gamma\left[3\varepsilon\right]}{(4\pi)^{6-3\varepsilon}\ell^{6\varepsilon}}\Bigg{(}-c_{1}+\frac{1}{8\varepsilon^{2}}+\frac{\pi^{2}}{12}+\frac{\ln
2-\gamma}{4\varepsilon}\\\ +\frac{\gamma^{2}}{2}-\gamma\ln 2\Bigg{)}.$ (45)

After a $\overline{MS}$ subtraction this becames

$\widehat{I}(\ell)=a_{0}+a_{1}\ln\ell^{2}+a_{2}\ln^{2}\ell^{2}+a_{3}\ln^{3}\ell^{2},$
(46a) where the constants are $\displaystyle
a_{3}=-\frac{3}{65536\pi^{6}}=-4.761\,481\,354\times 10^{-8},$ (46b)
$\displaystyle
a_{2}=\frac{-15\gamma+3\ln(4^{4}\pi^{3})}{65536\pi^{6}}=2.901\,308\,090\times
10^{-7},$ (46c) $\displaystyle a_{1}=-7.640\,504\,523\times 10^{-7},$ (46d)
$\displaystyle a_{0}=6.440\,917\,284\times 10^{-7}.$ (46e)

Therefore, the chain of cat’s eyes has the $\ln^{3}\ell$ behavior that can
produce a Gevrey-3 divergent series. This logarithmic behavior is a property
of $D=4$ and is responsible for the renormalon divergence. If we consider the
same diagram in any other dimension we will obtain something like
$\ell^{a}\ln^{b}\ell$ that does not produce a renormalon divergence.

## References

  * Beneke (1999) M. Beneke, Phys. Rept. 317, 1 (1999), arXiv:hep-ph/9807443 .
  * Loewe and Valenzuela (2000) M. Loewe and C. Valenzuela, Mod. Phys. Lett. A 15, 1181 (2000), arXiv:hep-th/9911151 .
  * Kneur and Reynaud (2003) J. Kneur and D. Reynaud, JHEP 01, 014 (2003), arXiv:hep-th/0111120 .
  * Guralnik and Guralnik (2010) G. Guralnik and Z. Guralnik, Annals Phys. 325, 2486 (2010), arXiv:0710.1256 [hep-th] .
  * Unsal and Yaffe (2008) M. Unsal and L. G. Yaffe, Phys. Rev. D 78, 065035 (2008), arXiv:0803.0344 [hep-th] .
  * Argyres and Unsal (2012a) P. C. Argyres and M. Unsal, JHEP 08, 063 (2012a), arXiv:1206.1890 [hep-th] .
  * Argyres and Unsal (2012b) P. Argyres and M. Unsal, Phys. Rev. Lett. 109, 121601 (2012b), arXiv:1204.1661 [hep-th] .
  * Dunne and Unsal (2012) G. V. Dunne and M. Unsal, JHEP 11, 170 (2012), arXiv:1210.2423 [hep-th] .
  * Dunne and Ünsal (2013) G. V. Dunne and M. Ünsal, Phys. Rev. D 87, 025015 (2013), arXiv:1210.3646 [hep-th] .
  * Poppitz  _et al._ (2013) E. Poppitz, T. Schäfer, and M. Ünsal, JHEP 03, 087 (2013), arXiv:1212.1238 [hep-th] .
  * Cherman  _et al._ (2014) A. Cherman, D. Dorigoni, G. V. Dunne, and M. Ünsal, Phys. Rev. Lett. 112, 021601 (2014), arXiv:1308.0127 [hep-th] .
  * Shifman (2015a) M. Shifman, Int. J. Mod. Phys. A 30, 1543001 (2015a), arXiv:1310.1966 [hep-th] .
  * Dabrowski and Dunne (2013) R. Dabrowski and G. V. Dunne, Phys. Rev. D 88, 025020 (2013), arXiv:1306.0921 [hep-th] .
  * Cherman  _et al._ (2015) A. Cherman, D. Dorigoni, and M. Unsal, JHEP 10, 056 (2015), arXiv:1403.1277 [hep-th] .
  * Shifman (2015b) M. Shifman, J. Exp. Theor. Phys. 120, 386 (2015b), arXiv:1411.4004 [hep-th] .
  * Anber and Sulejmanpasic (2015) M. M. Anber and T. Sulejmanpasic, JHEP 01, 139 (2015), arXiv:1410.0121 [hep-th] .
  * Sulejmanpasic and Ünsal (2018) T. Sulejmanpasic and M. Ünsal, Comput. Phys. Commun. 228, 273 (2018), arXiv:1608.08256 [hep-th] .
  * Sulejmanpasic (2017) T. Sulejmanpasic, Phys. Rev. Lett. 118, 011601 (2017), arXiv:1610.04009 [hep-th] .
  * Dunne and Ünsal (2016) G. V. Dunne and M. Ünsal, Ann. Rev. Nucl. Part. Sci. 66, 245 (2016), arXiv:1601.03414 [hep-th] .
  * Buividovich and Valgushev (2017) P. Buividovich and S. Valgushev, (2017), arXiv:1706.08954 [hep-lat] .
  * Caprini  _et al._ (2017) I. Caprini, J. Fischer, G. Abbas, and B. Ananthanarayan, (2017), arXiv:1711.04445 [hep-ph] .
  * Aniceto  _et al._ (2019) I. Aniceto, G. Basar, and R. Schiappa, Phys. Rept. 809, 1 (2019), arXiv:1802.10441 [hep-th] .
  * Cherman  _et al._ (2019) A. Cherman, M. Shifman, and M. Ünsal, Phys. Rev. D 99, 105001 (2019), arXiv:1812.04642 [hep-th] .
  * Maiezza and Vasquez (2018) A. Maiezza and J. C. Vasquez, Annals Phys. 394, 84 (2018), arXiv:1802.06022 [hep-th] .
  * Cavalcanti  _et al._ (2018) E. Cavalcanti, J. A. Lourenço, C. A. Linhares, and A. P. C. Malbouisson, Phys. Rev. D 98, 045013 (2018), arXiv:1804.10708 [hep-th] .
  * Mariño and Reis (2019) M. Mariño and T. Reis,  (2019), 10.1088/1742-5468/ab4802, arXiv:1905.09569 [hep-th] .
  * Mariño and Reis (2020a) M. Mariño and T. Reis, JHEP 04, 160 (2020a), arXiv:1909.12134 [hep-th] .
  * Mariño and Reis (2020b) M. Mariño and T. Reis, JHEP 07, 216 (2020b), arXiv:1912.06228 [hep-th] .
  * Correa  _et al._ (2019) M. Correa, M. Loewe, D. Valenzuela, and R. Zamora, Phys. Rev. D 99, 096024 (2019), arXiv:1901.06426 [hep-ph] .
  * Ishikawa  _et al._ (2020a) K. Ishikawa, O. Morikawa, K. Shibata, H. Suzuki, and H. Takaura, PTEP 2020, 013B01 (2020a), arXiv:1909.09579 [hep-th] .
  * Ishikawa  _et al._ (2020b) K. Ishikawa, O. Morikawa, A. Nakayama, K. Shibata, H. Suzuki, and H. Takaura, PTEP 2020, 023B10 (2020b), arXiv:1908.00373 [hep-th] .
  * Ashie  _et al._ (2020a) M. Ashie, O. Morikawa, H. Suzuki, H. Takaura, and K. Takeuchi, PTEP 2020, 023B01 (2020a), arXiv:1909.05489 [hep-th] .
  * Borinsky and Dunne (2020) M. Borinsky and G. V. Dunne, Nucl. Phys. B 957, 115096 (2020), arXiv:2005.04265 [hep-th] .
  * Cavalcanti (2020) E. Cavalcanti, (2020), arXiv:2011.04099 [hep-th] .
  * Dondi  _et al._ (2020) N. A. Dondi, G. V. Dunne, M. Reichert, and F. Sannino, Phys. Rev. D 102, 035005 (2020), arXiv:2003.08397 [hep-th] .
  * Ashie  _et al._ (2020b) M. Ashie, O. Morikawa, H. Suzuki, and H. Takaura, PTEP 2020, 093B02 (2020b), arXiv:2005.07407 [hep-th] .
  * Ünsal (2020) M. Ünsal, (2020), arXiv:2007.03880 [hep-th] .
  * Costin and Dunne (2020) O. Costin and G. V. Dunne, Phys. Lett. B 808, 135627 (2020), arXiv:2003.07451 [hep-th] .
  * Cvetič (2020) G. Cvetič, Nucl. Part. Phys. Proc. 309-311, 87 (2020), arXiv:1909.13593 [hep-ph] .
  * Hadjimichef  _et al._ (2020) D. Hadjimichef, E. Luna, and M. Peláez, Phys. Lett. B 804, 135350 (2020), arXiv:1907.07577 [hep-ph] .
  * Takaura (2019) H. Takaura, PoS ALPHAS2019, 025 (2019).
  * Hayashi and Sumino (2019) Y. Hayashi and Y. Sumino, Phys. Lett. B 795, 107 (2019), arXiv:1904.02563 [hep-ph] .
  * Corcella (2019) G. Corcella, Front. in Phys. 7, 54 (2019), arXiv:1903.06574 [hep-ph] .
  * Maiezza and Vasquez (2019) A. Maiezza and J. C. Vasquez, Annals Phys. 407, 78 (2019), arXiv:1902.05847 [hep-th] .
  * Nason (2019a) P. Nason, in  _11th International Workshop on Top Quark Physics_ (2019) arXiv:1901.04737 [hep-ph] .
  * Cvetič (2019) G. Cvetič, Phys. Rev. D 99, 014028 (2019), arXiv:1812.01580 [hep-ph] .
  * Ortega  _et al._ (2018) P. G. Ortega, V. Mateu, D. R. Entem, and F. Fernández, PoS Confinement2018, 121 (2018), arXiv:1811.12191 [hep-ph] .
  * Kataev and Molokoedov (2018a) A. Kataev and V. Molokoedov, JETP Lett. 108, 777 (2018a), arXiv:1811.02867 [hep-ph] .
  * Mateu  _et al._ (2019) V. Mateu, P. G. Ortega, D. R. Entem, and F. Fernández, Eur. Phys. J. C 79, 323 (2019), arXiv:1811.01982 [hep-ph] .
  * Boito  _et al._ (2019) D. Boito, P. Masjuan, and F. Oliani, SciPost Phys. Proc. 1, 049 (2019), arXiv:1811.01601 [hep-ph] .
  * Ferrario Ravasio  _et al._ (2019) S. Ferrario Ravasio, P. Nason, and C. Oleari, JHEP 01, 203 (2019), arXiv:1810.10931 [hep-ph] .
  * Brambilla  _et al._ (2018a) N. Brambilla, H. S. Chung, and J. Komijani, Phys. Rev. D 98, 114020 (2018a), arXiv:1810.02586 [hep-ph] .
  * Braun  _et al._ (2019) V. M. Braun, A. Vladimirov, and J.-H. Zhang, Phys. Rev. D 99, 014013 (2019), arXiv:1810.00048 [hep-ph] .
  * Kataev and Molokoedov (2018b) A. Kataev and V. Molokoedov, EPJ Web Conf. 191, 04005 (2018b), arXiv:1809.04395 [hep-ph] .
  * Bell  _et al._ (2019) G. Bell, A. Hornig, C. Lee, and J. Talbert, JHEP 01, 147 (2019), arXiv:1808.07867 [hep-ph] .
  * Takaura  _et al._ (2019a) H. Takaura, T. Kaneko, Y. Kiyo, and Y. Sumino, Phys. Lett. B 789, 598 (2019a), arXiv:1808.01632 [hep-ph] .
  * Takaura  _et al._ (2019b) H. Takaura, T. Kaneko, Y. Kiyo, and Y. Sumino, JHEP 04, 155 (2019b), arXiv:1808.01643 [hep-ph] .
  * Suzuki and Takaura (2019) H. Suzuki and H. Takaura, PTEP 2019, 103B04 (2019), arXiv:1807.10064 [hep-ph] .
  * Del Debbio  _et al._ (2018) L. Del Debbio, F. Di Renzo, and G. Filaci, Eur. Phys. J. C 78, 974 (2018), arXiv:1807.09518 [hep-lat] .
  * Kataev and Molokoedov (2018c) A. Kataev and V. Molokoedov, (2018c), arXiv:1807.05406 [hep-ph] .
  * Peset  _et al._ (2018) C. Peset, A. Pineda, and J. Segovia, JHEP 09, 167 (2018), arXiv:1806.05197 [hep-ph] .
  * Bazavov  _et al._ (2018) A. Bazavov _et al._ (Fermilab Lattice, MILC, TUMQCD), Phys. Rev. D 98, 054517 (2018), arXiv:1802.04248 [hep-lat] .
  * Brambilla  _et al._ (2018b) N. Brambilla, J. Komijani, A. Kronfeld, and A. Vairo (TUMQCD), Phys. Rev. D 97, 034503 (2018b), arXiv:1712.04983 [hep-ph] .
  * Nason (2019b) P. Nason, “The Top Mass in Hadronic Collisions,” in  _From My Vast Repertoire …: Guido Altarelli’s Legacy_, edited by A. Levy, S. Forte, and G. Ridolfi (2019) pp. 123–151, arXiv:1712.02796 [hep-ph] .
  * Ortega and Mateu (2018) P. G. Ortega and V. Mateu, PoS Hadron2017, 031 (2018), arXiv:1711.09998 [hep-ph] .
  * Mateu and Ortega (2018) V. Mateu and P. G. Ortega, JHEP 01, 122 (2018), arXiv:1711.05755 [hep-ph] .
  * Hoang  _et al._ (2017) A. H. Hoang, C. Lepenik, and M. Preisser, JHEP 09, 099 (2017), arXiv:1706.08526 [hep-ph] .
  * Ahmadov  _et al._ (2017) A. Ahmadov, C. Aydin, and O. Uzun, J. Phys. Conf. Ser. 804, 012001 (2017).
  * Steinhauser (2016) M. Steinhauser, PoS DIS2016, 148 (2016).
  * Jamin and Miravitllas (2016) M. Jamin and R. Miravitllas, JHEP 10, 059 (2016), arXiv:1606.06166 [hep-ph] .
  * Ayala  _et al._ (2016) C. Ayala, G. Cvetic, and A. Pineda, J. Phys. Conf. Ser. 762, 012063 (2016), arXiv:1606.01741 [hep-ph] .
  * Mishima  _et al._ (2017) G. Mishima, Y. Sumino, and H. Takaura, Phys. Rev. D 95, 114016 (2017), arXiv:1612.08711 [hep-ph] .
  * Marino and Reis (2020) M. Marino and T. Reis, (2020), arXiv:2006.05131 [hep-th] .
  * Bender and Orszag (2013) C. M. Bender and S. A. Orszag,  _Advanced mathematical methods for scientists and engineers I: Asymptotic methods and perturbation theory_ (Springer Science & Business Media, 2013).
  * Shalaby (2020) A. M. Shalaby, Phys. Rev. D 101, 105006 (2020), arXiv:2004.08711 [hep-th] .
  * Weniger (1996) E. J. Weniger, Ann. Phys. 246, 133 (1996).
  * Bergere  _et al._ (1978) M. Bergere, C. de Calan, and A. Malbouisson, Commun. Math. Phys. 62, 137 (1978).
  * Smirnov (1995) V. A. Smirnov, Mod. Phys. Lett. A 10, 1485 (1995), arXiv:hep-th/9412063 .
  * Linhares  _et al._ (2008) C. Linhares, A. Malbouisson, and I. Roditi, Rev. Math. Phys. 20, 933 (2008), arXiv:0711.4972 [hep-th] .
  * Palmer and Carrington (2002) C. Palmer and M. Carrington, Can. J. Phys. 80, 847 (2002), arXiv:hep-th/0108088 .
  * Gradshteyn and Ryzhik (2007) I. Gradshteyn and J. Ryzhik,  _Table of Integrals, Series, and Products_, 7th ed., edited by A. Jeffrey and D. Zwillinger (Elsevier, Amsterdam, 2007).


# Generalised Approximate Message Passing for Non-I.I.D. Sparse Signals

Christian Schou Oxvig1 and Thomas Arildsen2.
1 Formerly also at: 2 Aalborg University Technical Faculty of IT & Design
Department of Electronic Systems

###### Abstract

Generalised approximate message passing (GAMP) is an approximate Bayesian
estimation algorithm for signals observed through a linear transform with a
possibly non-linear subsequent measurement model. By leveraging prior
information about the observed signal, such as sparsity in a known dictionary,
GAMP can for example reconstruct signals from under-determined measurements –
known as compressed sensing. In the sparse signal setting, most existing
signal priors for GAMP assume the input signal to have i.i.d. entries. Here we
present sparse signal priors for GAMP to estimate non-i.d.d. signals through a
non-uniform weighting of the input prior, for example allowing GAMP to support
model-based compressed sensing.

##  1 Introduction

Generalised approximate message passing (GAMP) was introduced by [1] in [1,
2]. GAMP addresses the estimation of signals $\mathbf{x}$ observed through a
linear transform as follows

$\mathbf{y}=\mathbf{Ax}+\mathbf{e}$ (1)

where $\mathbf{y}\in\mathbb{C}^{m},\mathbf{A}\in\mathbb{C}^{m\times
n},\mathbf{x}\in\mathbb{C}^{n},\mathbf{e}\in\mathbb{C}^{m}$. Here we express
$\mathbf{e}$ as an additive noise which is classical in linear measurement
models. We define the intermediate measurements without noise:

$\mathbf{z}=\mathbf{Ax},\quad\mathbf{y}=\mathbf{z}+\mathbf{e}$ (2)

The noise term $\mathbf{e}$ does not have to be a strictly additive term
independent of $\mathbf{x}$. It can more generally be a (possibly non-linear)
separable measurement channel expressed through a p.d.f. of $\mathbf{y}$;
$p(\mathbf{y}|\mathbf{z};\bm{\theta}_{O})=\prod_{l=0}^{m-1}p(y_{l}|[\mathbf{z}]_{l};[\bm{\theta}_{O}]_{l})$
where $[\bm{\theta}_{O}]_{l}$ represents the parameters of the channel.

GAMP can be used quite generally for estimation of signals from many different
distributions of $\mathbf{x}$. Here we consider the compressed sensing setting
(see [3]) where $m<n$, i.e. an under-determined system which may be solved if

$\|\mathbf{x}\|_{0}=k\ll n,\quad k<m,$ (3)

where the $\|\mathbf{x}\|_{0}$ operator counts the number of non-zero entries
in $\mathbf{x}$. GAMP can solve (1) in the compressed sensing setting when a
sparse prior can be imposed on $\mathbf{x}$ to model (3).

For the compressed sensing setting, the algorithm approximate message passing
(AMP) was proposed to estimate $\mathbf{x}$ with an i.i.d. Laplacian prior and
i.i.d additive white Gaussian noise $\mathbf{e}$ [4]. GAMP can be seen as a
generalisation of AMP that allows for a wider range of probability
distributions on the signals $\mathbf{x}$ and on the measurements $\mathbf{y}$
given $\mathbf{x}$.

Another prior that can be used to model sparse signals in Bayesian estimators
such as GAMP is the so-called spike-and-slab model [5]. According to this
model, each entry $x_{j}$ in $\mathbf{x},\ j=0,1,\ldots n$ is distributed
according to a linear combination of a Dirac delta p.d.f $\delta(x_{j})$ and
another p.d.f. $\phi(x_{j};\bm{\theta}_{I})$:

$p(x_{j};\tau,\bm{\theta}_{I})=(1-\tau)\delta(x_{j})+\tau\phi(x_{j};\bm{\theta}_{I})$
(4)

The $\delta(x)$ models the fact that many of the entries $x_{i}$ are zero (by
the sparsity of $\mathbf{x}$). The variable $0\leq\tau\leq 1$ controls the
sparsity of $\mathbf{x}$, i.e. how likely the entries are to be zero. The
function $\phi(x_{j};\bm{\theta}_{I})$ can be chosen to represent the p.d.f.
of entries of $\mathbf{x}$ that are not zero; $\bm{\theta}_{I}$ represents the
parameters of $\phi$.

One example of such a spike-and-slab prior is the Bernoulli-Gauss distribution
[6]:

$p(x_{j};\tau,\mu,\sigma)=(1-\tau)\delta(x_{j})+\tau\mathcal{N}(x_{j};\mu,\sigma),$
(5)

where the p.d.f. representing the non-zero entries is the Gaussian
distribution with mean $\mu$ and std. deviation $\sigma$ (corresponding to the
parameters $\bm{\theta}_{I}$). The term Bernoulli-Gauss (BG) reflects the fact
that each entry $x_{i}$ can be seen as the product of a Bernoulli random
variable (values 0 or 1) and a Gaussian random variable.

Applying GAMP with the signal prior (5) assumes that the entries of
$\mathbf{x}$ are i.i.d. In many cases, signals of interest exhibit additional
structure that can be exploited to estimate them more accurately [7].

Many different algorithmic approaches to modelling and leveraging such signal
structure can be taken. Schniter et al. have for example produced substantial
results on incorporating the ability to learn the structure of non-i.i.d.
priors into the GAMP framework, see e.g. [8]. We take a different approach
here and propose a weighted spike-and-slab model that can be used to model
non-i.i.d. signals. We present the corresponding GAMP equations derived for
this prior model on $\mathbf{x}$ as well as results from numerical simulations
that show the improvements in reconstruction capabilities that are achievable
using such a structured prior.

##  2 Weighted-Prior GAMP

We re-state the uniform variance MMSE GAMP algorithm [2] in Algorithm 1 as
given in [9] with some variable changes to match (1).

1
initialise:$\bar{\mathbf{x}}_{0}=\mathbb{E}_{\mathbf{x}|\bm{\theta}_{I}}[\mathbf{x}]$,
$\breve{x}_{0}=\frac{1}{n}\sum\left(\text{Var}_{\mathbf{x}|\bm{\theta}_{I}}(\mathbf{x})\right)$,
$\mathbf{q}_{0}=\mathbf{0}_{m}$

2  for $t=1\dots T_{\text{max}}$ do

3  $\breve{v}_{t}=\frac{1}{m}||\mathbf{A}||_{F}^{2}\breve{x}_{t\mbox{-}1}$

4
$\mathbf{o}_{t}=\mathbf{A}\bar{\mathbf{x}}_{t\mbox{-}1}-\breve{v}_{t}\mathbf{q}_{t\mbox{-}1}$

5
$\bar{\mathbf{z}}_{t}=f_{\bar{z}}(\breve{v}_{t},\mathbf{o}_{t};\mathbf{y},\bm{\theta}_{o})$

6
$\tilde{\mathbf{z}}_{t}=f_{\tilde{z}}(\breve{v}_{t},\mathbf{o}_{t};\mathbf{y},\bm{\theta}_{o})$

7  $\mathbf{q}_{t}$ =
$\frac{\bar{\mathbf{z}}_{t}-\mathbf{o}_{t}}{\breve{v}_{t}}$

8  $\breve{u}_{t}$ =
$\frac{1}{m}\sum\left(\frac{\breve{v}_{t}-\tilde{\mathbf{z}}_{t}}{\breve{v}_{t}^{2}}\right)$

9  $\breve{s}_{t}=[\frac{1}{n}||\mathbf{A}||_{F}^{2}\breve{u}_{t}]^{-1}$

10
$\mathbf{r}_{t}=\bar{\mathbf{x}}_{t\mbox{-}1}+\breve{s}_{t}\mathbf{A}^{H}\mathbf{q}_{t}$

11
$\bar{\mathbf{x}}_{t}=f_{\bar{x}}(\breve{s}_{t},\mathbf{r}_{t};\bm{\theta}_{I})$

12
$\breve{x}_{t}=\frac{1}{n}\sum\left(f_{\tilde{x}}(\breve{s}_{t},\mathbf{r}_{t};\bm{\theta}_{I})\right)$

13  if stop criterion is met then

14  break

15  end if

16  end for

Algorithm 1 \- Uniform variance MMSE GAMP

We stress that our proposed prior may as well be used with the non-uniform
variants of GAMP that track individual variances. Due to limited space, a
detailed explanation of the algorithms can be found in [9] and references
therein.

In order to apply the GAMP algorithm with a specific entry-wise input prior
$p(x_{j};\tau,\bm{\theta}_{I})$ and measurement channel function
$p(\mathbf{y}|\mathbf{z};\bm{\theta}_{O})$, it is necessary to derive the
posterior p.d.f. $p(x_{j}|\mathbf{y};s_{j},r_{j},[\bm{\theta}_{I}]_{j})$. The
evaluation of input posterior and measurement channel functions is
incorporated in the GAMP iterations in the form of $\bar{\mathbf{x}}_{t}$ and
$\breve{x}_{t}$ in Algorithm 1, lines 11-12, respectively
$\bar{\mathbf{z}}_{t}$ and $\tilde{\mathbf{z}}_{t}$ in lines 5-6. We refer to
these functions, $f_{\bar{x}},f_{\tilde{x}}$ and $f_{\bar{z}},f_{\tilde{z}}$
as the GAMP input- and output channnel functions, respectively.

Channel functions for the BG input channel (5) can be found in [10, 11, 6] and
for the additive white Gaussian noise output channel in [1, 2].

Here we propose a modified Bernoulli-Gauss input channel that supports non-
uniform sparsity over the signal $\mathbf{x}$, i.e. different entries $x_{j}$
can have different probabilities of being zero. We extend the spike-and slab
model (4) as follows [9, p. 15]:

$p(x_{j};\tau,\bm{\theta}_{I})=(1-w_{j}\tau)\delta(x_{j})+w_{j}\tau\phi(x_{j};\bm{\theta}_{I})$
(6)

General input channel functions have been derived for this model [9, p. 16].
As one example of such a weighted spike-and-slab input channel, we have
derived the following closed-form expressions for the weighted BG input
channel functions [9, p. 26], i.e. where
$\phi(x_{j};\bm{\theta}_{I})=\mathcal{N}(x_{j};\bar{\theta},\tilde{\theta})$:

$\displaystyle
f_{\bar{x}_{j}}(s_{j},r_{j};\bm{\theta}_{I})=\pi_{j}^{\text{w}}(r_{j},s_{j},\bm{\theta}_{I})\left(\frac{\frac{\bar{\theta}}{\tilde{\theta}}+\frac{r_{j}}{s_{j}}}{\frac{1}{\tilde{\theta}}+\frac{1}{s_{j}}}\right)$
(7) $\displaystyle
f_{\tilde{x}_{j}}(s_{j},r_{j};\bm{\theta}_{I})=\pi_{j}^{\text{w}}(r_{j},s_{j},\bm{\theta}_{I})$
$\displaystyle\ \ \
\cdot\left(\frac{1}{\frac{1}{\tilde{\theta}}+\frac{1}{s_{j}}}+\left(\frac{\frac{\bar{\theta}}{\tilde{\theta}}+\frac{r_{j}}{s_{j}}}{\frac{1}{\tilde{\theta}}+\frac{1}{s_{j}}}\right)^{2}\right)-f_{\bar{x}_{j}}(s_{j},r_{j};\bm{\theta}_{I})^{2}$
(8)

where the function $\pi_{j}^{\text{w}}$ is given in eqs. (3.46)-(3.50) in [9,
p. 15] and $\bar{\theta}$ and $\tilde{\theta}$ are the mean and the variance,
respectively, of the Gaussian term in each entry of the GAMP input channel.

With the proposed weighted input prior (6) it is possible to model signals
with varying probability of zero entries across the signal $\mathbf{x}$.
Additionally, similar to the mechanism for the i.i.d. BG input channel in [6],
we have derived formulas for updating the parameters $\tau,\bm{\theta}_{I},$
and $\bm{\theta}_{O}$ using expectation-maximization (EM) as part of the GAMP
algorithm [9, sec. 6]. We retain separate parameters $\tau$ and $w_{j}$ to
enable estimating the overall sparsity via EM without too many free
parameters.

Next, we demonstrate by numerical experiments how the proposed model can
improve estimation of compressively sensed signals with a known non-uniform
sparsity structure.

##  3 Numerical Example

We perform a set of numerical experiments to demonstrate the benefits of the
proposed weighted sparse prior for GAMP when such a model matches the signal
of interest. We simulate the algorithm’s reconstruction capabilities in the
form of a phase transition diagram over the full sparsity / under-sampling
parameter space. See [12] for an introduction to the phase transition in
compressed sensing problems.

We simulate sparse BG signals according to the proposed weighted model (6)
where the weights $w_{j}$ are selected to have a Gaussian shape over the
support of the signal vector $\mathbf{x}$:

$w_{j}=e^{-25\left(\frac{j}{n}-\frac{1}{2}\right)^{2}},\quad j=0,1,\ldots,n$
(9)

We partition the parameter space of $\delta=m/n$ and $\rho=k/m$ into a grid of
points in each of which we reconstruct 10 random signals $\mathbf{x}$ with $k$
non-zero entries on average, measured with a randomly generated matrix
$\mathbf{A}$ with i.i.d. random Gaussian entries. The reconstruction success
(exact to within a small tolerance) of each signal is evaluated and the
success rates in the $(\delta,\rho)$ points are used to estimate the phase
transitions shown in Figure 1. The results shown here are a small subset of
the simulation results available along with source code in [13].

Figure 1: Numerical phase transition simulation results.

For reference, the figure includes the theoretical $\ell_{1}$-optimisation
phase transition curve [12], simulation results for a non-weighted BG prior
(5) as well as simulation results for the implied Laplacian model used by AMP,
“DMM AMP” [4]. We present results for a weighted prior that matches our
Gaussian weights in (9) for: 1. The algorithm knows the true model parameters
(genie) and 2. EM is used with a re-weighting scheme to estimate the
parameters [9, sec. 6]. Comparing the two, we can see that knowing the
underlying sparsity structure of the signal of interest improves the
reconstruction capabilities substantially compared to simply assuming no
weighting. Note, however, that it is important to allow the GAMP algorithm
some “slack” in the form of estimating the parameters $\tau,\bar{\theta}_{j}$,
and $\tilde{\theta}_{j}$ using EM to get the best performance. The error bars
shown on the result curves correspond to the 10% to 90% percentile range of
the logistic sigmoid functions fitted to the reconstruction outcomes to
estimate the phase transition.

##  4 Conclusion

We have proposed a model for a class of non-uniformly structured sparse
signals for use in the generalised approximate message passing (GAMP)
algorithm. The proposed approach models sparse signals where the probability
of zero vs. non-zero entries can vary across the support of the signal of
interest. We have demonstrated through numerical examples how exploiting such
structure when present in signals can substantially improve reconstruction of
compressively sensed signals.

## References

  * [1] Sundeep Rangan  “Generalized Approximate Message Passing for Estimation with Random Linear Mixing”  In _IEEE International Symposium on Information Theory (ISIT)_ , 2011, pp. 2168–2172  DOI: 10.1109/ISIT.2011.6033942
  * [2] Sundeep Rangan  “Generalized Approximate Message Passing for Estimation with Random Linear Mixing”  In _arXiv pre-prints_ , 2012  arXiv:1010.5141v2
  * [3] E.. Candès and M.. Wakin  “An Introduction To Compressive Sampling”  In _IEEE Signal Processing Magazine_ 25.2, 2008, pp. 21–30  DOI: 10.1109/MSP.2007.914731
  * [4] David L. Donoho, Arian Maleki and Andrea Montanari  “Message-passing algorithms for compressed sensing”  In _Proceedings of the National Academy of Sciences of the United States of America_ 106.45, 2009, pp. 18914–18919  DOI: 10.1073/pnas.0909892106
  * [5] T.. Mitchell and J.. Beauchamp  “Bayesian Variable Selection in Linear Regression”  In _Journal of the American Statistical Association_ 83.404 Taylor & Francis, 1988, pp. 1023–1032  DOI: 10.1080/01621459.1988.10478694
  * [6] Jeremy Vila and Philip Schniter  “Expectation-Maximization Bernoulli-Gaussian Approximate Message Passing”  In _Forty Fifth Asilomar Conference on Signals, Systems and Computers (ASILOMAR)_ , 2011, pp. 799–803  DOI: 10.1109/ACSSC.2011.6190117
  * [7] R.. Baraniuk, V. Cevher, M.. Duarte and C. Hegde  “Model-Based Compressive Sensing”  In _IEEE Transactions on Information Theory_ 56.4, 2010, pp. 1982–2001  DOI: 10.1109/TIT.2010.2040894
  * [8] Justin Ziniel, Sundeep Rangan and Philip Schniter  “A Generalized Framework for Learning and Recovery of Structured Sparse Signals”  In _IEEE Statistical Signal Processing Workshop (SSP)_ , 2012, pp. 325–328  DOI: 10.1109/SSP.2012.6319694
  * [9] Christian Schou Oxvig, Thomas Arildsen and Torben Larsen  “Generalized Approximate Message Passing: Relations and Derivations”, 2017  DOI: 10.5278/VBN.GAMPTechReport
  * [10] Florent Krzakala et al.  “Probabilistic reconstruction in compressed sensing: algorithms, phase diagrams, and threshold achieving matrices”  In _Journal of Statistical Mechanics: Theory and Experiment_ P08009, 2012, pp. 1–57  DOI: 10.1088/1742-5468/2012/08/P08009
  * [11] Jason T. Parker  “Approximate Message Passing Algorithms for Generalized Bilinear Inference”, 2014  URL: http://rave.ohiolink.edu/etdc/view?acc_num=osu1405690178
  * [12] David Donoho and Jared Tanner  “Observed universality of phase transitions in high-dimensional geometry, with implications for modern data analysis and signal processing”  In _Philosophical Transactions of the Royal Society A_ 367.1906, 2009, pp. 4273–4293  DOI: 10.1098/rsta.2009.0152
  * [13] Christian Schou Oxvig, Thomas Arildsen and Torben Larsen  “Weighted GAMP Phase Transitions”, 2018  DOI: 10.5281/zenodo.1409655