# LoRA-XS: Low-Rank Adaptation with Extremely Small Number of Parameters

Klaudia Bałazy∗1 &Mohammadreza Banaei∗2 &Karl Aberer2 &Jacek Tabor1
1Jagiellonian University, 2EPFL

###### Abstract

The recent trend in scaling language models has led to a growing demand for
parameter-efficient tuning (PEFT) methods such as LoRA (Low-Rank Adaptation).
LoRA consistently matches or surpasses the full fine-tuning baseline with
fewer parameters. However, handling numerous task-specific or user-specific
LoRA modules on top of a base model still presents significant storage
challenges. To address this, we introduce LoRA-XS (Low-Rank Adaptation with
eXtremely Small number of parameters), a novel approach leveraging Singular
Value Decomposition (SVD) for parameter-efficient fine-tuning. LoRA-XS
introduces a small $r\times r$ weight matrix between frozen LoRA matrices,
which are constructed by SVD of the original weight matrix. Training only
$r\times r$ weight matrices ensures independence from model dimensions,
enabling more parameter-efficient fine-tuning, especially for larger models.
LoRA-XS achieves a remarkable reduction of trainable parameters by over 100x
in 7B models compared to LoRA. Our benchmarking across various scales,
including GLUE, GSM8k, and MATH benchmarks, shows that our approach
outperforms LoRA and recent state-of-the-art approaches like VeRA in terms of
parameter efficiency while maintaining competitive performance.111Our code is
available at https://github.com/MohammadrezaBanaei/LoRA-XS.

**footnotetext: Equal contribution. Correspondence to:
<EMAIL_ADDRESS><EMAIL_ADDRESS>
##  1 Introduction

In recent years, the development of large language models (LLM) has
revolutionized the field of natural language processing (NLP), enabling
unprecedented performance across various tasks. However, these state-of-the-
art models often come with a huge number of parameters, presenting significant
challenges for fine-tuning and adaptation to specific downstream tasks.
Modifying and storing these immense models introduces computational and
storage challenges.

Given these challenges, Parameter-Efficient Fine-Tuning (PEFT) methods, where
only a relatively small number of parameters are fine-tuned, emerged as a
potential solution to compensate for the tremendous compute/storage cost of
full parameter fine-tuning [8, 9, 12, 13, 31]. Among PEFT methods, LoRA [9] is
widely used in recent literature due to its good generalization and the fact
that it does not introduce extra modules during the inference phase.
Nevertheless, even these techniques can require considerable storage and
computational resources, particularly when the objective is to enable large-
scale personalized or task-specific adaptation. As an example, applying LoRA
on the GPT-3 model [2] with a rank of 16 and only query and value matrices
being adapted, would result in 144MB of memory per checkpoint, which would
amount to 144TB of memory when serving 1 million personalized models.

Following LoRA, many successors have been proposed to further reduce the
number of parameters and improve efficiency [11, 16, 32]. One such recent
state-of-the-art method is VeRA [11], which reduces the number of trainable
parameters by freezing the LoRA matrices and using a single pair of low-rank
matrices shared across all layers while learning small scaling vectors
instead. Although VeRA improves parameter efficiency, its parameter count
remains dependent on the model hidden dimensions, which becomes increasingly
significant for larger language models.222For example, while early transformer
models like BERT have a hidden dimension of 768, the recent GPT-3 model has a
hidden dimension of 12288, which directly affects the trainable parameter
count for LoRA and VeRA methods. This dependency can result in substantial
storage and computational requirements as model sizes continue to grow.

Figure 1: Average performance of RoBERTa-large over a selection of GLUE tasks
as a function of the number of trainable parameters (in millions) for
different adaptation methods: Full Fine-Tuning (FT), LoRA, VERA, and LoRA-XS.
LoRA-XS points correspond to various ranks from r=4 up to r=25. LoRA-XS
outperforms the other methods in both average performance and parameter
efficiency.

In this paper, we propose LoRA-XS, a highly parameter-efficient LoRA-based
method designed to achieve similar or superior adaptation performance with
significantly fewer parameters. In particular, the number of LoRA-XS trainable
parameters is independent of the model’s hidden dimensions. Consequently,
LoRA-XS marks a new paradigm in parameter-efficient fine-tuning (PEFT),
overcoming the limitations of existing approaches and offering a more
efficient path to model personalization and task-specific optimization. Using
the previous example on low-rank adaptation of GPT-3 with a rank of 16,
serving 1 million personalized models with LoRA-XS would require only 96GB of
storage, compared to LoRA’s 144TB, resulting in more than a 1500x reduction in
storage requirements.

Moreover, with LoRA-XS, we can precisely control the number of additional
parameters, allowing for flexible memory usage (see Figure 1). This
flexibility is particularly beneficial for increasingly larger models, where
traditional methods impose a certain minimum number of additional parameters.
Furthermore, LoRA-XS retains the core advantages of LoRA, such as not
requiring any modifications to the model architecture and introducing no
additional latency during inference, making it an efficient and seamless
solution for practical deployment.

LoRA-XS achieves this extreme parameter efficiency by setting LoRA’s
projection matrices using Singular Value Decomposition (SVD) of the pretrained
module weights and keeping them frozen during training (see Figure 2). The
only trainable parameter of LoRA-XS is an $r\times r$ matrix (i.e., $R$)
between the frozen LoRA projection matrices, where $r$ denotes LoRA’s rank.
Fixing these matrices during training transforms our method into a _latent
editing_ as done by the matrix $R$ using only $r^{2}$ parameters. Although
LoRA-XS trains the model in a highly constrained parameter space, as we will
show in later sections, its performance remains competitive or better than the
LoRA baseline across different benchmarks and model scales.

In summary, our contributions are as follows:

  * •

We introduce LoRA-XS, a highly parameter-efficient method that achieves an
extreme reduction in the number of trainable parameters for fine-tuning,
reducing them by over 100x in our experiments with 7B models.

  * •

LoRA-XS outperforms LoRA and recent state-of-the-art variants like VeRA [11]
in terms of parameter efficiency while providing competitive or superior
performance. We evaluate LoRA-XS across different model scales and a diverse
set of benchmarks, including GLUE [28], GSM8k [3], and MATH [7] datasets.

  * •

Compared to other LoRA variants, LoRA-XS offers significantly better
flexibility in the storage needed to save the method’s parameters, due to its
parameter count independence from model dimensions (see Figure 1).

  * •

We demonstrate the effectiveness of SVD-based initialization in enhancing
model performance and training efficiency through a comprehensive set of
experiments.

##  2 Related work

#### Efficient Adaptation

Recently, there has been many variants of adapter-based finetuning methods
proposed where a set of adapter modules are introduced into the transformer
model [27]. These modules can either be introduced as extra _adapter_ layers
into the transformer block [8, 15, 24], or as an additional set of parameters
modifying input layer activations [1, 12, 13, 17]. Although these approaches
introduce a relatively small number of parameters into the model, they
deteriorate model’s latency in online inference, especially in large-scale
production scenarios.

#### Low-rank Adaptation

Low-rank adaptation of transformer models, proposed by LoRA [9], offers a
strong alternative to previous PEFT methods, where the generalization
performance is competitive to full finetuning while not introducing any
further latency during inference phase. Building upon the LoRA method, there
has been many recent efforts to improve its learning curve [16, 6, 21], reduce
the trainable parameters [32, 11, 25], or even training it with quantized
pretrained weights to substantially improve memory footprint during training
[4, 14]. Our method, LoRA-XS, falls in the second category, where we aim to
significantly reduce trainable parameters while performing competitively to
LoRA over various benchmarks and different model scales. It is worth noting
that our proposed approach is orthogonal to methods like QLoRA [4] as LoRA-XS
can also be applied on a quantized model to both address pretrained model’s
memory usage, as well as significantly reducing trainable parameters333In this
work, all experiments are performed on 16-bit models and we leave experiments
on quantized models for future work..

#### Parameter-constrained LoRA variants

As mentioned above, many recent works try to propose LoRA variants that
further constrain its trainable parameters while performing competitive to the
conventional LoRA. AdaLoRA [32] investigates a dynamic rank adjustment for
different modules’ low-rank matrices as opposed to uniform parameter
allocation in LoRA. Tied-LoRA [25] tries to enhance the parameter efficiency
by having LoRA matrices tied across all layers of the transformer model.
Perhaps the most similar work to ours is VeRA [11] which shares randomly
initialized frozen LoRA matrices across layers, and introduces trainable
scaling vectors. Unlike VeRA, our low-rank matrices are initialized using the
SVD of the model’s pretrained weight, and we demonstrate its advatange over
random initialization in Section 4.4. Moreover, unlike VeRA, the number of
trainable parameters in LoRA-XS is not dependent on the model’s hidden
dimensions which enables us to significantly decrease its trainable
parameters, especially for large-scale models.

##  3 Method

Figure 2: Visual comparison of LoRA and LoRA-XS techniques. The key
distinction of LoRA-XS lies in its use of a small trainable matrix $R$ between
frozen low-rank matrices $A$ and $B$ derived from truncated SVD (to $r$ top
components) of pretrained weights.

This section presents LoRA-XS (Low-Rank Adaptation with eXtremely Small number
of parameters), a novel method designed to enhance the parameter efficiency of
large language models’ fine-tuning by leveraging insights from low-rank
adaptation methods. Building upon the foundational concepts of LoRA, our
approach specifically addresses scalability and storage issues while
maintaining highly competitive performance.

In recent years, LoRA [9] has significantly impacted the field of parameter-
efficient tuning by introducing adaptation through low-rank matrices,
effectively reducing the overall number of trainable parameters. However,
deploying LoRA at scale, particularly across large-scale task-specific or
user-specific models, can substantially increase the required storage needs.
As modern models grow in size and complexity, the need for even more
parameter-efficient tuning strategies becomes a crucial issue.

The LoRA paper revealed that its low-rank weight updates (i.e., $\Delta W$)
has a strong correlation with the pretrained weights, demonstrating that LoRA
amplifies some _directions_ that are already in the model’s weights. Inspired
by this empirical observation, we propose setting LoRA adaptation matrices
(i.e., matrices $A$ and $B$ in Figure 2) using the top left/right singular
vectors of the pretrained weight. To achieve this, we set LoRA’s $A$ and $B$
matrices using the truncated SVD of the pretrained weight matrix $W$ and keep
them frozen during training. We then introduce a $r\times r$ trainable matrix
between these fixed matrices. This intermediary matrix $R$ is the only
trainable component in LoRA-XS, drastically reducing the number of trainable
parameters and making its parameter count independent of the model dimensions.
In Figure 2, we present an overview of the LoRA-XS method, comparing it with
the original LoRA framework.

###  3.1 Formulation of LoRA-XS

The main idea behind the LoRA-XS is to modify the adaptation process by
introducing a small square matrix $R\in\mathbb{R}^{r\times r}$ between frozen
LoRA matrices that are set using truncated SVD of the pretrained weight matrix
$W\in\mathbb{R}^{m\times n}$.

The traditional LoRA forward path for an input $x\in\mathbb{R}^{n}$ is:

$h=Wx+\Delta Wx=Wx+BAx$ (1)

where $\Delta W\in\mathbb{R}^{m\times n}$ is the low-rank weight update. The
matrices $A\in\mathbb{R}^{r\times n}$ and $B\in\mathbb{R}^{m\times r}$ are
low-rank matrices with $r\ll\min(m,n)$. During training, $W$ is kept frozen,
and $A$ and $B$ are the trainable parameters.

In LoRA-XS, we improve parameter efficiency by introducing a small trainable
matrix $R\in\mathbb{R}^{r\times r}$, while keeping matrices $A$ and $B$
frozen, modifying the forward path to:

$h=Wx+\Delta Wx=Wx+BRAx$ (2)

Here, $A$ and $B$ are set using the truncated SVD of the original weight
matrix $W$. Formally, the SVD of $W$ is given by:

$W=U\Sigma V^{T}$ (3)

where $U\in\mathbb{R}^{m\times m}$, $\Sigma\in\mathbb{R}^{m\times n}$, and
$V\in\mathbb{R}^{n\times n}$. We set (frozen) matrices $A$ and $B$ as:

$A=U_{r}\Sigma_{r}\quad\text{and}\quad B=V_{r}^{T}$ (4)

where $U_{r}\in\mathbb{R}^{m\times r}$ and $V_{r}\in\mathbb{R}^{n\times r}$
contain the left/right singular vectors corresponding to the top $r$ singular
values, and $\Sigma_{r}\in\mathbb{R}^{r\times r}$ is a diagonal matrix that
contain these top $r$ singular values of $\Sigma$.

Our newly introduced $R$ matrix is initialized with a Gaussian distribution
$N(0,\sigma^{2})$, where $\sigma$ is set to a small but non-zero value444In
all LoRA-XS experiments, we set $\sigma$ to $10^{-5}$.. This ensures that we
start fine-tuning with a model almost identical to the original pretrained
model. During fine-tuning, the matrices $A$ and $B$ are kept frozen, and only
$R$ is updated, significantly reducing the number of trainable parameters.

Our empirical results (see Table 1 and Table 2) show that aligning the
adaptation matrices with the top-$r$ singular vectors of the pretrained
weights, while having the intermediary matrix $R$ perform _latent editing_ ,
can maintain model performance while drastically reducing the number of
trainable parameters to $r^{2}$ per target module. This alignment ensures that
the model operates in a subspace that captures the most significant
components/directions of the pretrained weights.

Compared to the other LoRA variants, LoRA-XS provides better control over the
number of trainable parameters, allowing for more flexible required storage
for the finetuned models. This flexibility is particularly beneficial for
increasingly larger models, where traditional methods are often limited by
model’s hidden dimensions. Similar to LoRA and its successors, LoRA-XS does
not introduce any extra computational overhead or latency during inference, as
this module can be merged into the original matrix post-training.

###  3.2 Parameter Efficiency of LoRA-XS

We make the following observation on the parameter efficiency of LoRA-XS
compared to LoRA and VeRA methods.555It is worth noting that in these
calculations we only consider the additional LoRA/VeRA/LoRA-XS parameters, and
for encoder-only models, additional trainable parameters such as classifiers
parameters may be added. However, these are common to all approaches and thus
do not influence our comparative calculations.

Observation: LoRA-XS demonstrates superior parameter efficiency compared to
both LoRA and VeRA.

For simplicity, let’s consider a transformer model with $L$ finetuned layers,
each consisting of $q$ number of $W\in\mathbb{R}^{n\times n}$ matrices.

For LoRA, the number of trainable parameters is given by:

$P_{\text{LoRA}}=L\times q\times r\times 2n,$ (5)

For VeRA, the number of trainable parameters is given by:

$P_{\text{VeRA}}=L\times q\times(n+r),$ (6)

For LoRA-XS, the number of trainable parameters is given by:

$P_{\text{LoRA-XS}}=L\times q\times r^{2}.$ (7)

To compare the parameter efficiency, we compute the ratios of the number of
trainable parameters between the methods. The ratio of trainable parameters
for LoRA to LoRA-XS is:

$\frac{P_{\text{LoRA}}}{P_{\text{LoRA-XS}}}=\frac{L\times q\times r\times
2n}{L\times q\times r^{2}}=\frac{2n}{r},$ (8)

Similarly, the ratio of trainable parameters for VeRA to LoRA-XS is:

$\frac{P_{\text{VeRA}}}{P_{\text{LoRA-XS}}}=\frac{L\times
q\times(n+r)}{L\times q\times r^{2}}=\frac{n+r}{r^{2}}.$ (9)

As the model dimension $n$ becomes very large compared to the rank $r$, the
benefit of LoRA-XS over LoRA and VeRA becomes more pronounced. Specifically
for large $n$:

$\frac{P_{\text{LoRA}}}{P_{\text{LoRA-
XS}}}\approx\frac{2n}{r}\quad\text{and}\quad\frac{P_{\text{VeRA}}}{P_{\text{LoRA-
XS}}}\approx\frac{n}{r^{2}}.$ (10)

This indicates that for large models, LoRA and VeRA require significantly more
parameters than LoRA-XS, with the difference growing linearly with $n$ (i.e.,
model’s hidden dimension). This makes LoRA-XS especially suitable for fine-
tuning large language models where parameter efficiency is crucial.

To provide an example, for RoBERTa-large [18], which consists of 24 layers,
assuming $q=2$ (two additional trainable modules per layer), with each $W$
matrix of size $1024\times 1024$ and $r=16$, the number of trainable
parameters for each method is as follows: For LoRA,
$P_{\text{LoRA}}=1,572,864$; for VeRA, $P_{\text{VeRA}}=50,400$; for LoRA-XS,
$P_{\text{LoRA-XS}}=12,288$. Thus, LoRA requires about 31.2 times more
parameters than VeRA and 128 times more than LoRA-XS, while VeRA requires
about 4 times more parameters than LoRA-XS, highlighting the substantial
parameter efficiency of LoRA-XS.

##  4 Experiments

This section describes our experiments evaluating the effectiveness of LoRA-
XS. We begin by detailing the experimental setup and then present our
evaluation on the GLUE benchmark [28], comparing LoRA-XS with full fine-
tuning, LoRA, and VeRA baselines across six GLUE tasks. Various ranks for
LoRA-XS are explored to demonstrate their impact on performance and parameter
efficiency.

Next, we discuss instruction tuning experiments on the Mistral-7B and Gemma-7B
models. These experiments evaluate LoRA-XS’s ability to enable LLMs to follow
the provided instructions with minimal parameter overhead. In these
experiments, the models are first trained on the MetaMathQA dataset [30], and
then the fine-tuned models are evaluated on GSM8K [3] and MATH [7] benchmarks
to assess their capabilities in following instructions using mathematical
reasoning as the target downstream task.

Finally, we include an ablation study that examines the empirical benefits of
initializing matrices $A$ and $B$ in LoRA-XS using SVD of the pretrained
weights compared to the random initialization that is typically used in LoRA-
based methods. This study provides insights into the initialization strategy’s
impact on learning efficiency (i.e., learning curve) and the final
performance.

###  4.1 Experimental Setup

For the GLUE benchmark experiments, we use the RoBERTa-large model [18] and
explore different ranks for LoRA-XS, ranging from $r=4$ to $r=25$. This range
allows us to examine the impact of varying numbers of trainable parameters on
performance. For GLUE experiments, we add LoRA-XS modules to the Query
($W_{q}$), Value ($W_{v}$), Attention Output ($W_{o}$), and the output fully
connected weight matrices of the RoBERTa model. Hyperparameters were selected
through grid search, and the chosen values are summarized in Table 4. Similar
to previous efforts, when finetuning the model on MRPC, RTE and STS-B tasks,
the model is initialized using weights finetuned on the MNLI task666The model
is trained for 10 epochs on the MNLI dataset, and we only use the matrices $R$
for initialization of these three tasks.[9, 11].

For the instruction tuning experiments, we use the Mistral-7B-v0.1 and
Gemma-7B decoder-only models, training them on 100k samples of the MetaMathQA
[30] dataset and evaluating on the GSM8K [3] and MATH [7] benchmarks. LoRA-XS
modules are added to the Query ($W_{q}$), Key ($W_{k}$), Value ($W_{v}$),
Attention Output ($W_{o}$), and all three fully connected weight matrices.

Each LoRA-XS module in our main experiments is initialized with the SVD of the
corresponding pretrained weights $W$. The initialization process involves
using truncated SVD [5] with 10 iterations, implemented via the scikit-learn
library [23]. Further details on the experiments are provided in Appendix A.

In our ablation study, we investigate the differences in model performance
when initializing LoRA-XS $A$ and $B$ matrices with Singular Value
Decomposition (SVD) compared to random initialization, as proposed in the LoRA
and VeRA approaches. We examine the RoBERTa-large [18] model across three GLUE
tasks: CoLA, SST-2, and QNLI. For our proposed approach, the $A$ and $B$
matrices are initialized using truncated SVD with 10 iterations of the
original matrix. For the baseline approach, we use the standard Kaiming
Initialization for the linear layers, as implemented in PyTorch [22]. We also
explore the impact of different ranks, specifically 4, 8, 12, and 20. The
hyperparameters for these experiments were selected through grid search, and
the chosen values are detailed in Appendix A.

###  4.2 GLUE Benchmark

In Table 1, we present the performance of the RoBERTa-large model on the GLUE
benchmark using full fine-tuning (FT) and parameter-efficient fine-tuning
methods: LoRA, VeRA and our LoRA-XS.

Method |  # Trainable Parameters | Rank | SST-2 | MRPC | CoLA | QNLI | RTE | STS-B | Avg.
---|---|---|---|---|---|---|---|---|---
FT | 355,000K | - | 96.4 | 90.9 | 68.0 | 94.7 | 86.6 | 92.4 | 88.17
LoRA | 800K | 8 |  96.2 $\pm$ 0.5 |  90.2 $\pm$ 1.0 |  68.2 $\pm$ 1.9 |  94.8 $\pm$ 0.3 |  85.2 $\pm$ 1.1 |  92.3 $\pm$ 0.5 | 87.82
VeRA | 61K | 256 |  96.1 $\pm$ 0.1 |  90.9 $\pm$ 0.7 |  68.0 $\pm$ 0.8 |  94.4 $\pm$ 0.2 |  85.9 $\pm$ 0.7 |  91.7 $\pm$ 0.8 | 87.83
LoRA-XS | 60K | 25 |  96.33 $\pm$ 0.15 |  91.18 $\pm$ 0.82 |  68.55 $\pm$ 0.81 |  94.34 $\pm$ 0.22 |  89.53 $\pm$ 0.48 |  92.22 $\pm$ 0.10 | 88.69
38.4K | 20 |  95.87 $\pm$ 0.28 |  90.44 $\pm$ 0.41 |  68.08 $\pm$ 1.21 |  94.05 $\pm$ 0.16 |  88.81 $\pm$ 0.20 |  91.76 $\pm$ 0.18 | 88.17
24.6K | 16 |  95.87 $\pm$ 0.24 |  90.69 $\pm$ 0.37 |  66.96 $\pm$ 1.23 |  93.89 $\pm$ 0.06 |  88.81 $\pm$ 0.30 |  91.98 $\pm$ 0.13 | 88.03
13.8K | 12 |  95.87 $\pm$ 0.31 |  90.20 $\pm$ 0.32 |  65.47 $\pm$ 0.90 |  93.32 $\pm$ 0.51 |  87.73 $\pm$ 0.68 |  91.40 $\pm$ 0.12 | 87.03
6.1K | 8 |  95.30 $\pm$ 0.34 |  88.48 $\pm$ 0.64 |  64.39 $\pm$ 0.75 |  92.49 $\pm$ 0.09 |  86.28 $\pm$ 0.59 |  90.57 $\pm$ 0.25 | 86.25
1.5K | 4 |  94.84 $\pm$ 0.29 |  87.75 $\pm$ 0.33 |  60.52 $\pm$ 1.54 |  90.94 $\pm$ 0.27 |  82.67 $\pm$ 0.53 |  88.88 $\pm$ 0.22 | 84.27

Table 1: RoBERTa-large performance on a selection of tasks from the GLUE
benchmark with different adaptation methods. We report Matthew’s correlation
for CoLA, Pearson correlation for STS-B, and accuracy for the other tasks.
Full fine-tuning, LoRA, and VeRA results are taken from prior works [9, 11].
Results indicate the median and standard deviation of 5 runs with different
seeds. LoRA-XS demonstrates competitive or superior performance over the
baselines while offering much better parameter efficiency. Notably, LoRA-XS
with rank=16 (i.e., only 256 parameters per target module) outperforms both
VeRA and LoRA baselines while reducing trainable parameters by 2.5x and 30x,
respectively. Higher is better for all metrics.

As shown in Table 1, LoRA-XS with ranks of 25, 20, and 16 (corresponding to
60K, 38.4K, and 24.6K trainable parameters, respectively) outperform baseline
PEFT methods (LoRA and VeRA), achieving the highest average performance across
the tested GLUE tasks. Notably, with a rank of 16, LoRA-XS achieves better
accuracy than VeRA while having 2.5x less trainable parameters. Following our
baselines’ evaluation method, we conduct 5 runs with different seeds,
recording the best epoch’s outcome for each run, and report the median of
these results. Similar to VeRA [11], we only include the added LoRA-XS modules
in the trainable parameters calculation, excluding the classifier parameters
for a clear comparison.

When examining even lower ranks to explore extreme parameter efficiency, we
observe very competitive performance. Although there is a slight drop in
performance as the rank decreases, this is expected due to the reduction in
parameter count. The most significant drop in average performance is observed
at the smallest rank of 4. However, even with this extremely small number of
parameters (approximately 1500), LoRA-XS shows only about a 4 percentage point
drop in accuracy. This demonstrates that LoRA-XS maintains competitive
performance while significantly reducing the number of trainable parameters.

Model | Method | Rank | # trainable parameters | GSM8K | MATH
---|---|---|---|---|---
Mistral-7B | Full FT | - | 7242M | 67.02 | 18.60
LoRA | 64 | 168M | 67.70 | 19.68
LoRA-XS | 64 | 0.92M | 68.01 | 17.86
32 | 0.23M | 63.23 | 15.88
16 | 0.057M | 57.92 | 14.44
Gemma-7B | Full FT | - | 8538M | 71.34 | 22.74
LoRA | 64 | 200M | 74.90 | 31.28
LoRA-XS | 64 | 0.80M | 74.22 | 27.62
32 | 0.20M | 71.72 | 27.32
16 | 0.050M | 68.46 | 26.38
Table 2: Instruction tuning performance on GSM8K and MATH Benchmarks for
Mistral-7B and Gemma-7B models using full fine-tuning, LoRA, and LoRA-XS. Full
finetuning and LoRA performance values are taken from prior work [21]. Higher
is better for all metrics.

###  4.3 Instruction tuning

The results of our instruction tuning experiments are presented in Table 2. As
mentioned earlier, we first fine-tune Mistral-7B and Gemma-7B models on the
MetaMathQA dataset, and then we evaluate them on the GSM8K and MATH datasets
to assess their instruction following capabilities in challenging mathematical
reasoning problems.

As demonstrated in Table 2, applying LoRA-XS to a 7B-scale model performs
competitively with both LoRA and full fine-tuning, showcasing the
applicability of our method to larger-scale models. Notably, LoRA-XS with only
0.92M parameters (rank=64) achieves close performance to LoRA, which uses 168M
parameters, across both the GSM8k and MATH benchmarks. This represents a
reduction of over 150x in the number of trainable parameters.

###  4.4 Random versus SVD initialization

In Table 3, we present the detailed results from our ablation study comparing
the initialization of the $A$ and $B$ matrices of LoRA-XS using SVD versus
random initialization. The reported metrics are accuracy for SST-2 and QNLI,
and Matthews correlation for CoLA on the RoBERTa-large model. Results are
reported as the median over 5 random seeds, with the best epoch result for
each run (column "Best epoch"). Additionally, we show results after 1 and 2
epochs to illustrate the learning curve and how quickly the models learn with
each initialization method.

The results empirically confirm our hypothesis that aligning adaptation
matrices with the principal components of pretrained weights can enhance
parameter efficiency while maintaining model accuracy. As shown in Table 3,
initializing the $A$ and $B$ matrices with SVD generally leads to improved
performance compared to random initialization.

Moreover, the performance after 1 and 2 epochs is often higher with SVD
initialization. This suggests that proper initialization using SVD may help
the models converge more quickly and achieve better overall accuracy. This
finding underscores the importance of initialization strategy in fine-tuning
large language models, highlighting that SVD initialization not only boosts
performance but also accelerates the training process.

Task | Rank | Init Type | Performance
---|---|---|---
|  |  | After 1 epoch | After 2 epoch | Best epoch
Cola | 4 | Random |  0 $\pm$ 7.68 |  0 $\pm$ 20.58 |  56.65 $\pm$ 1.09
SVD |  0 $\pm$ 12.12 |  21.97 $\pm$ 16.63 |  60.52 $\pm$ 1.54
8 | Random |  0 $\pm$ 17.59 |  40.91 $\pm$ 15.57 |  60.29 $\pm$ 1.22
SVD |  12.59 $\pm$ 11.09 |  43.84 $\pm$ 15.62 |  64.39 $\pm$ 0.75
12 | Random |  5.8 $\pm$ 14.71 |  26.77 $\pm$ 20.43 |  62.96 $\pm$ 1.25
SVD |  13.16 $\pm$ 17.84 |  44.22 $\pm$ 13.04 |  65.47 $\pm$ 0.90
20 | Random |  47.17 $\pm$ 2.59 |  43.42 $\pm$ 5.70 |  64.7 $\pm$ 1.10
SVD |  18.83 $\pm$ 9.47 |  44.77 $\pm$ 9.65 |  68.08 $\pm$ 1.21
SST-2 | 4 | Random |  91.63 $\pm$ 0.74 |  92.66 $\pm$ 0.51 |  93.92 $\pm$ 0.46
SVD |  92.78 $\pm$ 0.31 |  93.12 $\pm$ 0.47 |  94.84 $\pm$ 0.29
8 | Random |  92.55 $\pm$ 0.74 |  93.58 $\pm$ 0.53 |  95.30 $\pm$ 0.34
SVD |  93.23 $\pm$ 0.27 |  93.81 $\pm$ 0.22 |  94.95 $\pm$ 0.07
12 | Random |  93.46 $\pm$ 0.54 |  94.27 $\pm$ 0.46 |  95.64 $\pm$ 0.54
SVD |  93.46 $\pm$ 0.86 |  93.35 $\pm$ 0.49 |  95.87 $\pm$ 0.28
20 | Random |  94.15 $\pm$ 0.67 |  94.15 $\pm$ 0.58 |  95.87 $\pm$ 0.24
SVD |  94.61 $\pm$ 0.46 |  93.92 $\pm$ 0.87 |  95.87 $\pm$ 0.31
QNLI | 4 | Random |  84.40 $\pm$ 1.26 |  86.84 $\pm$ 0.39 |  88.83 $\pm$ 0.21
SVD |  87.42 $\pm$ 0.64 |  89.20 $\pm$ 0.46 |  90.94 $\pm$ 0.27
8 | Random |  88.5 $\pm$ 0.86 |  89.99 $\pm$ 0.72 |  91.4 $\pm$ 0.59
SVD |  89.84 $\pm$ 0.37 |  91.31 $\pm$ 0.10 |  92.49 $\pm$ 0.09
12 | Random |  89.27 $\pm$ 0.55 |  90.3 $\pm$ 0.58 |  92.51 $\pm$ 0.42
SVD |  90.74 $\pm$ 0.25 |  92.17 $\pm$ 0.53 |  93.32 $\pm$ 0.51
20 | Random |  89.73 $\pm$ 0.96 |  91.45 $\pm$ 0.11 |  93.23 $\pm$ 0.30
SVD |  91.49 $\pm$ 0.18 |  92.24 $\pm$ 0.24 |  94.05 $\pm$ 0.16
Table 3: The impact of initialization scheme on the performance of RoBERTa-
large on three GLUE tasks. Matrices $A$ and $B$ in LoRA-XS are initialized
either randomly or using SVD in this ablation study (see Section 4.4). We
report Matthew’s Correlation for CoLA and accuracy for SST-2 and QNLI tasks.
Each table entry reports median and standard deviation of five runs with
different seeds. Our study indicates that the SVD-based initialization
generally results in better performance, especially when dealing with lower
ranks (i.e., less trainable parameters).

##  5 Discussion

#### Parameter allocation

Our preliminary experiments indicated that when restricting the number of
parameters, it is more effective to use a lower rank $r$ and incorporate more
LoRA-XS modules. This insight guided our approach to using small ranks $r$
while adding more LoRA-XS modules compared to LoRA and VeRA, which added their
adaptation modules primarily to the Query and Value matrices in the main
experiments. By spreading LoRA-XS modules across additional components, such
as the Query, Key, Value, Attention Output, and Fully Connected weight
matrices, while keeping the rank $r$ low, we achieve a more balanced and
efficient parameter allocation without significantly increasing the overall
number of trainable parameters.

#### Learning curve of LoRA-XS

As discussed in the ablation study Section 4.4, LoRA-XS leverages Singular
Value Decomposition (SVD) to initialize adaptation matrices, aligning them
with the top principal components of pretrained weights. We observed that this
technique may lead to accelerated convergence during training. In the majority
of cases in our experiments Section 4.4, we noted better performance at the
beginning of the training (first epochs), indicating faster initial learning.
This rapid convergence sets LoRA-XS apart from other ultra parameter-efficient
approaches, such as soft prompt tuning, which typically exhibit slower
training dynamics [12, 13].

##  6 Conclusion

We introduce LoRA-XS, a novel parameter-efficient fine-tuning (PEFT) technique
that significantly reduces the number of trainable parameters while
maintaining or even enhancing model performance. LoRA-XS builds on the
principles of Low-Rank Adaptation (LoRA), incorporating an innovative approach
that aligns adaptation matrices with the principal components of pretrained
weights using SVD. This alignment not only improves parameter efficiency but
also accelerates convergence during training. Our experiments on the GLUE
benchmark demonstrate that LoRA-XS outperforms existing PEFT methods,
including the popular LoRA and recent state-of-the-art VeRA, in terms of
parameter efficiency across various tasks. We also showed that LoRA-XS is
highly effective in instruction tuning tasks on the 7B models, providing
competitive performance with significantly fewer parameters.

Due to constrained computational resources, our study is limited to only two
datasets for the 7B models, and we were unable to experiment with larger
models. Moreover, for our 7B experiments, we did not conduct an extensive grid
search for the hyperparameter optimization, which could potentially lead to
even better performance for our method.

In conclusion, LoRA-XS represents a significant advancement in parameter-
efficient fine-tuning, offering substantial reductions in trainable parameters
while maintaining high performance. Our findings underscore the importance of
initialization strategies and provide a promising direction for future
research in efficient model adaptation.

## Acknowledgments and Disclosure of Funding

The work of Klaudia Bałazy was supported by the National Centre of Science
(Poland) Grant No. 2020/39/D/ST6/ 01332\. Klaudia Bałazy is affiliated with
Doctoral School of Exact and Natural Sciences at the Jagiellonian University.

## References

  * [1] Akari Asai, Mohammadreza Salehi, Matthew E Peters, and Hannaneh Hajishirzi.  Attempt: Parameter-efficient multi-task tuning via attentional mixtures of soft prompts.  arXiv preprint arXiv:2205.11961, 2022.
  * [2] Tom Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared D Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, et al.  Language models are few-shot learners.  Advances in neural information processing systems, 33:1877–1901, 2020.
  * [3] Karl Cobbe, Vineet Kosaraju, Mohammad Bavarian, Mark Chen, Heewoo Jun, Lukasz Kaiser, Matthias Plappert, Jerry Tworek, Jacob Hilton, Reiichiro Nakano, et al.  Training verifiers to solve math word problems.  arXiv preprint arXiv:2110.14168, 2021.
  * [4] Tim Dettmers, Artidoro Pagnoni, Ari Holtzman, and Luke Zettlemoyer.  Qlora: Efficient finetuning of quantized llms.  Advances in Neural Information Processing Systems, 36, 2024.
  * [5] Nathan Halko, Per-Gunnar Martinsson, and Joel A Tropp.  Finding structure with randomness: Probabilistic algorithms for constructing approximate matrix decompositions.  SIAM review, 53(2):217–288, 2011.
  * [6] Soufiane Hayou, Nikhil Ghosh, and Bin Yu.  Lora+: Efficient low rank adaptation of large models.  arXiv preprint arXiv:2402.12354, 2024.
  * [7] Dan Hendrycks, Collin Burns, Saurav Kadavath, Akul Arora, Steven Basart, Eric Tang, Dawn Song, and Jacob Steinhardt.  Measuring mathematical problem solving with the math dataset.  arXiv preprint arXiv:2103.03874, 2021.
  * [8] Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin De Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly.  Parameter-efficient transfer learning for nlp.  In International conference on machine learning, pages 2790–2799. PMLR, 2019.
  * [9] Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen.  Lora: Low-rank adaptation of large language models.  arXiv preprint arXiv:2106.09685, 2021.
  * [10] Albert Q Jiang, Alexandre Sablayrolles, Arthur Mensch, Chris Bamford, Devendra Singh Chaplot, Diego de las Casas, Florian Bressand, Gianna Lengyel, Guillaume Lample, Lucile Saulnier, et al.  Mistral 7b.  arXiv preprint arXiv:2310.06825, 2023.
  * [11] Dawid Jan Kopiczko, Tijmen Blankevoort, and Yuki Markus Asano.  Vera: Vector-based random matrix adaptation.  arXiv preprint arXiv:2310.11454, 2023.
  * [12] Brian Lester, Rami Al-Rfou, and Noah Constant.  The power of scale for parameter-efficient prompt tuning.  arXiv preprint arXiv:2104.08691, 2021.
  * [13] Xiang Lisa Li and Percy Liang.  Prefix-tuning: Optimizing continuous prompts for generation.  arXiv preprint arXiv:2101.00190, 2021.
  * [14] Yixiao Li, Yifan Yu, Chen Liang, Pengcheng He, Nikos Karampatziakis, Weizhu Chen, and Tuo Zhao.  Loftq: Lora-fine-tuning-aware quantization for large language models.  arXiv preprint arXiv:2310.08659, 2023.
  * [15] Haokun Liu, Derek Tam, Mohammed Muqeeth, Jay Mohta, Tenghao Huang, Mohit Bansal, and Colin A Raffel.  Few-shot parameter-efficient fine-tuning is better and cheaper than in-context learning.  Advances in Neural Information Processing Systems, 35:1950–1965, 2022.
  * [16] Shih-Yang Liu, Chien-Yi Wang, Hongxu Yin, Pavlo Molchanov, Yu-Chiang Frank Wang, Kwang-Ting Cheng, and Min-Hung Chen.  Dora: Weight-decomposed low-rank adaptation.  arXiv preprint arXiv:2402.09353, 2024.
  * [17] Xiao Liu, Yanan Zheng, Zhengxiao Du, Ming Ding, Yujie Qian, Zhilin Yang, and Jie Tang.  Gpt understands, too.  AI Open, 2023.
  * [18] Yinhan Liu, Myle Ott, Naman Goyal, Jingfei Du, Mandar Joshi, Danqi Chen, Omer Levy, Mike Lewis, Luke Zettlemoyer, and Veselin Stoyanov.  Roberta: A robustly optimized bert pretraining approach.  arXiv preprint arXiv:1907.11692, 2019.
  * [19] Ilya Loshchilov and Frank Hutter.  Decoupled weight decay regularization.  arXiv preprint arXiv:1711.05101, 2017.
  * [20] Sourab Mangrulkar, Sylvain Gugger, Lysandre Debut, Younes Belkada, Sayak Paul, and Benjamin Bossan.  Peft: State-of-the-art parameter-efficient fine-tuning methods.  https://github.com/huggingface/peft, 2022.
  * [21] Fanxu Meng, Zhaohui Wang, and Muhan Zhang.  Pissa: Principal singular values and singular vectors adaptation of large language models.  arXiv preprint arXiv:2404.02948, 2024.
  * [22] Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, Luca Antiga, et al.  Pytorch: An imperative style, high-performance deep learning library.  Advances in neural information processing systems, 32, 2019.
  * [23] F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay.  Scikit-learn: Machine learning in Python.  Journal of Machine Learning Research, 12:2825–2830, 2011.
  * [24] Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych.  Adapterfusion: Non-destructive task composition for transfer learning.  arXiv preprint arXiv:2005.00247, 2020.
  * [25] Adithya Renduchintala, Tugrul Konuk, and Oleksii Kuchaiev.  Tied-lora: Enhacing parameter efficiency of lora with weight tying.  arXiv preprint arXiv:2311.09578, 2023.
  * [26] Gemma Team, Thomas Mesnard, Cassidy Hardin, Robert Dadashi, Surya Bhupatiraju, Shreya Pathak, Laurent Sifre, Morgane Rivière, Mihir Sanjay Kale, Juliette Love, et al.  Gemma: Open models based on gemini research and technology.  arXiv preprint arXiv:2403.08295, 2024.
  * [27] Ashish Vaswani, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Łukasz Kaiser, and Illia Polosukhin.  Attention is all you need.  Advances in neural information processing systems, 30, 2017.
  * [28] Alex Wang, Amanpreet Singh, Julian Michael, Felix Hill, Omer Levy, and Samuel R Bowman.  Glue: A multi-task benchmark and analysis platform for natural language understanding.  arXiv preprint arXiv:1804.07461, 2018.
  * [29] Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, et al.  Huggingface’s transformers: State-of-the-art natural language processing.  arXiv preprint arXiv:1910.03771, 2019.
  * [30] Longhui Yu, Weisen Jiang, Han Shi, Jincheng Yu, Zhengying Liu, Yu Zhang, James T Kwok, Zhenguo Li, Adrian Weller, and Weiyang Liu.  Metamath: Bootstrap your own mathematical questions for large language models.  arXiv preprint arXiv:2309.12284, 2023.
  * [31] Elad Ben Zaken, Shauli Ravfogel, and Yoav Goldberg.  Bitfit: Simple parameter-efficient fine-tuning for transformer-based masked language-models.  arXiv preprint arXiv:2106.10199, 2021.
  * [32] Qingru Zhang, Minshuo Chen, Alexander Bukharin, Pengcheng He, Yu Cheng, Weizhu Chen, and Tuo Zhao.  Adaptive budget allocation for parameter-efficient fine-tuning.  In The Eleventh International Conference on Learning Representations, 2023.

##  Appendix A Experimental Setup Details

In this section, we provide the detailed experimental setup and selected
hyperparameters for all our experiments.

As mentioned previously in Section 4, for all our main experiments, each LoRA-
XS module is initialized using Singular Value Decomposition (SVD) of the
corresponding pretrained weights $W$. The matrices are initialized using
truncated SVD with 10 iterations, implemented via the scikit-learn
library777https://scikit-
learn.org/stable/modules/generated/sklearn.decomposition.TruncatedSVD.html.

For all our experiments and the ablation study, models are trained with the
AdamW optimizer [19], following the methodology used in LoRA and VERA [9, 11].
We utilize the HuggingFace Transformers [29] implementation for Transformer-
based models [27] and implement our LoRA-XS method on top of the Huggingface
PEFT repository[20]. Detailed experimental setups and hyperparameters for the
various experiments are discussed in the following sections.

###  A.1 GLUE Benchmark

For all experiments involving the GLUE Benchmark, we incorporate LoRA-XS
modules into the Query ($W_{q}$), Value ($W_{v}$), Attention Output ($W_{o}$),
and first Fully Connected ($FC_{1}$) weight matrices of the transformer model
[27]. We use the RoBERTa-large [18] model and experiment with varying ranks
for LoRA-XS’s $R$ matrix, ranging from $r=4$ to $r=25$ (i.e., the number of
trainable parameters per linear layer being 16 and 625, respectively).
Following previous work [9, 11], for MRPC, RTE, and STS-B datasets, the LoRA
modules are initialized with weights fine-tuned on the MNLI task. Due to
computing limitations, large GLUE datasets (i.e., MNLI and QQP) are not
included in our experiments.

Hyperparameters for the experiments were selected through grid search, and the
chosen values are detailed in Table 4. The model sequence length is set to
128, with a warmup ratio of 0.06 (consistent with LoRA [9]). A fixed batch
size of 32 is used for all tasks. Training is performed on one GPU A100 40GB,
and a fixed $\alpha=8$ is used for LoRA-XS modules, following the HuggingFace
PEFT default settings.

Task | Rank | LoRA-XS LR | Classifier LR | Epochs
---|---|---|---|---
SST-2 | 4 | 1E-3 | 1E-3 | 20
8 | 1E-3 | 1E-3 | 20
12 | 5E-3 | 1E-3 | 20
16 | 1E-3 | 5E-4 | 20
20 | 1E-3 | 5E-3 | 20
25 | 2E-3 | 1E-3 | 20
MRPC | 4 | 1E-3 | 1E-3 | 50
8 | 1E-3 | 6E-4 | 50
12 | 1E-3 | 1E-3 | 50
16 | 1E-3 | 6E-4 | 50
20 | 1E-3 | 6E-4 | 50
25 | 1E-3 | 6E-4 | 50
CoLA | 4 | 1E-3 | 5E-3 | 50
8 | 1E-3 | 5E-3 | 50
12 | 1E-3 | 5E-3 | 50
16 | 1E-3 | 1E-2 | 50
20 | 1E-3 | 5E-3 | 50
25 | 1E-3 | 5E-3 | 50
QNLI | 4 | 1E-3 | 5E-4 | 10
8 | 1E-3 | 1E-3 | 10
12 | 1E-3 | 5E-4 | 10
16 | 1E-3 | 1E-3 | 15
20 | 1E-3 | 5E-4 | 10
25 | 2E-3 | 6E-4 | 15
RTE | 4 | 1E-3 | 6E-4 | 50
8 | 6E-4 | 6E-4 | 50
12 | 1E-3 | 6E-4 | 50
16 | 1E-3 | 1E-3 | 50
20 | 1E-3 | 1E-3 | 50
25 | 1E-3 | 6E-4 | 50
STS-B | 4 | 6E-4 | 6E-4 | 50
8 | 1E-3 | 1E-3 | 50
12 | 1E-3 | 1E-3 | 50
16 | 1E-3 | 1E-3 | 50
20 | 1E-3 | 6E-4 | 50
25 | 1E-3 | 6E-4 | 50
Table 4: Hyperparameters for RoBERTa-large fine-tuned with LoRA-XS for
different GLUE tasks.

###  A.2 Instruction Tuning Experiments

We perform the instruction tuning experiments on Mistral-7B-v0.1 [10] and
Gemma-7B [26] decoder-only models. We use a batch size of 128 and train for 2
epochs on 100k samples of the MetaMathQA dataset. Models are evaluated on the
GSM8K and MATH datasets. The learning rate is set to 4E-3 with the AdamW
optimizer [19]. The warmup ratio is 0.02, and a cosine learning rate scheduler
is used. The parameter $\alpha$ for LoRA-XS modules is always equal to the
rank. We used two A100 80GB GPUs for the finetuning.

###  A.3 Ablation Study

Our ablation study investigates the benefits of initializing LoRA-XS $A$ and
$B$ matrices with Singular Value Decomposition (SVD). For the baseline
approach, we use the random Kaiming Initialization for the linear layers, as
implemented in PyTorch [22]. For our proposed approach, the matrices are
initialized using truncated SVD with 10 iterations [23].

Experiments are conducted on the RoBERTa-large model across three GLUE tasks:
CoLA, SST-2, and QNLI, using different ranks for the LoRA-XS $R$ matrix,
specifically 4, 8, 12, and 20. We use a fixed batch size of 32, a model
sequence length of 128, and a warmup ratio of 0.06, consistent with LoRA [9].
The parameter $\alpha$ for LoRA-XS modules is set to 8, following the
HuggingFace PEFT default settings. Training is performed on a single GPU (H100
or A100). Hyperparameters were selected through grid search and are detailed
in Table 5.

Task | Init | Rank | LoRA-XS LR | Classifier LR | Epochs
---|---|---|---|---|---
SST-2 | SVD | 4 | 1E-3 | 1E-3 | 20
8 | 1E-3 | 1E-3 | 20
12 | 5E-3 | 1E-3 | 20
20 | 1E-3 | 5E-3 | 20
Random | 4 | 1E-3 | 5E-4 | 20
8 | 1E-3 | 1E-3 | 20
12 | 1E-3 | 1E-3 | 20
20 | 5E-3 | 1E-4 | 20
CoLA | SVD | 4 | 1E-3 | 5E-3 | 50
8 | 1E-3 | 5E-3 | 50
12 | 1E-3 | 5E-3 | 50
20 | 1E-3 | 5E-3 | 50
Random | 4 | 1E-3 | 5E-3 | 50
8 | 1E-3 | 1E-2 | 50
12 | 1E-3 | 1E-2 | 50
20 | 5E-3 | 5E-3 | 50
QNLI | SVD | 4 | 1E-3 | 5E-4 | 10
8 | 1E-3 | 1E-3 | 10
12 | 1E-3 | 5E-4 | 10
20 | 1E-3 | 5E-4 | 10
Random | 4 | 1E-3 | 5E-4 | 10
8 | 1E-3 | 1E-3 | 10
12 | 1E-3 | 5E-4 | 10
20 | 1E-3 | 1E-3 | 10
Table 5: Hyperparameters for the ablation study with different $A$ and $B$
initialization methods (SVD and random) on RoBERTa-large with LoRA-XS across
various GLUE tasks.


# Nonlocal electrodynamics and the penetration depth of superconducting
Sr2RuO4

Henrik S. Røising<EMAIL_ADDRESS>Niels Bohr Institute, University
of Copenhagen, DK-2200 Copenhagen, Denmark  Andreas Kreisel  Niels Bohr
Institute, University of Copenhagen, DK-2200 Copenhagen, Denmark  Brian M.
Andersen  Niels Bohr Institute, University of Copenhagen, DK-2200 Copenhagen,
Denmark

###### Abstract

The thermal quasiparticles in a clean type-II superconductor with line nodes
give rise to a quadratic low-temperature change of the penetration depth,
$\Delta\lambda\sim T^{2}$, as first shown by Kosztin and Leggett [I. Kosztin
and A. J. Leggett, Phys. Rev. Lett. 79, 135 (1997)]. Here, we generalize this
result to multiple nodes and compare it to numerically exact evaluations of
the temperature-dependent penetration depth in Sr2RuO4 using a high-precision
tight-binding model. We compare the calculations to recent penetration depth
measurements in high purity single crystals of Sr2RuO4 [J. F. Landaeta et al.,
arXiv:2312.05129]. When assuming the order parameter to have
$\mathrm{B}_{1\mathrm{g}}$ symmetry, we find that both a simple
$d_{x^{2}-y^{2}}$-wave and complicated gap structures with contributions from
higher harmonics and accidental nodes can accommodate the experimental data.

##  I Introduction

Obtaining a detailed understanding of the superconducting state in Sr2RuO4
remains an important outstanding problem [1, 2, 3]. Experimentally, the quest
is complicated due to challenging material properties of Sr2RuO4 and the low
energy scale of the superconducting phase. Theoretically, Sr2RuO4 provides an
important testbed for modeling of unconventional superconductivity. More
specifically, the well-characterized electronic properties of the normal state
offers a rather unique opportunity to test various electronic fluctuation-
based mechanisms for pairing against detailed measurements of the
superconducting gap structure [4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16,
17, 18, 19, 20, 21, 22, 23].

The magnetic penetration depth, $\lambda$, can elucidate nodal features of the
gap, as revealed through its dependency on temperature and disorder [24, 25,
26, 27]. Fully-gapped superconductors exhibit exponential temperature
dependence at low temperatures, whereas nodal gaps display power law
dependence. In Sr2RuO4, measurements of the change in penetration depth,
$\Delta\lambda(T)$, via a tunnel diode oscillator method yielded
$\Delta\lambda(T)\sim T^{2}$, signalling the existence of nodal quasiparticle
excitations [28]. Renewed measurements of $\Delta\lambda(T)$ in Sr2RuO4 high-
purity spherical single crystals by applying scanning SQUID microscopy [29]
and ac-susceptibility measurements [30] have confirmed this behavior and
highlighted the importance of nonlocal Meissner screening. Indeed, as
initially demonstrated by Kosztin and Leggett, nonlocal effects may change the
low temperature dependence of the penetration depth, e.g., from linear to
quadratic in $d_{x^{2}-y^{2}}$-wave superconductors, such as the cuprates [31,
32]. The relevance of nonlocal effects in an extended temperature regime is
controlled by the zero-temperature Ginzburg–Landau parameter, i.e., the ratio
of the penetration depth to the coherence length. In Sr2RuO4 this ratio is
$\kappa_{0}\approx 1.92$ [30], placing this material unusually close to the
Pippard limit compared to most clean, unconventional superconductors. Finally,
we note that the penetration depth was also recently extracted from SQUID
susceptometry in thin Sr2RuO4 films, again yielding $\Delta\lambda(T)\sim
T^{2}$ behavior at low $T$ [33]. There, however, the origin of the $T^{2}$
dependence was interpreted in terms of disorder scattering similar to
disordered $d_{x^{2}-y^{2}}$-wave superconductors [34].

Here we perform a theoretical study of the nonlocal electrodynamics
specifically relevant for nodal multi-band superconductors. We perform both an
exact numerical evaluation of $\Delta\lambda(T)$ and compare this to a
generalized nonlocal node-expansion similar to the method by Kosztin and
Leggett [31], but for cases with several distinct nodes in the Brillouin zone,
possibly distributed across multiple bands. To examine whether the
superconducting order parameter can be constrained from low-temperature
penetration depth data, we apply the developed theory to Sr2RuO4. To minimize
uncertainties in the description of the normal state, we use a tight-binding
model that fits both the experimental Fermi surface and Fermi velocity with
unprecedented accuracy. We explore two nodal superconducting gaps with B1g
symmetry, relevant for Sr2RuO4, and compare the node-expansion with the
numerically exact result. The analytical analysis shows that the low-
temperature slope of $\Delta\lambda(T)$ as a function of $(T/T_{c})^{2}$ is
controlled by the sum of reciprocal gap velocities at the order parameter
nodes and the corresponding Fermi velocities, the latter are experimentally
known from ARPES measurements [35]. The analysis implies that the penetration
depth in isolation is not sensitive to the details of the gap structure since
the effect of having one shallow node can be compensated for by instead having
two steeper nodes etc. The numerical simulations reveal that both gap
structures explored can explain the presently available experimental data for
the penetration depth of Sr2RuO4.

##  II Penetration depth

Kosztin and Leggett showed that thermal quasiparticles in a clean, type-II,
nodal $d$-wave superconductor are responsible for the behavior
$\Delta\lambda\sim T^{2}$ for $T<T^{\ast}$ with
$T^{\ast}=\Delta_{0}/\kappa_{0}$, where $\Delta_{0}$ is gap scale and
$\kappa_{0}=\lambda_{0}/\xi_{0}$ ($\lambda_{0}$ and $\xi_{0}$ is the
penetration depth and the coherence length, respectively) the zero-temperature
Ginzburg–Landau parameter [31]. The key ingredient here are the nonlocal
effects (diverging coherence length) experienced by the Cooper pairs formed by
momenta near the nodal points, see Fig. 1. This causes the penetration depth
increase to acquire an additional factor of $T$ (on top of the
$2k_{B}T\ln{2}/\Delta_{0}$ local contribution) coming from the inverse thermal
de Broglie wavelength at low temperatures.

In Sr2RuO4 the behaviour $\Delta\lambda\sim T^{2}$ is observed in a dominant
fraction of the temperature window below $T_{c}\approx 1.5$ K, which is
consistent with this material being a marginal type-II superconductor with
$\kappa_{0}\approx 1.92>1/\sqrt{2}$ [30, 28, 29] and $\Delta_{0}\approx 0.35$
meV [36, 29]. Interestingly, Sr2RuO4 is closer to the Pippard limit than most
unconventional type-II superconductors. Owing to its clean crystals, this
points to the importance of nonlocal electrodynamics to understand its
superconducting state, possibly also its response to a weak magnetic field as
posed by muons [37].

###  II.1 Nonlocal electrodynamics

In a weak magnetic field, linear response theory for the Meissner state
dictates the decay of the magnetic field into the superconductor, as per
$j(y)=-\int\mathrm{d}y^{\prime}~{}K(y-y^{\prime})A(y^{\prime})$. Here, $j$ is
the screening supercurrent density (along $\hat{x}$) with the boundary between
vacuum and the superconductor at $y=0$, and $A$ is the magnetic vector
potential. The geometry is shown in Fig. 1.

Figure 1: Geometry and magnetic field (along $\hat{z}$) penetration into the
Meissner state of the superconductor located in the half-plane $y>0$. The
magnetic penetration depth is denoted by $\lambda$. Inset: reciprocal space of
a tetragonal crystal with the Fermi surface of Sr2RuO4 and the nodal line
(dashed) of the B1g irreducible representation indicated. Close to the nodal
point(s), the Cooper pairs are characterized by a long coherence length and
hence acquire nonlocal contributions in their electromagnetic response.

In dimensionless units, $\tilde{K}=(4\pi\lambda_{0}^{2}/c)K$, and with the
kernel difference defined as
$\delta\tilde{K}(\tilde{q};T)\equiv\tilde{K}(\tilde{q};T)-\tilde{K}(\tilde{q};0)$,
the penetration depth change can be expressed as

$\frac{\Delta\lambda(T)}{\lambda_{0}}=\frac{2}{\pi}\int_{0}^{\infty}\frac{-\delta\tilde{K}(\tilde{q};T)~{}\mathrm{d}\tilde{q}}{\big{(}\tilde{q}^{2}+\tilde{K}(\tilde{q};0)\big{)}\big{(}\tilde{q}^{2}+1+\delta\tilde{K}(\tilde{q};T)\big{)}},$
(1)

where $\tilde{q}=q\lambda_{0}$. We emphasize that this is an exact expression
that includes a correction responsible for an upturn in $\Delta\lambda$ at
higher $T$ as compared to Ref. 31. The dimensionless kernel can be computed by
means of standard Green’s function methods, assuming linear response in the
Meissner state and solving the relevant Maxwell equation in the superconductor
[38]. In the Matsubara representation the kernel can be expressed as [38, 31]

$\tilde{K}(\tilde{q};T)=2\pi
k_{B}T\sum_{n=-\infty}^{\infty}\left\langle\frac{\hat{p}_{\parallel}^{2}\Delta_{\boldsymbol{p}}^{2}}{\sqrt{\omega_{n}^{2}+\Delta_{\boldsymbol{p}}^{2}}(\omega_{n}^{2}+\Delta_{\boldsymbol{p}}^{2}+\alpha^{2})}\right\rangle_{\mathrm{FS}},$
(2)

where the fermionic Matsubara frequencies are given by $\omega_{n}=\pi
k_{B}T(2n+1)$, $\hat{p}_{\parallel}=\cos(\theta)$ is the Fermi surface
momentum projected onto the boundary ($\theta$ is the polar angle of
$\boldsymbol{p}$), and the parameter
$\alpha=\pi\Delta_{0}\tilde{q}\sin(\theta)/(2\kappa_{0})$ is the projected
magnetic field penetration and is responsible for the nonlocal effects. Here,
we employed the BCS expression for the coherence length, such that
$\lambda_{0}=v_{F}\kappa_{0}/(\pi\Delta_{0})$. Finally,
$\Delta_{\boldsymbol{p}}$ is the (temperature-dependent) order parameter, and
the (dimensionless) Fermi surface average is evaluated as $\langle
h\rangle_{\mathrm{FS}}\equiv(\bar{v}_{F}/\lvert
S_{F}\rvert)\int_{S_{F}}\mathrm{d}\hat{\boldsymbol{k}}~{}h(\hat{\boldsymbol{k}})/v_{F}(\hat{\boldsymbol{k}})$
where $v_{F}(\hat{\boldsymbol{k}})$ is the Fermi velocity and $\lvert
S_{F}\rvert$ the Fermi surface area. It can be noted that the zero-temperature
kernel, $\tilde{K}(\tilde{q};0)$, has a simple closed-form expression, as
stated in Appendix A.

In practice, realistic modelling of $\Delta\lambda/\lambda_{0}$ in the entire
temperature window below $T_{c}$ is achieved by feeding in a high-precision
multiband tight-binding model, as well as experimental values for
$\kappa_{0}$, $\Delta_{0}$, and $T_{c}$. Since the sum in Eq. (2) converges
rapidly, one can in practice calculate the Fermi surface average for each
frequency and truncate the Matsubara series at some
$n\gg\frac{\Delta_{0}}{2\pi k_{B}T}$ for any non-zero $T$ 111If one instead
approaches the problem of evaluating $\delta\tilde{K}$ after recasting the
Matsubara sum as an integral, care has to be taken with respect to the
integrable singularities that emerge at the endpoints of the $\omega$
integration, cf. Eq. (15).. If one models the $T$ dependence of the gap by the
interpolation formula $\tanh{(1.74\sqrt{T/T_{c}-1})}$, the only remaining
degrees of freedom lie in the momentum-dependent gap structure.

###  II.2 The node approximation

Building on the Kosztin–Leggett philosophy at the lowest $T$, useful insights
can be harvested by first rewriting Eq. (2) using contour integration
techniques, and then linearizing the gap around its nodes,
$\Delta_{\boldsymbol{p}}\approx\sum_{j}v_{\Delta,j}(\theta-\theta_{j})$, where
the thermally active quasiparticles are situated at the lowest $T$. We will
henceforth refer to $v_{\Delta,j}$ as the “gap velocity” at node $j$. The
dimensionless kernel difference, with the intermediate steps shown in Appendix
A, can in this case be recast as

$\displaystyle-\delta\tilde{K}(\tilde{q};T)$ $\displaystyle\approx
k_{B}T\frac{2\pi\ln{2}}{\lvert
S_{F}\rvert}\sum_{j}\frac{2\hat{p}_{\parallel,j}^{2}\bar{v}_{F}}{v_{F,j}v_{\Delta,j}}\Big{\lvert}\frac{\partial\boldsymbol{k}_{F}}{\partial\theta}\Big{\rvert}_{j}$
(3)
$\displaystyle\times\left[1-\frac{1}{\ln{2}}\int_{0}^{\alpha_{j}/T}\mathrm{d}x~{}\tilde{f}(x)\sqrt{1-\left(xT/\alpha_{j}\right)^{2}}\right],$

where $\tilde{f}(x)=(1+\exp(x))^{-1}$ is the Fermi function and where the sum
runs over distinct nodes in the Brillouin zone (possibly distributed across
multiple bands). The symbols are otherwise explained earlier. This result is a
multiband generalization of the node approximation proposed by Kosztin and
Leggett. To make evaluation fast we further approximate
$\tilde{K}(\tilde{q};0)\approx 1=\tilde{K}(0;0)$ in Eq. (1) when evaluating
the penetration depth from the node approximation in the next section.

To validate the above expression, we benchmark it in the simple
$d_{x^{2}-y^{2}}$-wave case
($\Delta_{\boldsymbol{p}}=\Delta_{0}\cos(2\theta)$) using a circular Fermi
surface and isotropic Fermi velocity. The four distinct nodes all have
$\hat{p}_{\parallel}^{2}=1/2$ and the associated gap velocities are
$v_{\Delta}=2\Delta_{0}$. In this case we recover the standard result
$\delta\tilde{K}(\tilde{q};T)=\delta\tilde{K}(0;T)F(\tilde{q}/t)$, where
$t=T/T^{\ast}$, with $T^{\ast}=\Delta_{0}/\kappa_{0}$, and $F$ is a universal
function similar to the lower line in Eq. (3), stated in Appendix A. The
prefactor reduces to the well-known
$-\delta\tilde{K}(0;T)=2k_{B}T\ln{2}/\Delta_{0}$, i.e., the local result [34,
32, 40]. The function $F$ ensures that below the characteristic temperature
scale $T^{\ast}$, the change in the penetration depth depends quadratically on
temperature, $\Delta\lambda\sim T^{2}$.

For the multinode generalization in Eq. (3) the above-mentioned factorization
breaks down, and the kernel difference instead depends on all of the distinct
$\tilde{q}/t_{j}$ where
$t_{j}\propto\frac{\Delta_{0}}{\kappa_{0}}\sin(\theta_{j})$ for each distinct
gap node $j$. Thus, each distinct gap node is associated with a characteristic
temperature scale, the minimum of which dictates the regime in which
$\Delta\lambda\sim T^{2}$. Some key insights are gained by the details added
to Eq. (3) caused by having a non-isotropic Fermi surface and higher-harmonic
gap structure, possibly with multiple distinct nodes. Equation (3) tells us
that $\lim_{T\to T_{c}}\Delta\lambda(T)/(T/T_{c})^{2}$ is proportional to a
weighted sum over the distinct gap nodes, where the weight contains a product
of the reciprocal gap velocity and the reciprocal Fermi velocity. The primer
is strain-tunable and implies an enhanced sensitivity to gap structures with
nodes at strain-induced van Hove points. In the context of Sr2RuO4, no
substantial variations in the penetration depth slope change are detected as
uniaxial strain is applied [29], which argues against both B2g ($d_{xy}$) and
A2g ($g_{xy(x^{2}-y^{2})}$) type orders, which is also consistent with
elastocaloric measurements [41]. Generally, however, the dependency on the
reciprocal gap velocities implies that penetration depth data alone do not
impose any crisp constraints on the momentum structure of the gap, since the
effect of having one shallow node can be compensated for by having two steeper
nodes.

There are at least two reasons why there are few candidate materials for
observing $\Delta\lambda\sim T^{2}$ from the above mechanism. First, disorder
is known to cause a saturation of $\Delta\lambda$ below a characteristic
temperature set by the scattering rate [34], which will effectively blur the
observation of a quadratic temperature dependence. Second, the most well-
characterized $d$-wave superconductors are strong type-II, $\kappa_{0}\gg 1$,
suppressing $T^{\ast}$ to a tiny fraction of the gap. In both of these
respects Sr2RuO4 poses as a counterexample, since it has accumulated strong
evidence of being both nodal [42, 43], marginal type-II [30], and is known to
produce clean and disorder-sensitive crystals [44]. Other candidate
superconductors possibly relevant to a low-$T$ penetration depth of
$\Delta\lambda\sim T^{2}$ caused by nonlocal electrodynamics include cuprates
[31], KFe2As2 [45], and the heavy-fermion material CeCoIn5 [46, 24]. Below we
focus on the case of Sr2RuO4.

##  III The case of Strontium Ruthenate

###  III.1 Tight-binding model

We depart from a standard three-band tight-binding model ansatz for Sr2RuO4,

$H_{0}=\sum_{\boldsymbol{k},\sigma}\vec{\psi}^{\dagger}_{\sigma}(\boldsymbol{k})\pazocal{H}_{\sigma}(\boldsymbol{k})\vec{\psi}_{\sigma}(\boldsymbol{k}),$
(4)

where $\vec{\psi}_{s}(\boldsymbol{k})=[c_{xz,s}(\boldsymbol{k}),\hskip
2.84526ptc_{yz,s}(\boldsymbol{k}),\hskip
2.84526ptc_{xy,-s}(\boldsymbol{k})]^{\mathrm{T}}$, $\sigma=\pm$ denotes spin,
and $a\in\\{xz,yz,xy\\}$ labels the Ru $d$-orbitals (the $t_{2\mathrm{g}}$
triplet), which are the only relevant orbitals close to the Fermi energy [47].

Figure 2: Fermi surface and Fermi velocities of Sr2RuO4. Model ($k_{z}=0$):
solid lines (labelled “TB”) show the model Fermi surface (top panel) and the
Fermi velocities (bottom panel). The second quadrant in the top panel shows
the Ru $d_{xy}$ orbital content of the bands. The average model Fermi velocity
is $\bar{v}_{F}=0.498$ eVÅ. Data: high-resolution ARPES Fermi surface (dark
blue, top panel) and the Fermi velocities (circles with error bars, bottom
panel) extracted from Ref. 35.

The form of $\pazocal{H}_{\sigma}$ is

$\pazocal{H}_{\sigma}(\boldsymbol{k})=\begin{pmatrix}\xi_{xz}(\boldsymbol{k})&\xi_{xz,yz}(\boldsymbol{k})-i\sigma\eta&i\eta\\\
\xi_{xz,yz}(\boldsymbol{k})+i\sigma\eta&\xi_{yz}(\boldsymbol{k})&-\sigma\eta\\\
-i\eta&-\sigma\eta&\xi_{xy}(\boldsymbol{k})\end{pmatrix},$ (5)

where spin-orbit coupling is parametrized by $\eta$ and originates from the
(dominant) onsite term $2\eta\sum_{i}\vec{L}_{i}\cdot\vec{S}_{i}$ as projected
onto the $t_{2g}$ Ru triplet. Explicit forms of the inter- and intraband
energies are listed in Appendix B.

To obtain a quantitatively accurate parametrization, we consider as a starting
point the tight-binding parameters from Ref. 23 as derived from relativistic
DFT calculations. While these parameters provide a realistic energy scale and
Fermi surface, they still do not quantitatively match the Fermi velocities as
extracted from high-resolution ARPES measurements for bands $\beta$ and
$\gamma$ in Ref. 35. To correct for this discrepancy, which is also present in
another tight-binding model widely employed in the literature [48], we
manually tune tight-binding parameters until a reasonable match with both the
experimental Fermi surface _and_ Fermi velocity $v_{F}$ is obtained. The
result is shown in Fig. 2. The effective model, with the set of parameters
listed in Appendix B, provides a high-precision effective normal state
description of Sr2RuO4. We stress that a model that is qualitatively and
quantitatively accurate in the above respect could be crucial to accurately
perform calculations sensitive to $v_{F}$.

###  III.2 Numerical evaluation of the penetration depth

Equipped with an accurate description of the normal state, we now turn to the
evaluation of the penetration depth difference,
$\Delta\lambda(T)/\lambda_{0}$, for some illustrative order parameters.

Figure 3: Dimensionless kernel differences, $-\delta\tilde{K}(\tilde{q};T)$,
calculated at $T=0.15T_{c}$ and $T=0.6T_{c}$ for both the node approximation
of Eq. (3) (“node approx.”) and in the Matsubara representation (“model”). We
use the three-band tight-binding model of Sec. III.1 and order parameters of
Fig. 4(a). Due to the curvature of $\Delta(\theta)$, the node approximation
overestimates the gap and underestimates the penetration depth at higher
temperatures. Figure 4: Representative B1g order parameters [(a) and (b)], and
associated penetration depths [(c) associated with (a), and (d) associated
with (b)] using both the node approximation (“node approx.”) and the Mastubara
representation (“model”) to calculate the kernel. We use the three-band tight-
binding model of Sec. III.1. Experimental data points from Refs. 28, 30 are
plotted as gray triangles and orange disks for comparison. In (a) and (b)
linearizations around the gap nodes are shown with dashed lines.

First, we calculate the kernel difference $-\delta\tilde{K}(\tilde{q};T)$ at
two temperatures using both the node approximation of Eq. (3) and the full
Matsubara sum of Eq. (2). For illustration we employ the simplest gap
structure consistent with B1g symmetry, i.e., $\Delta_{0}\cos(2\theta)$ on all
bands with $\Delta_{0}=0.35$ meV on bands $\alpha$ and $\beta$, and half the
magnitude on band $\gamma$. These values are motivated by STM experiments [36,
29], and the gap magnitude on $\gamma$ was reduced to match the experimental
penetration depth slope at low $T$. We otherwise fixed $\kappa_{0}=1.92$ and
$T_{c}=1.5$ K, in agreement with experiments [30]. The resulting kernel
differences are shown in Fig. 3, and the order parameter and penetration depth
are shown in Fig. 4(a) and (c).

Comparing the penetration depth calculations with the experimental data
reveals that the temperature window in which the node approximation matches
with the realistic modelling is roughly $T/T_{c}\lesssim 0.2$. The primary
reason for this is the overestimated slope of $\lvert\Delta(\theta)\rvert$
when linearizing the gap, causing the node approximation to, in this case,
monotonically underestimate the penetration depth. Additionally, the $T$
dependence of the gap and the correction posed by the denominator of Eq. (1)
both contribute with an upturn in $\Delta\lambda(T)$ close to the transition
temperature $T_{c}$, approximately consistent with the experimental data. The
realistic modelling shows that a simple $d_{x^{2}-y^{2}}$-wave gap is
sufficient to explain the experimental data, albeit with a slight discrepancy
at the highest $T$.

In Fig. 4(b) and (d) we show the results of calculating the penetration depth
for a more involved order parameter, still within the B1g irreducible
representation, but with an angular dependence inspired by weak-coupling and
RPA spin-fluctuation calculations [6, 11, 49, 50]. In this case, the node
approximation performs better when comparing to the realistic modelling. Since
the order parameter is more involved by having contributions from multiple
harmonics, the error introduced by linearizing the gap is non-monotonic, i.e.,
the gap is both over- and underestimated on the various bands. As the
realistic modelling shows, an equally convincing $T$-dependent penetration
depth is obtained for this order parameter. Therefore, not surprisingly, the
change in the penetration depth is not capable of resolving subtle differences
in the the nodal structure of the order parameter. As the nodal expansion
reveals, the slope of $\Delta\lambda(T)$ as a function of $(T/T_{c})^{2}$ at
the lowest temperatures is a weighted sum of reciprocal gap velocities at the
nodes, so the increase in slope gained by an additional node can be
compensated for by increasing the gap velocity of one or more nodes.

##  IV Conclusions

In summary, we have calculated the temperature-dependent change of the
penetration depth $\Delta\lambda$ including nonlocal effects from line nodes.
We have provided both a multi-band multi-node generalization of the Kosztin-
Leggett result [31], and demonstrated a straightforward numerical procedure to
evaluate $\Delta\lambda$ numerically exact, given a multi-band tight-binding
description. Focusing on the case of Sr2RuO4, posing as a prime candidate
material owing to its clean crystals and evidence of nodal order, we
investigated two $d$-wave gap structures with different nodal properties. The
analysis reveals that the low-temperature penetration depth is sensitive to
the sum of reciprocal gap velocities at the nodes, and that both orders
investigated have nodal properties compatible with the presently available
data.

##  V Acknowledgements

We acknowledge useful discussions with C. Hicks. H.S.R. was supported by
research Grant No. 40509 from VILLUM FONDEN. A.K. acknowledges support by the
Danish National Committee for Research Infrastructure (NUFI) through the ESS-
Lighthouse Q-MAT.

##  Appendix A Generalized Kosztin–Leggett theory

Here, we re-derive and generalize the central result of Kosztin and Leggett
[31] for the penetration depth of a nodal type-II superconductor.

In a weak magnetic field, the Meissner state of a superconductor responds
linearly to the perturbation,

$j(y)=-\int\mathrm{d}y^{\prime}K(y-y^{\prime})A(y^{\prime}).$ (6)

Here, $j$ is the screening supercurrent density (pointing along $\hat{x}$,
$y=0$ is the position of the superconductor boundary), $K$ is the
electromagnetic response kernel, and $A$ is the magnetic vector potential.
With a specular boundary, the magnetic penetration depth is given by

$\frac{\lambda(T)}{\lambda_{0}}=\frac{2}{\pi}\int_{0}^{\infty}\frac{\mathrm{d}\tilde{q}}{\tilde{q}^{2}+\tilde{K}(\tilde{q};T)},$
(7)

where $\tilde{q}=\lambda_{0}q$ with $\lambda_{0}\equiv\lambda(0)$, and
$\tilde{K}=(4\pi\lambda_{0}^{2}/c)K$. The dimensionless kernel satisfies
$\tilde{K}(\tilde{q}\to 0;0)=1$ in the local limit. Writing
$\tilde{K}(\tilde{q};T)=1+\delta\tilde{K}(\tilde{q};T)$, with
$\delta\tilde{K}(\tilde{q};T)\equiv\tilde{K}(\tilde{q};T)-\tilde{K}(\tilde{q};0)$
leads to the exact expression for
$\Delta\lambda(T)\equiv\lambda(T)-\lambda_{0}$ given in the main text Eq. (1).
The zero temperature kernel can be evaluated analytically [31], with

$\tilde{K}(\tilde{q};0)=1-\left\langle
2\hat{p}_{\parallel}^{2}\left[1-\frac{\mathrm{arcsinh}(\alpha/\Delta_{\boldsymbol{p}})}{\alpha/\Delta_{\boldsymbol{p}}\sqrt{1+(\alpha/\Delta_{\boldsymbol{p}})^{2}}}\right]\right\rangle_{\mathrm{FS}}.$
(8)

Using contour integration techniques [38], the response kernel correction can
be evaluated as

$\displaystyle-\delta\tilde{K}(\tilde{q};T)$
$\displaystyle=2\int_{0}^{\infty}\mathrm{d}\omega~{}f(\omega)$ (9)
$\displaystyle\times\left\langle
2\hat{p}_{\parallel}^{2}\mathrm{Re}\frac{\Delta_{\boldsymbol{p}}^{2}}{\sqrt{\omega^{2}-\Delta_{\boldsymbol{p}}^{2}}(\Delta_{\boldsymbol{p}}^{2}-\omega^{2}+\alpha^{2})}\right\rangle_{\mathrm{FS}},$

where $f(\omega)=(1+\exp(\beta\omega))^{-1}$ is the Fermi function,
$\Delta_{\boldsymbol{p}}$ is the order parameter,
$\hat{p}_{\parallel}=\cos(\theta)$ is the projection of the Fermi surface
momentum on the $\hat{x}$-axis, and $\alpha=qv_{F}\sin{\theta}/2$. Setting
$\alpha=0$ reproduces the local result [34].

To evaluate the Fermi surface average, we first recast momentum sums as
integrals over $(\xi,\hskip 2.84526pt\hat{\boldsymbol{k}})$, where
$\hat{\boldsymbol{k}}$ lies on the Fermi surface defined by
$S_{F}(\xi)\equiv\\{\hat{\boldsymbol{k}}:~{}\xi_{\hat{\boldsymbol{k}}}=\xi\\}$
in the following manner (introducing also the electronic cutoff $\omega_{c}$):

$\sum_{\boldsymbol{k}:~{}\lvert\xi_{\boldsymbol{k}}\rvert<\omega_{c}}h_{\boldsymbol{k}}=\int_{-\omega_{c}}^{\omega_{c}}\mathrm{d}\xi\hskip
2.84526pt\rho_{\xi}\int_{S_{F}(\xi)}\frac{\mathrm{d}\hat{\boldsymbol{k}}}{\lvert
S_{F}\rvert}\frac{\bar{v}_{F}}{v_{F}(\hat{\boldsymbol{k}})}h(\hat{\boldsymbol{k}}),$
(10)

where the Fermi velocity, the average Fermi velocity, and the density of
states are given by

$\displaystyle v_{F}(\hat{\boldsymbol{k}})$
$\displaystyle=\lvert\nabla\xi_{\hat{\boldsymbol{k}}}\rvert,$ (11)
$\displaystyle\bar{v}_{F}$
$\displaystyle=\Big{[}\int_{S_{F}(\xi)}\frac{\mathrm{d}\hat{\boldsymbol{k}}}{\lvert
S_{F}\rvert}\frac{1}{v_{F}(\hat{\boldsymbol{k}})}\Big{]}^{-1},$ (12)
$\displaystyle\rho_{\xi}$
$\displaystyle=\int_{S_{F}(\xi)}\frac{\mathrm{d}\hat{\boldsymbol{k}}}{(2\pi)^{d}}\frac{1}{v_{F}(\hat{\boldsymbol{k}})},$
(13)

respectively, and where $\lvert S_{F}\rvert$ is the Fermi surface area. From
the above we define the (dimensionless) Fermi surface average as

$\langle
A\rangle_{\mathrm{FS}}\equiv\int_{S_{F}(\xi)}\frac{\mathrm{d}\hat{\boldsymbol{k}}}{\lvert
S_{F}\rvert}\frac{\bar{v}_{F}}{v_{F}(\hat{\boldsymbol{k}})}A,$ (14)

such that $\langle 1\rangle_{\mathrm{FS}}=1$.

We next expand the order parameter around its nodes, situated at angles
$\theta_{j}$,
$\Delta_{\boldsymbol{p}}\approx\sum_{j}v_{\Delta,j}(\theta-\theta_{j})$, where
$v_{\Delta,j}$ is the “gap velocity” of node $j$. Close to the nodes (at low
temperatures), we can safely ignore the angular dependence of $\alpha$ and
$\hat{p}_{\parallel}$. Since Eq. (9) picks up contributions around each node,
we then get

$\displaystyle-\delta\tilde{K}(\tilde{q};T)$
$\displaystyle=2\int_{0}^{\infty}\mathrm{d}\omega~{}f(\omega)\left\langle
2\hat{p}_{\parallel}^{2}\mathrm{Re}\frac{\Delta_{\boldsymbol{p}}^{2}}{\sqrt{\omega^{2}-\Delta_{\boldsymbol{p}}^{2}}(\Delta_{\boldsymbol{p}}^{2}-\omega^{2}+\alpha^{2})}\right\rangle_{\mathrm{FS}}$
(15) $\displaystyle\approx\frac{2}{\lvert
S_{F}\rvert}\int_{0}^{\infty}\mathrm{d}\omega~{}f(\omega)\sum_{j}\frac{2\hat{p}_{\parallel,j}^{2}\bar{v}_{F}}{v_{F,j}v_{\Delta,j}}\Big{\lvert}\frac{\partial\boldsymbol{k}_{F}}{\partial\theta}\Big{\rvert}_{j}\int_{-\omega}^{\omega}\mathrm{Re}\frac{\mathrm{d}u~{}u^{2}}{\sqrt{\omega^{2}-u^{2}}(u^{2}-\omega^{2}+\alpha_{j}^{2})}$
$\displaystyle=\frac{2\pi}{\lvert
S_{F}\rvert}\int_{0}^{\infty}\mathrm{d}\omega~{}f(\omega)\sum_{j}\frac{2\hat{p}_{\parallel,j}^{2}\bar{v}_{F}}{v_{F,j}v_{\Delta,j}}\Big{\lvert}\frac{\partial\boldsymbol{k}_{F}}{\partial\theta}\Big{\rvert}_{j}\mathrm{Re}\left[1-\sqrt{1-\omega^{2}/\alpha_{j}^{2}}\right]$
$\displaystyle=k_{B}T\frac{2\pi\ln{2}}{\lvert
S_{F}\rvert}\sum_{j}\frac{2\hat{p}_{\parallel,j}^{2}\bar{v}_{F}}{v_{F,j}v_{\Delta,j}}\Big{\lvert}\frac{\partial\boldsymbol{k}_{F}}{\partial\theta}\Big{\rvert}_{j}\left[1-\frac{1}{\ln{2}}\int_{0}^{\alpha_{j}/T}\mathrm{d}x~{}\tilde{f}(x)\sqrt{1-\left(xT/\alpha_{j}\right)^{2}}\right],$

where $\alpha_{j}=\tilde{q}v_{F,j}\sin(\theta_{j})/(2\lambda_{0})$,
$\tilde{f}(x)\equiv(1+\exp(x))^{-1}$ is the Fermi function with dimensionless
argument, and where the sum runs over distinct nodes in the Brillouin zone
(possibly distributed across multiple bands). This result is a multiband
generalization of the node approximation proposed by Kosztin and Leggett,
which is a low-temperature approximation that is valid in the temperature
window in which the order parameter can be reasonably approximated by a linear
function of the angle deviation from the node.

To validate the generalized (multiband) expression of Eq. (15), we evaluate it
for the simple $d$-wave order parameter
$\Delta_{\boldsymbol{p}}=\Delta_{0}\cos(2\theta)$ and a circular Fermi surface
with isotropic Fermi velocity [31]. The four distinct nodes all have
$\hat{p}_{\parallel}^{2}=1/2$ and gap velocity $v_{\Delta}=2\Delta_{0}$.
Further writing $\lambda_{0}=\kappa_{0}\xi_{0}$, and using the BCS expression
for the coherence length, $\xi_{0}=v_{F}/(\pi\Delta_{0})$, leads to

$\displaystyle-\delta\tilde{K}(\tilde{q};T)$
$\displaystyle=\frac{2k_{B}T\ln{2}}{\Delta_{0}}$ (16)
$\displaystyle\times\left[1-\frac{1}{\ln{2}}\int_{0}^{\pi\frac{\sqrt{2}z}{4}}\mathrm{d}x~{}\tilde{f}(x)\sqrt{1-\frac{8x^{2}}{\pi^{2}z^{2}}}\right],$

where $z\equiv\frac{\tilde{q}}{T/T^{\ast}}$, and
$T^{\ast}\equiv\frac{\Delta_{0}}{\kappa_{0}}$, where $\kappa_{0}$ is the zero-
temperature Ginzburg–Landau parameter, and the prefactor is recognized as
$-\delta\tilde{K}(0;T)=2k_{B}T\ln{2}/\Delta_{0}$.

##  Appendix B Details of the tight-binding model

The inter- and intra-orbital energies in Eqs. (4) and (5) take the form

$\displaystyle\xi_{xz}(\boldsymbol{k})$
$\displaystyle=-t_{1}\cos{k_{x}}-t_{2}\cos{k_{y}}-t_{3}\cos{k_{x}}\cos{k_{y}}-t_{4}\cos(2k_{x})-t_{5}\cos(2k_{x})\cos{k_{y}}$
$\displaystyle\hskip 20.0pt-
t_{6}\cos(3k_{x})-t_{7}\cos(k_{x}/2)\cos(k_{y}/2)\cos(k_{z}/2)-\mu_{1},$ (17)
$\displaystyle\xi_{yz}(k_{x},k_{y},k_{z})$
$\displaystyle=\xi_{xz}(k_{y},k_{x},k_{z}),$ (18)
$\displaystyle\xi_{xy}(\boldsymbol{k})$
$\displaystyle=-t_{8}\left[\cos{k_{x}}+\cos{k_{y}}\right]-t_{9}\left[\cos(2k_{x})+\cos(2k_{y})\right]-t_{10}\cos{k_{x}}\cos{k_{y}}$
$\displaystyle\hskip 20.0pt-
t_{11}\left[\cos{k_{x}}\cos(2k_{y})+\cos(2k_{x})\cos{k_{y}}\right]-t_{12}\cos(2k_{x})\cos(2k_{y})$
$\displaystyle\hskip 20.0pt-
t_{13}\left[\cos{k_{x}}\cos(3k_{y})+\cos(3k_{x})\cos{k_{y}}\right]-t_{14}\left[\cos(3k_{x})+\cos(3k_{y})\right]$
$\displaystyle\hskip 20.0pt-
t_{15}\cos(k_{x}/2)\cos(k_{y}/2)\cos(k_{z}/2)-\mu_{2},$ (19)
$\displaystyle\xi_{xz,yz}(\boldsymbol{k})$
$\displaystyle=-t_{16}\sin(k_{x}/2)\sin(k_{y}/2)\cos(k_{z}/2).$ (20)

Tight-binding parameters providing a fit to both the Fermi surface _and_ the
Fermi velocity of the data in Ref. 35 are listed in Tab.
LABEL:tab:HoppingParameters1 and LABEL:tab:HoppingParameters2. These
parameters are largely taken from the relativistic DFT calculation of Ref. 23.
In particular, the $k_{z}$ dependent terms, responsible for the out-of-plane
warping, are identical. The calculations presented in the main text were done
with the effective 2D model obtained by fixing $k_{z}=0$.

Table 1: Tight-binding parameters for Eqs. (5), (17), (18) consistent with high-resolution ARPES measurements [35]. Parameter |  $t_{1}$ |  $t_{2}$ |  $t_{3}$ |  $t_{4}$ |  $t_{5}$ |  $t_{6}$ |  $t_{7}$ |  $\mu_{1}$ |  $\eta$
---|---|---|---|---|---|---|---|---|---
Value [meV] |  $562.7$ |  $99.9$ |  $-47.3$ |  $-174.3$ |  $-51.8$ |  $-11.0$ |  $102.0$ |  $209.9$ |  $81.0$
Table 2: Tight-binding parameters for Eqs. (19) and (20) consistent with high-resolution ARPES measurements [35]. Parameter |  $t_{8}$ |  $t_{9}$ |  $t_{10}$ |  $t_{11}$ |  $t_{12}$ |  $t_{13}$ |  $t_{14}$ |  $t_{15}$ |  $\mu_{2}$ |  $t_{16}$
---|---|---|---|---|---|---|---|---|---|---
Value [meV] |  $458.2$ |  $-7.5$ |  $330.0$ |  $25.3$ |  $32.8$ |  $8.8$ |  $3.5$ |  $-12.3$ |  $284.2$ |  $72.4$

## References

  * Maeno  _et al._ [2024] Y. Maeno, S. Yonezawa, and A. Ramires, Still mystery after all these years – Unconventional superconductivity of Sr2RuO4 –, arXiv e-prints 10.48550/arXiv.2402.12117 (2024).
  * Pustogow  _et al._ [2019] A. Pustogow, Y. Luo, A. Chronister, Y.-S. Su, D. A. Sokolov, F. Jerzembeck, A. P. Mackenzie, C. W. Hicks, N. Kikugawa, S. Raghu, and S. E. Bauer, E. D. Brown, Constraints on the superconducting order parameter in Sr2RuO4 from oxygen-17 nuclear magnetic resonance, Nature (London) 574, 72 (2019).
  * Mackenzie  _et al._ [2017] A. P. Mackenzie, T. Scaffidi, C. W. Hicks, and Y. Maeno, Even odder after twenty-three years: the superconducting order parameter puzzle of Sr2RuO4, npj Quantum Mater. 2, 40 (2017).
  * Scaffidi  _et al._ [2014] T. Scaffidi, J. C. Romers, and S. H. Simon, Pairing symmetry and dominant band in ${\mathrm{Sr}}_{2}$${\mathrm{RuO}}_{4}$, Phys. Rev. B 89, 220510(R) (2014).
  * Zhang  _et al._ [2018] L.-D. Zhang, W. Huang, F. Yang, and H. Yao, Superconducting pairing in ${\mathrm{Sr}}_{2}{\mathrm{RuO}}_{4}$ from weak to intermediate coupling, Phys. Rev. B 97, 060510 (2018).
  * Røising  _et al._ [2019] H. S. Røising, T. Scaffidi, F. Flicker, G. F. Lange, and S. H. Simon, Superconducting order of ${\mathrm{Sr}}_{2}{\mathrm{RuO}}_{4}$ from a three-dimensional microscopic model, Phys. Rev. Res. 1, 033108 (2019).
  * Ramires and Sigrist [2019] A. Ramires and M. Sigrist, Superconducting order parameter of ${\mathrm{Sr}}_{2}{\mathrm{RuO}}_{4}$: A microscopic perspective, Phys. Rev. B 100, 104501 (2019).
  * Lindquist and Kee [2020] A. W. Lindquist and H.-Y. Kee, Distinct reduction of Knight shift in superconducting state of ${\mathrm{Sr}}_{2}{\mathrm{RuO}}_{4}$ under uniaxial strain, Phys. Rev. Res. 2, 032055 (2020).
  * Wang  _et al._ [2019] W.-S. Wang, C.-C. Zhang, F.-C. Zhang, and Q.-H. Wang, Theory of Chiral $p$-Wave Superconductivity with Near Nodes for ${\mathrm{Sr}}_{2}{\mathrm{RuO}}_{4}$, Phys. Rev. Lett. 122, 027002 (2019).
  * Gingras  _et al._ [2019] O. Gingras, R. Nourafkan, A.-M. S. Tremblay, and M. Côté, Superconducting Symmetries of ${\mathrm{Sr}}_{2}{\mathrm{RuO}}_{4}$ from First-Principles Electronic Structure, Phys. Rev. Lett. 123, 217005 (2019).
  * Rømer  _et al._ [2019] A. T. Rømer, D. D. Scherer, I. M. Eremin, P. J. Hirschfeld, and B. M. Andersen, Knight Shift and Leading Superconducting Instability from Spin Fluctuations in ${\mathrm{Sr}}_{2}{\mathrm{RuO}}_{4}$, Phys. Rev. Lett. 123, 247001 (2019).
  * Suh  _et al._ [2020] H. G. Suh, H. Menke, P. M. R. Brydon, C. Timm, A. Ramires, and D. F. Agterberg, Stabilizing even-parity chiral superconductivity in ${\mathrm{Sr}}_{2}{\mathrm{RuO}}_{4}$, Phys. Rev. Res. 2, 032023(R) (2020).
  * Kivelson  _et al._ [2020] S. A. Kivelson, A. C. Yuan, B. Ramshaw, and R. Thomale, A proposal for reconciling diverse experiments on the superconducting state in Sr2RuO4, npj Quantum Materials 5, 43 (2020).
  * Rømer  _et al._ [2020a] A. T. Rømer, A. Kreisel, M. A. Müller, P. J. Hirschfeld, I. M. Eremin, and B. M. Andersen, Theory of strain-induced magnetic order and splitting of ${T}_{c}$ and ${T}_{\mathrm{TRSB}}$ in ${\mathrm{Sr}}_{2}\mathrm{Ru}{\mathrm{O}}_{4}$, Phys. Rev. B 102, 054506 (2020a).
  * Rømer  _et al._ [2021] A. T. Rømer, P. J. Hirschfeld, and B. M. Andersen, Superconducting state of ${\mathrm{Sr}}_{2}{\mathrm{RuO}}_{4}$ in the presence of longer-range Coulomb interactions, Phys. Rev. B 104, 064507 (2021).
  * Clepkens  _et al._ [2021] J. Clepkens, A. W. Lindquist, X. Liu, and H.-Y. Kee, Higher angular momentum pairings in interorbital shadowed-triplet superconductors: Application to Sr2RuO4, Phys. Rev. B 104, 104512 (2021).
  * Wang  _et al._ [2022] X. Wang, Z. Wang, and C. Kallin, Higher angular momentum pairing states in ${\mathrm{Sr}}_{2}{\mathrm{RuO}}_{4}$ in the presence of longer-range interactions, Phys. Rev. B 106, 134512 (2022).
  * Rømer  _et al._ [2022] A. T. Rømer, T. A. Maier, A. Kreisel, P. J. Hirschfeld, and B. M. Andersen, Leading superconducting instabilities in three-dimensional models for ${\mathrm{Sr}}_{2}{\mathrm{RuO}}_{4}$, Phys. Rev. Res. 4, 033011 (2022).
  * Roig  _et al._ [2022] M. Roig, A. T. Rømer, A. Kreisel, P. J. Hirschfeld, and B. M. Andersen, Superconductivity in multiorbital systems with repulsive interactions: Hund’s pairing versus spin-fluctuation pairing, Phys. Rev. B 106, L100501 (2022).
  * Røising  _et al._ [2022] H. S. Røising, G. Wagner, M. Roig, A. T. Rømer, and B. M. Andersen, Heat capacity double transitions in time-reversal symmetry broken superconductors, Phys. Rev. B 106, 174518 (2022).
  * Käser  _et al._ [2022] S. Käser, H. U. R. Strand, N. Wentzell, A. Georges, O. Parcollet, and P. Hansmann, Interorbital singlet pairing in ${\mathrm{Sr}}_{2}{\mathrm{RuO}}_{4}$: A Hund’s superconductor, Phys. Rev. B 105, 155101 (2022).
  * Gingras  _et al._ [2022] O. Gingras, N. Allaglo, R. Nourafkan, M. Côté, and A.-M. S. Tremblay, Superconductivity in correlated multiorbital systems with spin-orbit coupling: Coexistence of even- and odd-frequency pairing, and the case of ${\mathrm{Sr}}_{2}{\mathrm{RuO}}_{4}$, Phys. Rev. B 106, 064513 (2022).
  * Jerzembeck  _et al._ [2022] F. Jerzembeck, H. S. Røising, A. Steppke, H. Rosner, D. A. Sokolov, N. Kikugawa, T. Scaffidi, S. H. Simon, A. P. Mackenzie, and C. W. Hicks, The superconductivity of Sr2RuO4 under $c$-axis uniaxial stress, Nat. Commun. 13, 4596 (2022).
  * Prozorov and Giannetta [2006] R. Prozorov and R. W. Giannetta, Magnetic penetration depth in unconventional superconductors, Superconductor Science and Technology 19, R41 (2006).
  * Prozorov and Kogan [2011] R. Prozorov and V. G. Kogan, London penetration depth in iron-based superconductors, Reports on Progress in Physics 74, 124505 (2011).
  * Carrington [2011] A. Carrington, Studies of the gap structure of iron-based superconductors using magnetic penetration depth, Comptes Rendus. Physique 12, 502 (2011).
  * Hirschfeld [2016] P. J. Hirschfeld, Using gap symmetry and structure to reveal the pairing mechanism in Fe-based superconductors, Comptes Rendus. Physique 17, 197 (2016).
  * Bonalde  _et al._ [2000] I. Bonalde, B. D. Yanoff, M. B. Salamon, D. J. Van Harlingen, E. M. E. Chia, Z. Q. Mao, and Y. Maeno, Temperature Dependence of the Penetration Depth in ${\mathrm{Sr}}_{2}{\mathrm{RuO}}_{4}$: Evidence for Nodes in the Gap Function, Phys. Rev. Lett. 85, 4775 (2000).
  * Mueller  _et al._ [2023] E. Mueller, Y. Iguchi, F. Jerzembeck, J. O. Rodriguez, M. Romanelli, E. Abarca-Morales, A. Markou, N. Kikugawa, D. A. Sokolov, G. Oh, C. W. Hicks, A. P. Mackenzie, Y. Maeno, V. Madhavan, and K. A. Moler, Superconducting Penetration Depth Through a Van Hove Singularity: Sr2RuO4 Under Uniaxial Stress, arXiv e-prints , arXiv:2312.05130 (2023).
  * Landaeta  _et al._ [2023] J. F. Landaeta, K. Semeniuk, J. Aretz, K. Shirer, D. A. Sokolov, N. Kikugawa, Y. Maeno, I. Bonalde, J. Schmalian, A. P. Mackenzie, and E. Hassinger, Evidence for vertical line nodes in Sr2RuO4 from nonlocal electrodynamics, arXiv e-prints , arXiv:2312.05129 (2023).
  * Kosztin and Leggett [1997] I. Kosztin and A. J. Leggett, Nonlocal Effects on the Magnetic Penetration Depth in d-Wave Superconductors, Phys. Rev. Lett. 79, 135 (1997).
  * Scalapino [1995] D. Scalapino, The case for $d_{x^{2}-y^{2}}$ pairing in the cuprate superconductors, Physics Reports 250, 329 (1995).
  * Ferguson  _et al._ [2024] G. M. Ferguson, H. P. Nair, N. J. Schreiber, L. Miao, K. M. Shen, D. G. Schlom, and K. C. Nowack, Local magnetic response of superconducting Sr2RuO4 thin films and rings, arXiv e-prints , arXiv:2403.17152 (2024).
  * Hirschfeld and Goldenfeld [1993] P. J. Hirschfeld and N. Goldenfeld, Effect of strong scattering on the low-temperature penetration depth of a d-wave superconductor, Phys. Rev. B 48, 4219 (1993).
  * Tamai  _et al._ [2019] A. Tamai, M. Zingl, E. Rozbicki, E. Cappelli, S. Riccò, A. de la Torre, S. McKeown Walker, F. Y. Bruno, P. D. C. King, W. Meevasana, M. Shi, M. Radović, N. C. Plumb, A. S. Gibbs, A. P. Mackenzie, C. Berthod, H. U. R. Strand, M. Kim, A. Georges, and F. Baumberger, High-Resolution Photoemission on ${\mathrm{Sr}}_{2}{\mathrm{RuO}}_{4}$ Reveals Correlation-Enhanced Effective Spin-Orbit Coupling and Dominantly Local Self-Energies, Phys. Rev. X 9, 021048 (2019).
  * Sharma  _et al._ [2020] R. Sharma, S. D. Edkins, Z. Wang, A. Kostin, C. Sow, Y. Maeno, A. P. Mackenzie, J. C. S. Davis, and V. Madhavan, Momentum-resolved superconducting energy gaps of Sr2RuO4 from quasiparticle interference imaging, Proc. Natl. Acad. Sci. USA 117, 5222 (2020).
  * Huddart  _et al._ [2021] B. M. Huddart, I. J. Onuorah, M. M. Isah, P. Bonfà, S. J. Blundell, S. J. Clark, R. De Renzi, and T. Lancaster, Intrinsic Nature of Spontaneous Magnetic Fields in Superconductors with Time-Reversal Symmetry Breaking, Phys. Rev. Lett. 127, 237002 (2021).
  * Abrikosov  _et al._ [2012] A. Abrikosov, L. Gorkov, I. Dzyaloshinski, and R. Silverman,  _Methods of Quantum Field Theory in Statistical Physics_, Dover Books on Physics (Dover Publications, 2012).
  * Note [1] If one instead approaches the problem of evaluating $\delta\tilde{K}$ after recasting the Matsubara sum as an integral, care has to be taken with respect to the integrable singularities that emerge at the endpoints of the $\omega$ integration, cf. Eq. (15).
  * Li  _et al._ [2000] M.-R. Li, P. J. Hirschfeld, and P. Wölfle, Free energy and magnetic penetration depth of a d-wave superconductor in the Meissner state, Phys. Rev. B 61, 648 (2000).
  * Li  _et al._ [2022] Y.-S. Li, M. Garst, J. Schmalian, S. Ghosh, N. Kikugawa, D. A. Sokolov, C. W. Hicks, F. Jerzembeck, M. S. Ikeda, Z. Hu, B. J. Ramshaw, A. W. Rost, M. Nicklas, and A. P. Mackenzie, Elastocaloric determination of the phase diagram of Sr2RuO4, Nature 607, 276 (2022).
  * Deguchi  _et al._ [2004] K. Deguchi, Z. Q. Mao, and Y. Maeno, Determination of the Superconducting Gap Structure in All Bands of the Spin-Triplet Superconductor Sr2RuO4, J. Phys. Soc. Jpn. 73, 1313 (2004).
  * Hassinger  _et al._ [2017] E. Hassinger, P. Bourgeois-Hope, H. Taniguchi, S. René de Cotret, G. Grissonnanche, M. S. Anwar, Y. Maeno, N. Doiron-Leyraud, and L. Taillefer, Vertical Line Nodes in the Superconducting Gap Structure of ${\mathrm{Sr}}_{2}{\mathrm{RuO}}_{4}$, Phys. Rev. X 7, 011032 (2017).
  * Mackenzie and Maeno [2003] A. P. Mackenzie and Y. Maeno, The superconductivity of ${\mathrm{Sr}}_{2}{\mathrm{RuO}}_{4}$ and the physics of spin-triplet pairing, Rev. Mod. Phys. 75, 657 (2003).
  * Wilcox  _et al._ [2022] J. A. Wilcox, M. J. Grant, L. Malone, C. Putzke, D. Kaczorowski, T. Wolf, F. Hardy, C. Meingast, J. G. Analytis, J.-H. Chu, I. R. Fisher, and A. Carrington, Observation of the non-linear Meissner effect , Nat. Commun. 13, 1201 (2022).
  * Chia  _et al._ [2003] E. E. M. Chia, D. J. Van Harlingen, M. B. Salamon, B. D. Yanoff, I. Bonalde, and J. L. Sarrao, Nonlocality and strong coupling in the heavy fermion superconductor ${\mathrm{CeCoIn}}_{5}:$ A penetration depth study, Phys. Rev. B 67, 014527 (2003).
  * Veenstra  _et al._ [2014] C. N. Veenstra, Z.-H. Zhu, M. Raichle, B. M. Ludbrook, A. Nicolaou, B. Slomski, G. Landolt, S. Kittaka, Y. Maeno, J. H. Dil, I. S. Elfimov, M. W. Haverkort, and A. Damascelli, Spin-Orbital Entanglement and the Breakdown of Singlets and Triplets in ${\mathrm{Sr}}_{2}{\mathrm{RuO}}_{4}$ Revealed by Spin- and Angle-Resolved Photoemission Spectroscopy, Phys. Rev. Lett. 112, 127002 (2014).
  * Zabolotnyy  _et al._ [2013] V. Zabolotnyy, D. Evtushinsky, A. Kordyuk, T. Kim, E. Carleschi, B. Doyle, R. Fittipaldi, M. Cuoco, A. Vecchione, and S. Borisenko, Renormalized band structure of Sr2RuO4: A quasiparticle tight-binding approach, J. Electron. Spectrosc. Relat. Phenom. 191, 48 (2013).
  * Rømer  _et al._ [2020b] A. T. Rømer, T. A. Maier, A. Kreisel, I. Eremin, P. J. Hirschfeld, and B. M. Andersen, Pairing in the two-dimensional Hubbard model from weak to strong coupling, Phys. Rev. Res. 2, 013108 (2020b).
  * Sheng  _et al._ [2022] Y. Sheng, Y. Li, and Y.-f. Yang, Multipole-fluctuation pairing mechanism of ${d}_{{x}^{2}\text{$-$}{y}^{2}}+ig$ superconductivity in ${\mathrm{Sr}}_{2}{\mathrm{RuO}}_{4}$, Phys. Rev. B 106, 054516 (2022).