# Inference with Reference:
Lossless Acceleration of Large Language Models

Nan Yang, Tao Ge, Liang Wang, Binxing Jiao
Daxin Jiang, Linjun Yang, Rangan Majumder, Furu Wei
Microsoft
{nanya, tage, wangliang, binxjia,
djiang, linjya, ranganm<EMAIL_ADDRESS>
https://github.com/microsoft/unilm

###### Abstract

We propose LLMA, an LLM accelerator to losslessly speed up Large Language
Model (LLM) inference with references. LLMA is motivated by the observation
that there are abundant identical text spans between the decoding result by an
LLM and the reference that is available in many real world scenarios (e.g.,
retrieved documents). LLMA first selects a text span from the reference and
copies its tokens to the decoder and then efficiently checks the tokens’
appropriateness as the decoding result in parallel within one decoding step.
The improved computational parallelism allows LLMA to achieve over $2\times$
speed-up for LLMs with identical generation results as greedy decoding in many
practical generation scenarios where significant overlap between in-context
reference and outputs exists (e.g., search engines and multi-turn
conversations).

(a) Retrieval-augmented.

(b) Cache-assisted.

(c) Multi-turn conversations.

Figure 1: Significant overlaps between inputs and references exist in many LLM
applications such as retrieval-augmented generation, cache-assisted generation
and multi-turn conversations. By exploiting such overlaps, our LLMA method can
accelerate the inference of LLMs up-to 2$\sim$3 times without additional
models.

##  1 Introduction

With large foundation models (e.g., GPT-3.5/GPT-4) (OpenAI, 2023) becoming
widely used for various real-world applications, the concern of high
deployment cost has been increasingly raised. While there are general
methodologies that help reduce the serving cost of LLMs such as quantization
(Dettmers & Zettlemoyer, 2023), pruning (Frantar & Alistarh, 2023),
compression (Xu et al., 2020) and distillation (Wang et al., 2020), the
inference efficiency bottleneck of these transformer-based generative models
(e.g., GPT) is mainly associated with autoregressive decoding: at test time,
output tokens must be decoded (sequentially) one by one, which poses
significant challenges for the LLMs to be deployed at scale.

In this work, we study accelerating LLM’s inference by improving the
efficiency of autoregressive decoding. In many real world applications, we
observe that an LLM’s output tokens often come from its context. For example,
in a typical retrieval-augmented generation scenario for a search engine, an
LLM’s context usually includes relevant documents that are retrieved from an
external corpus as reference according to a query, and its output usually
contains many text spans found in the reference (i.e., retrieved documents),
as shown in Figure 1.

Motivated by the above observation, we propose LLMA, an inference-with-
reference decoding mechanism to accelerate LLM inference by exploiting the
overlap between an LLM’s output and reference that is available for many
practical scenarios. LLMA first selects a text span from the reference and
copies its tokens to the LLM decoder and then checks if they are acceptable
based on the output token probabilities, which can be conducted efficiently in
parallel. In this way, we can accelerate decoding by enabling better
parallelism on vector accelerators such as GPUs while ensuring the generation
results are identical to the vanilla greedy decoding method.

Compared to previous efficient decoding algorithms such as Speculative
Decoding111It was named Generalized Aggressive Decoding in the early version
of the manuscript (Xia et al., 2022b). (Xia et al., 2022a) and Speculative
Sampling (Chen et al., 2023) that need to introduce an additional efficient
drafter model to generate a draft for checking, LLMA does not require an
additional model and is easy to implement and deploy, which is an extension of
our previous work – (Input-guided) Aggressive Decoding (Sun et al., 2021; Ge
et al., 2022) that demonstrates a success in the rewriting tasks (e.g.,
Grammatical Error Correction) where inputs and outputs are similar.

Experiments show that our LLMA method can generate identical results as greedy
decoding but achieve over $2\times$ speed-up across different model sizes in
practical application scenarios like retrieval-augmented and cache-assisted
generation.

Figure 2: Illustration of LLMA decoding algorithm. At step 3, copy mechanism
is triggered as “from the” is matched against some reference document (see
Figure 1a for details). The text span “pancreases cows or pigs . Pork” is
copied from the reference document into the input of the LM decoder. The
copied tokens are then efficiently checked by running the LM to compute their
output tokens in one decoding step. “pancreases cows or pigs .” are identical
to their previous decoding output tokens and accepted as valid inputs, while
the last input token “Pork” does not match the previous output token “Until”
and is thus invalid and discarded. Overall, at step 3, LLMA generates six new
output tokens “pancreases cows or pigs . Until” compared to one output token
per step for the baseline decoding algorithm.

##  2 Method

###  2.1 Background: Stepwise Decoding in Autoregresive Language Models

Autoregressive language models typically follow a step-by-step decoding
algorithm. Let $\bm{x}$ be the user given prompt sequence, $\bm{y}$ be the LM
output sequence. At each decoding step $i$, the model takes the concatenation
of $\bm{x}$ and previous generated tokens $\bm{y}_{<i}$ as input and computes
the probability of the next token over the LM vocabulary $V$:

$y_{i}\sim P(y|\bm{x},\bm{y}_{<i})$ (1)

For greedy-decoding, we always select the token with the highest probability
as the output:

$y_{i}=\arg\max_{y\in V}P(y|\bm{x},\bm{y}_{<i})$ (2)

$y_{i}$ is then appended to $\bm{y}_{<i}$ and the process is repeated until
the full sequence is generated. Each decoding step depends on the results of
previous steps and the dependency slows down the decoding process as it
hampers the full utilization of parallel computing on vector accelerators such
as GPUs.

###  2.2 LLMA: Inference with Reference

Let $\bm{x}$ be the input prompt, $\bm{y}$ be the generation output and
$\bm{D}$ be a list of reference documents. We assume the generation result
$\bm{y}$ and the reference documents $\bm{D}$ have many identical text spans.
We exploit this redundancy to accelerate the decoding of language models by
copying texts from $\bm{D}$ into the input of LM decoder. Similar to the idea
of suffix matching in our previous work (Sun et al., 2021), we check if the
previously generated $n$ tokens $\bm{y}_{i-n:i}$ match some text spans in
$\bm{D}$ at each decoding step $i$. If no match is found, we fall back to
stepwise decoding described in Equation 2. If multiple matching spans are
found, we select the span with the longest matching prefixes with
$\bm{y}_{<i}$. We randomly break the tie if multiple spans exist with the
longest matching prefixes. If span $(pos-n,pos)$ in reference document
$\bm{d}$ is matched, we assume the model output is likely to follow $\bm{d}$.
Thus we preemptively copy $k$ subsequent tokens $\bm{d}_{pos:pos+k}$ from
$\bm{d}$ to the language model input. Given $\bm{y}_{<i}$ and
$\bm{d}_{pos:pos+k}$, the model can produce $k+1$ output tokens in a single
decoding step:

$y_{i+j}=\arg\max_{y\in V}P(y|\bm{x},\bm{y}_{<i},\bm{d}_{pos:pos+j}),j=0,..,k$
(3)

For $y_{i+j}$ to be valid output, we must ensure all tokens in its input are
valid, which means copied tokens $\\{d_{pos},...d_{pos+j-1}\\}$ are identical
to previous model output $\\{y_{i},...,y_{i+j-1}\\}$. The invalid output
tokens are discarded and the valid output tokens are appended to the output
sequence $\bm{y}$. The verification guarantees our decoding algorithm produces
identical generation output as the stepwise decoding method for greedy
decoding. Furthermore, LLMA can generate between $1$ and $k+1$ output tokens
per decoding step, compared to only one token per step for the stepwise
decoding method. See Algorithm 1 for the pseudo code and Figure 2 for the
illustration of our method.

Algorithm 1 LLMA Decoding.

Input: $\bm{x}$, $\bm{D}=(\bm{d}_{1},\dots,\bm{d}_{n})$, $n$, $k$, $N$;
Output: $\bm{y}$;

1:$\bm{y}\leftarrow[]$

2:while $\textsc{len}(\bm{y})<N$ do

3: $matched,\bm{d},pos\leftarrow\textsc{match\\_ngrams}(\bm{y},\bm{D},n)$

4: if $\neg matched$ then

5: $o=\textsc{LLM}(\bm{x},\bm{y})$

6: $\textsc{append}(\bm{y},o)$

7: continue

8: end if

9:
$(o_{0},o_{1},\dots,o_{k})\leftarrow\textsc{LLM}(\bm{x},\bm{y},d_{pos},...,d_{pos+k-1})$

10: $\textsc{append}(\bm{y},o_{0})$

11: for $i\textbf{ in }0,\dots,k-1$ do

12: if $o_{i}\textbf{ neq }d_{pos+i}$ then

13: break

14: end if

15: $\textsc{append}(\bm{y},o_{i+1})$

16: end for

17:end while

Overall, our decoding algorithm has two hyper-parameters: the match length $n$
and the copy length $k$, which control how aggressively we trigger and apply
the copy mechanism.

###  2.3 Application Scenarios

Our decoding algorithm can be beneficially applied to any scenarios where the
generation outputs have significant overlaps with reference documents. In this
work, we discuss three important practical scenarios that LLMA can benefit:

Retrieval-Augmented Generation. We let $\bm{q}$ be the user given query
tokens. In retrieval-augmented generation, a list of reference documents
$\bm{D}$ are retrieved from an external corpus based on their relevance to the
query $\bm{q}$. The documents $\bm{D}$ and the user query $\bm{q}$ are then
inserted into some pre-defined template to create the final input $\bm{x}$ for
the language model:

$\bm{x}=\texttt{Template}~{}(\bm{q},\bm{D})$ (4)

From $\bm{x}$, the language model generates output sequence $\bm{y}$, which
usually contains significant overlaps with the reference documents $\bm{D}$.
See Figure 1(a) for an example.

LLM Generation with Cached Sessions. When serving LLMs, previously generated
sessions can be stored in a cache to speed up future generations. The cache is
a collection of past query-output pairs $(\bm{q},\bm{d})$. Given a new query
$\bm{q}$, similar queries can be retrieved from the cache and their
corresponding outputs can be used as references $\bm{D}$ in our LLMA method to
accelerate decoding. Figure 1(b) gives an example of cached-assisted
generation.

Multi-turn Conversation with LLMs. New interaction patterns with LLMs emerge
with the powerful LLMs such as GPT-4. One important pattern is that LLMs are
repeatedly asked to refine their own outputs either by the users or the LLM
themselves (Gao et al., 2022). Under such scenarios, the generation outputs
between different turns often have significant overlaps, which can be
exploited by treating previous turns as reference documents. See Figure 1(c)
for an example.

##  3 Experiment

###  3.1 Dataset

Our dataset consists of triples of input prompt $\bm{x}$, target generation
result $\bm{y}$ and reference documents $\bm{D}$. Obtaining realistic
generation outputs is crucial for meaningful evaluation of our method as the
efficiency of our method depends on the overlaps between reference documents
and the generation output. Weak LLMs tend to produce either short, irrelevant
output with little overlaps with references or repeat the reference documents
with no modification, either underestimating or overestimating the efficiency
of the proposed method. While any strong LLMs would suffice, in this work, we
opt to use the davinci-003 variant of GPT-3.5 model through OpenAI APIs
222https://platform.openai.com/docs/models/gpt-3-5 to produce high-quality
generation results.

Retrieval-Augmented Generation (RAG). We start by sampling queries from the
MS-MARCO passage retrieval dataset (Bajaj et al., 2018). For each query
$\bm{q}$, we use a dual-encoder retrieval model E5 (Wang et al., 2022) to
retrieve a list of 10 passages $\\{\bm{d}_{i}\\}_{i=1}^{10}$ from the MS-MARCO
corpus. Davinci-003 is prompted to generate a response $\bm{y}$ for the query
according to the retrieved 10 passages. We then combine $\bm{q}$ and
$\\{\bm{d}_{i}\\}_{i=1}^{10}$ to get the prompt $\bm{x}$ for our retrieval-
augmented generation. See Appendix A for the detailed prompt templates.

Cache-Assisted Generation (CAG). We reuse queries from MS-MARCO passage
retrieval dataset. For each query $\bm{q}$, we use davinci-003 to generate 4
similar queries to simulate the cached queries. For both the original and the
similar queries, davinci-003 is prompted to respond without reference to the
retrieved documents. We treat the original query $\bm{q}$ as the input prompt
$\bm{x}$, the response to the original queries as generation result $\bm{y}$,
and the responses to the similar queries as the reference documents $\bm{D}$.

For both scenarios, we produce 200 triples of $(\bm{x},\bm{y},\bm{D})$. 100
triples are used as dev set for tuning hyper-parameters (match length $n$ and
copy length $k$) and conducting ablation studies, and the other 100 triples
are used for the final test evaluation. Table 1 shows the input and output
lengths for samples in all datasets. The input prompts are significant longer
in the RAG settings because the retrieved documents are inserted into the
prompts for RAG while the cached sessions are not in the inputs for cached-
assisted generation.

#tokens | Retrieval | Cache
---|---|---
dev | test | dev | test
Input | 903.6 | 898.8 | 15.6 | 17.1
Output | 111.2 | 122.0 | 162.5 | 177.3
Table 1: Numbers of input and output tokens per sample.

###  3.2 Target Guided Simulation

We test the proposed method using open sourced LLaMA (Touvron et al., 2023)
language models. Unfortunately, the outputs of LLaMA do not follow the
generation results from davinci-003 model. Fortunately, for greedy-decoding,
the decoding process of our method can be fully inferred from the davinci-003
output $\bm{y}$ and the reference documents $\bm{D}$. Given match length $n$
and copy length $k$, for every decoding step, we can determine the numbers of
input and output tokens using Algorithm 2. We can force LLaMA model to follow
the exact decoding steps regardless of its own output, which is sufficient for
measuring the execution time of our method.

Algorithm 2 Infer decoding sequence from target sequence and reference
documents.

Input: $\bm{y},\bm{D}=(\bm{d}_{1},\dots,\bm{d}_{n}),n,k$;
Output: $\bm{s}=(i_{1},o_{1}),\dots,(i_{m},o_{m})$;

1:$step\leftarrow 0$

2:$\bm{s}\leftarrow[]$

3:while $step<\textsc{len}(\bm{y})$ do

4: $matched,d,pos\leftarrow\textsc{match\\_ngrams}(\bm{y},step,D,n)$

5: if $\neg matched$ then

6: $step\leftarrow step+1$

7: $\textsc{append}(\bm{s},(1,1))$

8: continue

9: end if

10: $num\\_valid\leftarrow\textsc{get\\_matched\\_tokens}(d,pos,\bm{y},step)$

11: $num\\_valid\leftarrow\textsc{min}(k,num\\_valid)$

12: $num\\_output\\_tokens\leftarrow num\\_valid+1$

13: $step\leftarrow step+num\\_output\\_tokens$

14: $\textsc{append}(\bm{s},(1+k,num\\_output\\_tokens))$

15:end while

###  3.3 Implementation Details

We use the Huggingface Transformers library (Wolf et al., 2020) to implement
the inference for both the autoregressive decoding baseline and our LLMA
decoding method. We use the accelerate library (Gugger et al., 2022) to
implement larger models sharded to multiple GPUs. We perform tests on LLaMA
model of 7B, 13B and 30B parameters. All the inferences are done in half
floating numbers. For the 7B and 13B models, the inferences are done in one
NVidia 32G V100 GPU, and for the 30B model, the inference is performed on four
NVidia 32G V100 GPUs on a single machine. All inferences are done with greedy-
decoding, with batch size 1.

###  3.4 Main Results

We determine the match length $n$ and copy length $k$ by running grid search
on the dev set. Table 2 shows the optimal $n$ and $k$ values for different
scenarios and different model sizes. We then run our experiments on the test
set for three rounds and the averaged results are shown in Table 3 and Table
4. Our LLMA method achieves 2 to 3 times speed-up over baseline across
different model sizes and scenarios.

Model | Retrieval | Cache
---|---|---
$\bm{n}$ | $\bm{k}$ | $\bm{n}$ | $\bm{k}$
7B | 1 | 18 | 1 | 15
13B | 1 | 14 | 1 | 15
30B | 1 | 18 | 1 | 18
Table 2: Match length $n$ and copy length $k$ determined by grid search on dev set. Model |  Tokens/sec $\uparrow$ |  Time (sec) $\downarrow$ | Speed-up $\uparrow$
---|---|---|---
baseline | LLMA | baseline | LLMA
7B | 23.9 | 59.2 | 511.2 | 206.0 | 2.48x
13B | 18.5 | 41.1 | 658.4 | 296.8 | 2.22x
30B | 4.9 | 12.1 | 2503.2 | 1005.8 | 2.49x
Table 3: Time comparison for retrieval-augmented generation. The times are the total execution times in seconds of decoding 100 samples. All numbers are averaged over 3 runs. Model |  Tokens/sec $\uparrow$ |  Time (sec) $\downarrow$ | Speed-up $\uparrow$
---|---|---|---
baseline | LLMA | baseline | LLMA
7B | 24.3 | 53.8 | 730.8 | 329.8 | 2.22x
13B | 19.3 | 42.3 | 918.4 | 419.3 | 2.19x
30B | 5.1 | 15.6 | 3467.7 | 1133.0 | 3.06x
Table 4: Time comparison for generation with cached sessions. The times are
the total execution times in seconds of decoding 100 samples. All numbers are
averaged over 3 runs.

$4$$6$$8$$10$$12$$14$$16$$18$$1.8$$2$$2.2$$2.4$$2.6$$2.8$copy length $k$speed-
upn=1n=2n=3 (a) 7B RAG

$4$$6$$8$$10$$12$$14$$16$$18$$1.7$$1.8$$1.9$$2$$2.1$$2.2$$2.3$$2.4$$2.5$copy
length $k$speed-upn=1n=2n=3 (b) 13B RAG

$4$$5$$6$$7$$8$$9$$10$$11$$12$$13$$14$$15$$16$$17$$18$$1.6$$1.7$$1.8$$1.9$$2$$2.1$$2.2$$2.3$$2.4$copy
length $k$speed-upn=1n=2n=3 (c) 7B CAG

$4$$5$$6$$7$$8$$9$$10$$11$$12$$13$$14$$15$$16$$17$$18$$1.6$$1.7$$1.8$$1.9$$2$$2.1$$2.2$$2.3$$2.4$copy
length $k$speed-upn=1n=2n=3 (d) 13B CAG

Figure 3: Effect of match length ($n$) and copy length ($k$) on the dev set.

###  3.5 Effect of Match and Copy Length

We study the effect of match and copy lengths $n$ and $k$ using the dev set.
As can be seen in Figure 3, aggressive triggering ($n=1$) and longer copy
length gives larger speed-up across different settings, with the gains
plateaued when copy length $k$ grows past 15.

$4$$5$$6$$7$$8$$9$$10$$11$$12$$13$$14$$15$$16$$17$$18$$10$$15$$20$$25$$30$$35$$40$copy
length $k$# triggersn=1n=2n=3

$4$$5$$6$$7$$8$$9$$10$$11$$12$$13$$14$$15$$16$$17$$18$$50$$55$$60$$65$$70$$75$$80$copy
length $k$# accepted tokensn=1n=2n=3

$4$$5$$6$$7$$8$$9$$10$$11$$12$$13$$14$$15$$16$$17$$18$$30$$35$$40$$45$$50$$55$$60$copy
length k# decoding stepsn=1n=2n=3

$4$$5$$6$$7$$8$$9$$10$$11$$12$$13$$14$$15$$16$$17$$18$$20$$30$$40$$50$$60$copy
length $k$# triggersn=1n=2n=3

$4$$5$$6$$7$$8$$9$$10$$11$$12$$13$$14$$15$$16$$17$$18$$70$$80$$90$$100$copy
length $k$# accepted tokensn=1n=2n=3

$4$$5$$6$$7$$8$$9$$10$$11$$12$$13$$14$$15$$16$$17$$18$$70$$80$$90$copy length
$k$# decoding stepsn=1n=2n=3

Figure 4: Decoding statistics with varying match length $n$ and copy length
$k$ on the dev set. All statistics are shown on a per sample basis. The first
row shows retrieval-augmented scenario and the second row shows cache-assisted
scenario. The first column shows the triggering number of copy mechanism; the
second column shows the number of copied tokens accepted after verification;
the third column shows the decoding steps performed during LLMA decoding.

To further understand LLMA decoding behaviour, we collect the following
decoding statistics on the dev set: (1) the number of triggering for the copy
mechanism per sample; (2) the number of copied tokens accepted by the
verification per sample; (3) the decoding steps required per sample. As can be
seen in Figure 4, smaller match length $n$ allows more triggering of copying,
which leads to more accepted tokens and less decoding steps. Larger copying
length $k$ decreases the number of triggering because consecutive small
copying steps for small $k$ might be merged in one large copying step for
large $k$. Larger $k$ also leads to more accepted tokens and less decoding
steps. As noted in Figure 3, fewer decoding steps by larger $k$ give larger
speed-up until $k$ reaches 15 despite larger $k$ wastes more computations when
the copied tokens are not accepted. The advantages of fewer decoding steps are
1) better utilization of parallel computing of GPUs and 2) for larger models,
fewer data transfers and synchronizations between GPUs.

##  4 Conclusion

In this work, we propose LLMA, a new inference-with-reference decoding method
to accelerate LLM generations. LLMA exploits the overlaps between generation
contexts and outputs which naturally occur in several important LLM deployment
scenarios such as retrieval-augmented generation, cache-assist generation and
multi-turn conversations. LLMA is easy to deploy and requires no additional
models. Experiments demonstrate the effectiveness of our method, achieving
$2\times$ speed-up across different model sizes and application scenarios.

## References

  * Bajaj et al. (2018) Payal Bajaj, Daniel Campos, Nick Craswell, Li Deng, Jianfeng Gao, Xiaodong Liu, Rangan Majumder, Andrew McNamara, Bhaskar Mitra, Tri Nguyen, Mir Rosenberg, Xia Song, Alina Stoica, Saurabh Tiwary, and Tong Wang.  Ms marco: A human generated machine reading comprehension dataset, 2018\.
  * Chen et al. (2023) Charlie Chen, Sebastian Borgeaud, Geoffrey Irving, Jean-Baptiste Lespiau, Laurent Sifre, and John Jumper.  Accelerating large language model decoding with speculative sampling, 2023\.
  * Dettmers & Zettlemoyer (2023) Tim Dettmers and Luke Zettlemoyer.  The case for 4-bit precision: k-bit inference scaling laws, 2023.
  * Frantar & Alistarh (2023) Elias Frantar and Dan Alistarh.  Sparsegpt: Massive language models can be accurately pruned in one-shot, 2023.
  * Gao et al. (2022) Luyu Gao, Zhuyun Dai, Panupong Pasupat, Anthony Chen, Arun Tejasvi Chaganty, Yicheng Fan, Vincent Y. Zhao, Ni Lao, Hongrae Lee, Da-Cheng Juan, and Kelvin Guu.  Rarr: Researching and revising what language models say, using language models, 2022.
  * Ge et al. (2022) Tao Ge, Heming Xia, Xin Sun, Si-Qing Chen, and Furu Wei.  Lossless acceleration for seq2seq generation with aggressive decoding.  _arXiv preprint arXiv:2205.10350_ , 2022.
  * Gugger et al. (2022) Sylvain Gugger, Lysandre Debut, Thomas Wolf, Philipp Schmid, Zachary Mueller, and Sourab Mangrulkar.  Accelerate: Training and inference at scale made simple, efficient and adaptable.  https://github.com/huggingface/accelerate, 2022.
  * OpenAI (2023) OpenAI.  Gpt-4 technical report, 2023.
  * Sun et al. (2021) Xin Sun, Tao Ge, Furu Wei, and Houfeng Wang.  Instantaneous grammatical error correction with shallow aggressive decoding.  In _Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (Volume 1: Long Papers)_ , pp. 5937–5947, 2021.
  * Touvron et al. (2023) Hugo Touvron, Thibaut Lavril, Gautier Izacard, Xavier Martinet, Marie-Anne Lachaux, Timothée Lacroix, Baptiste Rozière, Naman Goyal, Eric Hambro, Faisal Azhar, Aurelien Rodriguez, Armand Joulin, Edouard Grave, and Guillaume Lample.  Llama: Open and efficient foundation language models, 2023.
  * Wang et al. (2022) Liang Wang, Nan Yang, Xiaolong Huang, Binxing Jiao, Linjun Yang, Daxin Jiang, Rangan Majumder, and Furu Wei.  Text embeddings by weakly-supervised contrastive pre-training, 2022.
  * Wang et al. (2020) Wenhui Wang, Furu Wei, Li Dong, Hangbo Bao, Nan Yang, and Ming Zhou.  Minilm: Deep self-attention distillation for task-agnostic compression of pre-trained transformers, 2020.
  * Wolf et al. (2020) Thomas Wolf, Lysandre Debut, Victor Sanh, Julien Chaumond, Clement Delangue, Anthony Moi, Pierric Cistac, Tim Rault, Rémi Louf, Morgan Funtowicz, Joe Davison, Sam Shleifer, Patrick von Platen, Clara Ma, Yacine Jernite, Julien Plu, Canwen Xu, Teven Le Scao, Sylvain Gugger, Mariama Drame, Quentin Lhoest, and Alexander M. Rush.  Transformers: State-of-the-art natural language processing.  In _Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing: System Demonstrations_ , pp. 38–45, Online, October 2020. Association for Computational Linguistics.  URL https://www.aclweb.org/anthology/2020.emnlp-demos.6.
  * Xia et al. (2022a) Heming Xia, Tao Ge, Si-Qing Chen, Furu Wei, and Zhifang Sui.  Speculative decoding: Lossless speedup of autoregressive translation.  _Openreview_ , 2022a.  URL https://openreview.net/forum?id=H-VlwsYvVi.
  * Xia et al. (2022b) Heming Xia, Tao Ge, Furu Wei, and Zhifang Sui.  Lossless speedup of autoregressive translation with generalized aggressive decoding, 2022b.
  * Xu et al. (2020) Canwen Xu, Wangchunshu Zhou, Tao Ge, Furu Wei, and Ming Zhou.  Bert-of-theseus: Compressing bert by progressive module replacing.  _arXiv preprint arXiv:2002.02925_ , 2020.

##  Appendix A Prompt Templates

The prompt template to get retrieval-augmented generation results from
davinci-003 model is in Figure 5.

> Respond to the queries according to the presented documents. The query and
> documents are given in a json string. Here are some guidelines for your
> response:
>
> 1\. The response should be informative, visual , logical and actionable.
>
> 2\. The response should be positive, interesting, entertaining and engaging.
>
> 3\. The response should avoid being vague, controversial or off-topic.
>
> 4\. The logics and reasoning in the response should be rigorous, intelligent
> and defensible.
>
> 5\. The response can provide additional relevant details to respond
> thoroughly and comprehensively to cover multiple aspects in depth.
>
> 6\. The presented documents may be incomplete or irrelevant to the query.
> The response shouldn’t make assumptions on presented documents beyond what’s
> presented.
>
> 7\. If the presented documents do not contain sufficient information to
> answer the query completely, the response should use only facts in the
> presented documents and should not add any information by itself.
>
> docs: [
>
> {doc 1}
>
> …
>
> {doc n}]
>
> query: {query}
>
> response:

Figure 5: Prompt template to get generate retrieval-augmented generation from
davinci-003.

A simplified prompt template used in our decoding experiments for our models
is in Figure 6.

> docs:
>
> {doc 1}
>
> …
>
> {doc n}
>
> query: {query}
>
> answer:

Figure 6: Prompt template used in decoding experiments for our models.


# Fast-forward approach to adiabatic quantum dynamics of regular spin
clusters:
nature of geometry-dependent driving interactions

Iwan Setiawan1,2, Bobby Eka Gunara2, Sanat Avazbaev3,4and Katsuhiro
Nakamura5,6 (1)Department of Physics Education, University of Bengkulu,
Kandang Limun, Bengkulu 38371, Indonesia

(2)Department of Physics, Institut Teknologi Bandung, Jalan Ganesha, Bandung
40132, Indonesia
(3)Faculty of Physics and Mathematics, Tashkent State Pedagogical University,
27 Bunyodkor Street, Tashkent 100070, Uzbekistan
(4)Yeoju Technical Institute in Tashkent, 156 Usmon Nosir Street, Tashkent
100056, Uzbekistan
(5)Faculty of Physics, National University of Uzbekistan, Vuzgorodok, Tashkent
100174, Uzbekistan
(6)Department of Applied Physics, Osaka City University, Sumiyoshi-ku, Osaka
558-8585, Japan

###### Abstract

The fast forward scheme of adiabatic quantum dynamics is applied to finite
regular spin clusters with various geometries and the nature of driving
interactions is elucidated. The fast forward is the quasi-adiabatic dynamics
guaranteed by regularization terms added to the reference Hamiltonian,
followed by a rescaling of time with use of a large scaling factor. With help
of the regularization terms consisting of pair-wise and 3-body interactions,
we apply the proposed formula (Phys. Rev. A $\bf{96}$, 052106(2017)) to
regular triangle and open linear chain for $N=3$ spin systems, and to
triangular pyramid, square, primary star graph and open linear chain for $N=4$
spin systems. The geometry-induced symmetry greatly decreases the rank of
coefficient matrix of the linear algebraic equation for regularization terms.
Choosing a transverse Ising Hamiltonian as a reference, we find: (1) for $N=3$
spin clusters, the driving interaction consists of only the geometry-dependent
pair-wise interactions and there is no need for the 3-body interaction; (2)
for $N=4$ spin clusters, the geometry-dependent pair-wise interactions again
constitute major part of the driving interaction, whereas the universal 3-body
interaction free from the geometry is necessary but plays a subsidiary role.
Our scheme predicts the practical driving interaction in accelerating the
adiabatic quantum dynamics of structured regular spin clusters.

###### pacs:

03.65.Ta, 32.80.Qk, 37.90.+j, 05.45.Yv

##  I INTRODUCTION

Effectively manipulating and optimizing the dynamics of given systems
constitutes one of big experimental and theoretical subjects in the current
technology. In particular, it is a challenging theme to find suitable driving
fields for tailoring a quantum system to rapidly generate a target state from
a given initial state. In designing quantum computers, the acceleration of
adiabatic quantum dynamics is desirable because the coherence of systems is
degraded by their interaction with the environment. Since naive numerical
trial-and-error methods are time- and resource-consuming, we must deeply
understand relevant quantum dynamics to find useful schemes for such
accelerations. In this context, various researches on the way to the shortcut
to adiabaticity (STA) have been developed, which include invariant-based
inverse engineering 1 ; 2 ; 3 , transitionless counter-diabatic (CD) driving 4
; 5 ; 6 , fast-forward approach 7 ; 8 ; 9 , and variational methods to
generate approximate CD protocols10 ; 11 ; 12 .

The fast-forward theory proposed by Masuda and Nakamura 7  was originally
concerned with acceleration of general reference quantum dynamics. This theory
was developed to accelerate the adiabatic quantum dynamics by introducing the
large time-scaling factor in the quasi-adiabatic dynamics guaranteed by
regularization terms added to the reference Hamiltonian 8 ; 9 , and was then
used to enhance the quantum tunneling power 13  and to construct the non-
equilibrium equation of state under a rapid piston 14 . The relation between
the fast-forward approach and other methods was rigorously investigated in 15
.

Recently, we proposed a fast forward scheme of adiabatic spin dynamics 16 .
Confining to a single and two spin systems there, we showed the acceleration
of Landau-Zener transition and that of a generation of entangled states, as
can be shown in other methods 4 ; 5 ; 6 ; 3extra ; 10 ; paul ; stef  .

The fast forward scheme of adiabatic quantum dynamics has advantages as
addressed by TK1 ; TK2 : (1) No need of writing the driving interaction in the
spectral representation with use of full spectral properties of given spin
systems. No necessity of worrying about the divergence of the driving
interaction due to the level crossing; (2) A great flexibility in choosing the
regularization Hamiltonian which leads to the driving interaction. Namely,
users can specify the regularization Hamiltonian by themselves so as to
satisfy the core equation (see Eq.(8) of this paper). The latter advantage
will play an important role when we shall investigate spin clusters of various
geometries. However, no technical guide was so far presented in solving the
core equation for unknown regularization terms.

Within a framework of the transitionless CD driving 4 ; 5 ; 6 , on the other
hand, there exist intensive works on a linear chain of many quantum spins
described by the Ising model in a transverse field ex1 ; ex2 ; ex3 ; ex33  and
the related model campb , which showed the complicated non-local multi-body CD
terms that are hard to achieve in experiment. While a variational method to
generate approximate local CD protocols11 ; 12  is being cultivated, it is
timely to sharpen the fast-forward approach by showing a guiding principle to
manage spin clusters with various geometries on the basis of the proposed
formula in 16 .

In this paper the fast forward scheme of adiabatic dynamics is applied to
regular spin clusters of various geometries with number of spins $N$ up to 4,
i.e., regular triangle and open linear chain for $N=3$ spins, and triangular
pyramid, square, primary star graph and open linear chain for $N=4$ spins.
(Note: the geometry is irrelevant for systems with $N=1$ and $2$ spins.)
Choosing the Hamiltonian for a transverse Ising model as a reference, we shall
reveal the nature of driving interactions. In Section II, a brief summary is
given on the fast forward scheme of adiabatic quantum spin dynamics. In
Section III we propose a candidate regularization Hamiltonian consisting of
geometry-dependent pair-wise interactions and a universal 3-body interaction,
and describe a method of solving the linear algebraic equation for
regularization terms. Sections IV and V are devoted to the analysis of spin
clusters of various geometries with $N=3$ and $N=4$, respectively. Summary and
discussions are given in Section VI. Appendix A gives matrices for some
regularization Hamiltonians.

##  II Fast-forward scheme of adiabatic spin dynamics

For self-containedness, we shall sketch the fast forward scheme of adiabatic
spin dynamics16 . Our strategy is as follows: (i) A given original (reference)
Hamiltonian $H_{0}$ is assumed to change adiabatically and to generate a
stationary state $\Psi_{0}$, which is an eigenstate of the time-independent
Schrödinger equation with the instantaneous Hamiltonian. Then $H_{0}$ is
regularized so that $\Psi_{0}$ should satisfy the time-dependent Schrödinger
equation (TDSE); (ii) Taking $\Psi_{0}$ as a reference state, we shall rescale
time in TDSE with use of the scaling factor $\alpha\left(t\right)$, where the
mean value $\bar{\alpha}$ of the infinitely-large time scaling factor
$\alpha(t)$ will be chosen to compensate the infinitesimally-small growth rate
$\epsilon$ of the quasi-adiabatic parameter and to satisfy
$\bar{\alpha}\times\epsilon=finite$.

Consider the Hamiltonian for spin systems to be characterized by a slowly
time-changing parameter $R(t)$ such as the exchange interaction, magnetic
field, etc. Then we can study the eigenvalue problem for the time-independent
Schrödinger equation :

$H_{0}(R)\textbf{C}^{(n)}(R)=E_{n}(R)\textbf{C}^{(n)}(R)$ (1)

with

$\textbf{C}^{(n)}(R)=\begin{pmatrix}C^{(n)}_{1}(R)\\\ \vdots\\\
C^{(n)}_{N}(R)\end{pmatrix},$ (2)

where

$R\equiv R(t)=R_{0}+\epsilon t$ (3)

is the adiabatically-changing parameter with $\epsilon\ll 1$. In Eq.(1), $n$
stands for the quantum number for each eigenvalue and eigenstate. Let us
assume

$\Psi^{(n)}_{0}(R(t))=\textbf{C}^{(n)}(R(t))e^{-\frac{i}{\hbar}\int_{0}^{t}E_{n}(R(t^{\prime}))dt^{\prime}}e^{i\xi_{n}(R(t))},$
(4)

to be a quasi-adiabatic state, i.e., adiabatically evolving state, where
$\xi_{n}$ is the adiabatic phase:

$\displaystyle\xi_{n}(R(t))=i\int_{0}^{t}dt^{\prime}\textbf{C}^{(n)\dagger}\partial_{t}\textbf{C}^{(n)}=i\epsilon\int_{0}^{t}dt^{\prime}\textbf{C}^{(n)\dagger}\partial_{R}\textbf{C}^{(n)}.$

$\Psi^{(n)}_{0}(R(t))$ in Eq.(4) is not a solution of TDSE. To make it to
satisfy the TDSE, we must regularize the Hamiltonian as

$H_{0}^{reg}(R(t))=H_{0}(R(t))+\epsilon\mathcal{\tilde{H}}_{n}(R(t)).$ (6)

Then TDSE becomes

$i\hbar\frac{\partial}{\partial
t}\Psi^{(n)}_{0}(R(t))=(H_{0}+\epsilon\mathcal{\tilde{H}}_{n})\Psi^{(n)}_{0}(R(t)).$
(7)

Here $\mathcal{\tilde{H}}_{n}$ is the $n$-th state-dependent regularization
term. Substituting $\Psi^{(n)}_{0}(R(t))$ in Eq.(4) into the above TDSE, we
see the eigenvalue problem in Eq.(1) in order of $O(\epsilon^{0})$, and the
algebraic equation for $\mathcal{\tilde{H}}_{n}$,

$\mathcal{\tilde{H}}_{n}\textbf{C}^{(n)}(R)=i\hbar\partial_{R}\textbf{C}^{(n)}(R)-i\hbar(\textbf{C}^{(n)\dagger}\partial_{R}\textbf{C}^{(n)})\textbf{C}^{(n)}(R),$
(8)

in order of $O(\epsilon^{1})$. Equation (8) is the core of the present study.
The state in Eq.(4) and TDSE in Eq.(7) are working on a very slow time scale.
We shall innovate them so that they can work on a laboratory time scale.

With time $t$ rescaled by the advanced time $\Lambda(t)$, the fast-forward
state is introduced as

$\displaystyle\Psi^{(n)}_{FF}(t)$ $\displaystyle\equiv$
$\displaystyle\Psi^{(n)}_{0}(R(\Lambda(t)))$ $\displaystyle=$
$\displaystyle\textbf{C}^{(n)}(R(\Lambda(t)))e^{-\frac{i}{\hbar}\int_{0}^{t}E_{n}(R(\Lambda(t^{\prime})))dt^{\prime}}e^{i\xi_{n}(R(\Lambda(t)))},$

where $\Lambda(t)$ is defined by

$\Lambda(t)=\int_{0}^{t}\alpha(t^{\prime})dt^{\prime},$ (10)

with the standard time $t$. $\alpha(t)$ is an arbitrary magnification time-
scale factor which satisfies $\alpha(0)$ = 1, $\alpha(t)>1(0<t<T_{FF})$ and
$\alpha(t)$ = $1(t\geq T_{FF})$. For a long final time $T$ in the original
adiabatic dynamics, we can consider the fast forward dynamics with a new time
variable which reproduces the target state $\Psi^{(n)}_{0}(R(T))$ in a shorter
final time $T_{FF}$ defined by

$T=\int_{0}^{T_{FF}}\alpha(t)dt.$ (11)

The simplest expression for $\alpha(t)$ in the fast-forward range ($0\leq
t\leq T_{FF}$) is given by 8  as :

$\alpha(t)=\bar{\alpha}-(\bar{\alpha}-1)\cos(\frac{2\pi}{T_{FF}}t),$ (12)

where $\bar{\alpha}$ is the mean value of $\alpha(t)$ and is given by
$\bar{\alpha}=T/T_{FF}$.

Then by taking the time derivative of $\Psi^{(n)}_{FF}$ in Eq.(II) and using
the equalities
$\partial_{t}\textbf{C}^{(n)}(R(\Lambda(t)))=\alpha\epsilon\partial_{R}\textbf{C}^{(n)}$
and $\partial_{t}\xi_{n}(R(\Lambda(t)))$ =
$i\textbf{C}^{(n)\dagger}\partial_{t}\textbf{C}^{(n)}$ =
$i\alpha\epsilon\textbf{C}^{(n)\dagger}\partial_{R}\textbf{C}^{(n)}$, we have

$\displaystyle i\hbar\dot{\Psi}^{(n)}_{FF}$ $\displaystyle=$
$\displaystyle\Big{[}i\hbar\alpha\epsilon\left(\partial_{R}\textbf{C}^{(n)}-(\textbf{C}^{(n)\dagger}\partial_{R}\textbf{C}^{(n)})\textbf{C}^{(n)}\right)+E\textbf{C}^{(n)}\Big{]}$
(13) $\displaystyle\times$ $\displaystyle
e^{-\frac{i}{\hbar}\int_{0}^{t}E_{n}(R(\Lambda(t^{\prime})))dt^{\prime}}e^{i\xi_{n}(R(\Lambda(t)))}.$

The first and second terms in the angular bracket on the r.h.s are replaced by
$\alpha\epsilon\mathcal{\tilde{H}}_{n}\textbf{C}^{(n)}(R(\Lambda(t)))$ and
$H_{0}\textbf{C}^{(n)}(R(\Lambda(t)))$, respectively, by using Eqs.(8) and
(1). Using the definition of $\Psi^{(n)}_{FF}(t)$ and taking the asymptotic
limit $\bar{\alpha}\rightarrow\infty$ and $\epsilon\rightarrow 0$ under the
constraint $\bar{\alpha}\cdot\epsilon\equiv\bar{v}=finite$, we obtain

$\displaystyle i\hbar\frac{\partial\Psi^{(n)}_{FF}}{\partial t}$
$\displaystyle=$
$\displaystyle\left(H_{0}(R(\Lambda(t)))+v(t)\mathcal{\tilde{H}}_{n}(R(\Lambda(t)))\right)\Psi^{(n)}_{FF}$
(14) $\displaystyle\equiv$ $\displaystyle H^{(n)}_{FF}\Psi^{(n)}_{FF}.$

Here $v(t)$ is a velocity function available from $\alpha(t)$ in the
asymptotic limit:

$\displaystyle v(t)=\lim_{\epsilon\to
0,\bar{\alpha}\to\infty}\epsilon\alpha(t)=\bar{v}\left(1-\cos\frac{2\pi}{T_{FF}}t\right).$
(15)

Consequently, for $0\leq t\leq T_{FF}$,

$\displaystyle R(\Lambda(t))$ $\displaystyle=$ $\displaystyle
R_{0}+\lim_{\epsilon\rightarrow
0,\bar{\alpha}\rightarrow\infty}\varepsilon\Lambda(t)=R_{0}+\int^{t}_{0}v(t^{\prime})dt^{\prime}$
(16) $\displaystyle=$ $\displaystyle
R_{0}+\bar{v}\left[t-\frac{T_{FF}}{2\pi}\sin\left(\frac{2\pi}{T_{FF}}t\right)\right].$

$H^{(n)}_{FF}$ is the fast-forward Hamiltonian and $\mathcal{\tilde{H}}_{n}$
is the regularization term obtained from Eq.(8) to generate the fast-forward
scheme in spin system. Eqs. (II) and (14) work on a laboratory time scale.

There is a relationship between our formula for $\mathcal{\tilde{H}}_{n}$ in
Eq.(8) and Demirplak-Rice-Berry (DRB)’s formula 4 ; 5 ; 6  for the CD term
$\mathcal{H}$. If there is a $n$-independent regularization term
$\mathcal{\tilde{H}}$ among $\\{\mathcal{\tilde{H}}_{n}\\}$, we can define
$\mathcal{H}\equiv v(t)\mathcal{\tilde{H}}$ with use of $v(t)=\frac{\partial
R(\Lambda(t))}{\partial t}$. Then Eq.(8) gives a solution $\mathcal{H}$ which
agrees with DRB’s formula for the CD term (See the proof in 16 ). It should be
noted, however, that the above correspondence works well only in the case that
we can find $n$-independent regulariztion terms $\mathcal{\tilde{H}}$ among
$\\{\mathcal{\tilde{H}}_{n}\\}$. Using the above notion, one may call
$v(t)\mathcal{\tilde{H}}_{n}$ as a state-dependent CD term. Hereafter we shall
be concerned with the fast forward of adiabatic dynamics of one of the
adiabatic states (i.e., the ground state) and therefore the suffix $n$ in
$\mathcal{\tilde{H}}_{n}$ will be suppressed.

(a)  (b)

Figure 1: (a) Regular triangle; (b) Open linear 3 spin chain. Solid lines
stand for the original exchange interactions. Dashed and dotted lines mean the
pair-wise regularization interactions. Each line species denotes the
geometrically-identical regularization interactions.

(a)  (b)  (c)  (d)

Figure 2: (a) Triangular pyramid; (b) Square; (c) Primary star graph; (d) Open
linear 4 spin chain. Solid lines stand for the original exchange interactions.
Dashed, dotted, dotted dashed and double-dotted dashed lines mean the pair-
wise regularization interactions. Each line species denotes the geometrically-
identical regularization interactions.

##  III Fast-forward driving interactions for spin clusters of various
geometries

To begin with, let us explain the method of solving the linear algebraic
equation for unknown regularization terms in Eq. (8). Then in the succeeding
Sections, we shall treat regular spin clusters of various geometries with $N$
up to 4, i.e., regular triangle and open linear chain for $N=3$ spins (see
Fig.1), and triangular pyramid, square, primary star graph and open linear
chain for $N=4$ spins (see Fig.2). Our scheme is free from obtaining all
eigenvectors for a given adiabatic Hamiltonian. As shown in the core equation
in Eq. (8), we need only information of a single eigenstate, typically of the
ground state.

As an original (reference) model, we choose the transverse Ising mode, whose
Hamiltonian for $N$ spin systems is written as

$H_{0}=J(R(t))\sum_{(i,j)\in
N.N.}\sigma_{i}^{z}\sigma_{j}^{z}-\frac{1}{2}B_{x}(R(t))\sum_{i=1}^{N}\sigma_{i}^{x},$
(17)

where $J(R(t))=R(t)=R_{0}+\epsilon t$ and $B_{x}(R(t))=B_{0}-R(t)$ with
$\epsilon\ll 1$ are adiabatically-changing exchange interaction and transverse
magnetic field, respectively. $(i,j)\in N.N.$ means nearest-neighbouring
pairs. Using the spin configuration bases, the dimension of Hilbert space is
$2^{N}$.

Energy matrix corresponding to the Hamiltonian in Eq.(17) is real symmetric,
which makes the eigenstates real, and the ground state is expressed by the
real components $\\{C_{k}:k=1,\cdots,2^{N}\\}$. This, in combination with the
fact that the length of the corresponding eigenvector is constant and equal to
1, leads to the conclusion that the adiabatic phase $\xi_{n}$ in Eq.(II) is
zero in all spin clusters in the present work. Further, because of the
geometrical symmetry of spin clusters in Figs. 1 and 2, some of the components
$C_{k}$s are degenerate which reduce the number of independent equations in
the core equation in Eq. (8).

As for the unknown regularization term ($\mathcal{\tilde{H}}$) in Eq.(8) , we
must impose a form which makes its matrix elements pure imaginary because the
right-hand side of Eq.(8) is now pure imaginary. Among several possibilities,
we assume the regularization term consisting of pair-wise interactions
described by $\tilde{W}_{ij}^{yz}=\tilde{W}_{ij}^{yz}(\epsilon t)$ and 3-body
interactions $\tilde{Q}_{ijk}^{xyz}=\tilde{Q}_{ijk}^{xyz}(\epsilon t)$. Other
possible contributions such as a single-particle energy due to $y$-component
of the magnetic field ($\tilde{B}_{y}$), pair-wise interaction
$\tilde{W}_{ij}^{xy}$ and 3-body interaction $\tilde{Q}_{ijk}^{xxy}$ lead to
incompatible algebraic equations in Eq.(8), and should be excluded. The
candidate for regularization Hamiltonian $\mathcal{\tilde{H}}$ then takes the
following form :

$\displaystyle\mathcal{\tilde{H}}$ $\displaystyle=$
$\displaystyle\sum_{(i,j)\in
all}\tilde{W}_{ij}^{yz}(\sigma_{i}^{y}\sigma_{j}^{z}+\sigma_{i}^{z}\sigma_{j}^{y})$
(18) $\displaystyle+$ $\displaystyle\sum_{(i,j,k)\in
all}\tilde{Q}_{ijk}^{xyz}(\sigma_{i}^{x}\sigma_{j}^{y}+\sigma_{i}^{y}\sigma_{j}^{x})\cdot\sigma_{k}^{z},$

where $(i,j)\in all$ and $(i,j,k)\in all$ mean all possible combinations (not
permutations), and are not limited to nearest neighbours. The 3-body
interaction here is not brought as a result of the truncation of long-range
and multi-body counter-diabatic interactions, but is introduced in advance to
make the core equation solvable.

Since regular spin clusters have geometric symmetry, some of the interactions
($\tilde{W}_{ij}^{yz}$) are degenerate as shown in Figs. 1 and 2, and the
reduced number of independent interactions should be equal to the number of
independent equations in Eq.(8). In the present paper, the 3-body interaction
will play a subsidiary role. Below we shall solve the regularization terms and
obtain the fast-forward Hamiltonian for spin clusters of various geometries.

##  IV Regular triangle and open linear 3 spins

In this Section we investigate a regular triangle and open linear 3 spins in
Fig. 1. We use the spin configuration bases as
$\Ket{1}=\Ket{\uparrow\uparrow\uparrow}$,
$\Ket{2}=\Ket{\uparrow\uparrow\downarrow}$,
$\Ket{3}=\Ket{\uparrow\downarrow\uparrow}$,
$\Ket{4}=\Ket{\downarrow\uparrow\uparrow}$,
$\Ket{5}=\Ket{\uparrow\downarrow\downarrow}$,
$\Ket{6}=\Ket{\downarrow\uparrow\downarrow}$,
$\Ket{7}=\Ket{\downarrow\downarrow\uparrow}$ and
$\Ket{8}=\Ket{\downarrow\downarrow\downarrow}$.

###  IV.1 Regular triangle

In the case of the regular triangle, the eigenvalue for the ground state is
$E_{0}=-\sqrt{B_{x}^{2}+2B_{x}J+4J^{2}}-\frac{B_{x}}{2}+J$. We have confirmed
in Fig. 3(a) that all eight eigenvalues show no mutual energy crossing in the
fast-forward time range where we choose $J(R(\Lambda(t)))\equiv R(\Lambda(t))$
and $B_{x}(R(\Lambda(t)))\equiv B_{0}-R(\Lambda(t))$ with $R(\Lambda(t))$
defined in Eq.(16).

(a)  (b)  (c)

Figure 3:  The time dependence in the case of the regular triangle in the
fast-forward time range where we choose $J=R(\Lambda(t))$ and
$B_{x}=B_{0}-R(\Lambda(t))$ with $R(\Lambda(t))$ defined in Eq.(16).
$B_{0}=10$ and $\bar{v}=100$. $T_{FF}=0.1$ and $R_{0}=0$. (a) All eight
eigenvalues. From the bottom, the 2nd and 4th lines are each doubly
degenerate; (b) Regularization term $v(t)\tilde{W}$; (c) Probability
amplitudes for the solution $\Psi_{FF}(t)$ of TDSE,
$|C_{2}^{FF}|^{2}=|C_{3}^{FF}|^{2}=|C_{4}^{FF}|^{2}=|C_{5}^{FF}|^{2}=|C_{6}^{FF}|^{2}=|C_{7}^{FF}|^{2}$
(solid line) and $|C_{1}^{FF}|^{2}=|C_{8}^{FF}|^{2}$ (dashed line).

The components of the eigenvector for the ground state are :
$C_{1}=V_{1}\zeta$, $C_{2}=V_{2}\zeta$, $C_{3}=V_{3}\zeta$,
$C_{4}=V_{4}\zeta$, $C_{5}=V_{5}\zeta$, $C_{6}=V_{6}\zeta$,
$C_{7}=V_{7}\zeta$, $C_{8}=V_{8}\zeta$, where $V_{1}=V_{8}=1$,
$V_{2}=V_{3}=V_{4}=V_{5}=V_{6}=V_{7}=\frac{2\sqrt{B_{x}^{2}+2B_{x}J+4J^{2}}+B_{x}+4J}{3B_{x}}$,
and $\zeta=\frac{1}{\sqrt{2+6V_{2}^{2}}}$.

Here we see the symmetry: $C_{1}=C_{8}$,
$C_{2}=C_{3}=C_{4}=C_{5}=C_{6}=C_{7}$. From $R$-derivative of the
normalization ($\sum_{j=1}^{8}C_{j}^{2}=2C_{1}^{2}+6C_{2}^{2}=1$), we see

$C_{1}\frac{\partial C_{1}}{\partial R}+3C_{2}\frac{\partial C_{2}}{\partial
R}=0,$ (19)

and then the adiabatic phase $\xi=0$.

As for the regularization Hamiltonian for the regular triangle, we can proceed
without having recourse to the 3-body interaction. Three
$\tilde{W}_{ij}^{yz}$s should be identical due to the triangular symmetry in
Fig. 1(a). Therefore the unknown pairwise interaction is only one:
$\tilde{W}\equiv\tilde{W}_{ij}^{yz}$, independent of the pairs $(i,j)$.

By using the spin configuration bases as above, the regularization Hamiltonian
in Eq.(18) is characterized by the matrix elements:
$\mathcal{\tilde{H}}_{1j}=-\mathcal{\tilde{H}}_{j1}=-2i\tilde{W}$ with
$j=2,3,4$, $\mathcal{\tilde{H}}_{8j}=-\mathcal{\tilde{H}}_{j8}=-2i\tilde{W}$
with $j=5,6,7$ and all other elements $=0$. The explicit expression for
$\mathcal{\tilde{H}}$ will help us to solve Eq.(8).

Due to the symmetry of $\\{C_{j}\\}$, the number of independent equations are
only two in Eq.(8) :

$\displaystyle-6\tilde{W}C_{2}$ $\displaystyle=$
$\displaystyle\hbar\frac{\partial C_{1}}{\partial R},$ $\displaystyle
2\tilde{W}C_{1}$ $\displaystyle=$ $\displaystyle\hbar\frac{\partial
C_{2}}{\partial R}.$ (20)

Noting the normalization-assisted relation in Eq.(19), one of the above two
equations becomes trivial, and Eq.(IV.1) has the solution:

$\displaystyle\tilde{W}$ $\displaystyle=$
$\displaystyle\hbar\frac{\partial_{R}C_{2}}{2C_{1}}=\hbar(C_{1}\partial_{R}C_{2}-C_{2}\partial_{R}C_{1})$
(21) $\displaystyle=$ $\displaystyle\frac{B_{x}\frac{\partial J}{\partial
R}-J\frac{\partial B_{x}}{\partial R}}{4(B_{x}^{2}+2B_{x}J+4J^{2})}.$

The second equality above is due to the normalization condition and Eq.(19).
Including the regularization term followed by rescaling of time, the fast
forward Hamiltonian is written as

$H_{FF}=H_{0}(R(\Lambda(t)))+v(t)\tilde{\mathcal{H}}(R(\Lambda(t)))$ (22)

with $H_{0}$ =
$J(R(\Lambda(t)))(\sigma_{1}^{z}\sigma_{2}^{z}+\sigma_{2}^{z}\sigma_{3}^{z}+\sigma_{3}^{z}\sigma_{1}^{z})-\frac{1}{2}(\sigma_{1}^{x}+\sigma_{2}^{x}+\sigma_{3}^{x})B_{x}(R(\Lambda(t)))$,
and $v\mathcal{\tilde{H}}$ =
$v(t)\tilde{W}(R(\Lambda(t)))\big{[}(\sigma_{1}^{y}\sigma_{2}^{z}+\sigma_{1}^{z}\sigma_{2}^{y})+(\sigma_{2}^{y}\sigma_{3}^{z}+\sigma_{2}^{z}\sigma_{3}^{y})+(\sigma_{3}^{y}\sigma_{1}^{z}+\sigma_{3}^{z}\sigma_{1}^{y})\big{]}$.

The fast forward Hamiltonian guarantees the fast forward of the adiabatic
dynamics of the ground state wave function. Figures 3(b) and 3(c) show the
time dependence of the regularization term and that of the wave function,
respectively. The wave function starts from the ground state with $J=0$, i.e.,
$C_{1}=C_{2}=C_{3}=C_{4}=C_{5}=C_{6}=C_{7}=C_{8}=\frac{1}{2\sqrt{2}}$. The
initial state is a linear combination of $\Ket{\uparrow\uparrow\uparrow}$,
$\Ket{\uparrow\uparrow\downarrow}$, $\Ket{\uparrow\downarrow\uparrow}$,
$\Ket{\downarrow\uparrow\uparrow}$, $\Ket{\uparrow\downarrow\downarrow}$,
$\Ket{\downarrow\uparrow\downarrow}$, $\Ket{\downarrow\downarrow\uparrow}$ and
$\Ket{\downarrow\downarrow\downarrow}$ states. As $J$ is increased from $0$
and $B_{x}$ is decreased, the system rapidly changes to the final state, a
linear combination of reduced bases $\Ket{\uparrow\uparrow\downarrow}$,
$\Ket{\uparrow\downarrow\uparrow}$, $\Ket{\downarrow\uparrow\uparrow}$,
$\Ket{\uparrow\downarrow\downarrow}$, $\Ket{\downarrow\uparrow\downarrow}$,
and $\Ket{\downarrow\downarrow\uparrow}$. In Fig. 3 (c) the solution
$\Psi_{FF}(t)$ of TDSE in Eq.(14) has reproduced the time-rescaled ground
state wave function, which means the perfect fidelity of $\Psi_{FF}(t)$ during
the fast-forward time range $0\leq t\leq T_{FF}$.

###  IV.2 Open linear 3 spin chain

In a similar way we can obtain the regularization term and fast-forward
Hamiltonian in the case of open linear 3 spin chain. In this case the
eigenvalue for the ground state is
$E_{0}=-\frac{1}{6}\big{(}B_{x}+(\beta+\bar{\beta})-\sqrt{3}i(\beta-\bar{\beta})\big{)}$,
where
$\beta=\big{(}18J^{2}B_{x}-8B_{x}^{3}+6Ji\sqrt{48J^{4}+39B_{x}^{2}J^{2}+24B_{x}^{4}}\big{)}^{1/3}$.
We have confirmed in Fig. 4 (a) that all eight eigenvalues show no mutual
energy crossing in the fast-forward time range where we choose
$J(R(\Lambda(t)))\equiv R(\Lambda(t))$ and $B_{x}(R(\Lambda(t)))\equiv
B_{0}-R(\Lambda(t))$ with $R(\Lambda(t))$ defined in Eq.(16).

The components of the eigenvector for the ground state are :
$C_{1}=C_{8}=V_{1}\zeta$, $C_{2}=C_{4}=C_{5}=C_{7}=V_{2}\zeta$,
$C_{3}=C_{6}=V_{3}\zeta$, where
$V_{1}={\frac{3{B_{x}}^{2}-8JB_{x}-4B_{x}E_{0}-4E_{0}^{2}-8E_{0}J}{4JB_{x}}}$,
$V_{2}=-\frac{1}{2}V_{1}-\frac{2J+E_{0}}{B_{x}}$, $V_{3}=1$, and
$\zeta=\frac{1}{\sqrt{2V_{1}^{2}+4V_{2}^{2}+2}}$.

Here we see the symmetry: $C_{1}=C_{8}$, $C_{2}=C_{4}=C_{5}=C_{7}$ and
$C_{3}=C_{6}$. From $R$-derivative of the normalization
($\sum_{j=1}^{8}C_{j}^{2}=2C_{1}^{2}+4C_{2}^{2}+2C_{3}^{2}=1$), we see

$C_{1}\frac{\partial C_{1}}{\partial R}+2C_{2}\frac{\partial C_{2}}{\partial
R}+C_{3}\frac{\partial C_{3}}{\partial R}=0,$ (23)

and then the adiabatic phase $\xi=0$.

(a)  (b)  (c)

Figure 4:  The same time dependence as in Fig. 3, but in the case of the open
linear 3 spin chain. (a) All eight eigenvalues; (b) Regularization terms
$v(t)\tilde{W}_{1}$ (dashed line) and $v(t)\tilde{W}_{2}$ (dotted line) ; (c)
Probability amplitudes for the solution $\Psi_{FF}(t)$ of TDSE,
$|C_{3}^{FF}|^{2}=|C_{6}^{FF}|^{2}$ (solid line),
$|C_{1}^{FF}|^{2}=|C_{8}^{FF}|^{2}$ (dashed line) and
$|C_{2}^{FF}|^{2}=|C_{4}^{FF}|^{2}=|C_{5}^{FF}|^{2}=|C_{7}^{FF}|^{2}$ (dotted
line).

The regularization Hamiltonian for the linear 3 spin system can also be
available without using the 3-body interaction. Because of the geometric
symmetry seen in Fig. 1(b), $\mathcal{\tilde{H}}$ is then characterized by two
independent pairwise interactions:
$\tilde{W}_{1}\equiv\tilde{W}_{12}^{yz}=\tilde{W}_{23}^{yz}$ and
$\tilde{W}_{2}\equiv\tilde{W}_{31}^{yz}$. $\tilde{W}_{1}$ and $\tilde{W}_{2}$
correspond to the nearest-neighboring (N.N.) and 2nd N.N. interactions,
respectively. With use of the spin configuration bases, the matrix form for
$\mathcal{\tilde{H}}$ in Eq. (18) is given by

$\mathcal{\tilde{H}}=i\begin{pmatrix}0&-\tilde{W}_{1}-\tilde{W}_{2}&-2\tilde{W}_{1}&-\tilde{W}_{1}-\tilde{W}_{2}&0&0&0&0\\\
\tilde{W}_{1}+\tilde{W}_{2}&0&0&0&0&-\tilde{W}_{1}+\tilde{W}_{2}&0&0\\\
2\tilde{W}_{1}&0&0&0&\tilde{W}_{1}-\tilde{W}_{2}&0&\tilde{W}_{1}-\tilde{W}_{2}&0\\\
\tilde{W}_{1}+\tilde{W}_{2}&0&0&0&0&-\tilde{W}_{1}+\tilde{W}_{2}&0&0\\\
0&0&-\tilde{W}_{1}+\tilde{W}_{2}&0&0&0&0&\tilde{W}_{1}+\tilde{W}_{2}\\\
0&\tilde{W}_{1}-\tilde{W}_{2}&0&\tilde{W}_{1}-\tilde{W}_{2}&0&0&0&2\tilde{W}_{1}\\\
0&0&-\tilde{W}_{1}+\tilde{W}_{2}&0&0&0&0&\tilde{W}_{1}+\tilde{W}_{2}\\\
0&0&0&0&-\tilde{W}_{1}-\tilde{W}_{2}&-2\tilde{W}_{1}&-\tilde{W}_{1}-\tilde{W}_{2}&0\end{pmatrix}.$
(24)

Due to the symmetry of $\\{C_{j}\\}$, the number of independent equations in
Eq. (8) are three:

$\displaystyle-2(\tilde{W}_{1}+\tilde{W}_{2})C_{2}-2\tilde{W}_{1}C_{3}$
$\displaystyle=$ $\displaystyle\hbar\frac{\partial C_{1}}{\partial R}$
$\displaystyle(\tilde{W}_{1}+\tilde{W}_{2})C_{1}+(-\tilde{W}_{1}+\tilde{W}_{2})C_{3}$
$\displaystyle=$ $\displaystyle\hbar\frac{\partial C_{2}}{\partial R}$
$\displaystyle 2\tilde{W}_{1}C_{1}+2(\tilde{W}_{1}-\tilde{W}_{2})C_{2}$
$\displaystyle=$ $\displaystyle\hbar\frac{\partial C_{3}}{\partial R}.$

By using Eq.(23), the 3rd line (for example) of the above equation proves
trivial. Then Eq.(IV.2), whose coefficient matrix has the rank 2, gives the
solution:

$\displaystyle\tilde{W}_{1}$ $\displaystyle=$
$\displaystyle-\frac{\hbar}{2}\frac{\partial(C_{1}-C_{3})}{\partial
R}/(C_{1}+2C_{2}+C_{3}),$ $\displaystyle\tilde{W}_{2}$ $\displaystyle=$
$\displaystyle-\frac{\hbar}{2}\frac{\partial(C_{1}-2C_{2}+C_{3})}{\partial
R}/(C_{1}+2C_{2}+C_{3}).$

Including the regularization terms followed by rescaling of time, the fast
forward Hamiltonian are written as

$H_{FF}=H_{0}(R(\Lambda(t)))+v(t)\tilde{\mathcal{H}}(R(\Lambda(t)))$ (27)

with $H_{0}$ =
$J(R(\Lambda(t)))(\sigma_{1}^{z}\sigma_{2}^{z}+\sigma_{2}^{z}\sigma_{3}^{z})-\frac{1}{2}(\sigma_{1}^{x}+\sigma_{2}^{x}+\sigma_{3}^{x})B_{x}(R(\Lambda(t)))$,
and $v\mathcal{\tilde{H}}$ =
$v(t)\tilde{W}_{1}(R(\Lambda(t)))\big{[}(\sigma_{1}^{y}\sigma_{2}^{z}+\sigma_{1}^{z}\sigma_{2}^{y})+(\sigma_{2}^{y}\sigma_{3}^{z}+\sigma_{2}^{z}\sigma_{3}^{y})\big{]}+v(t)\tilde{W}_{2}(R(\Lambda(t)))(\sigma_{1}^{y}\sigma_{3}^{z}+\sigma_{1}^{z}\sigma_{3}^{y})$.
The fast forward Hamiltonian guarantees the fast forward of the adiabatic
dynamics of the ground state wave function. Figures 4(b) and 4(c) show the
time dependence of the regularization terms and that of the wave function,
respectively. The wave function starts from the ground state with $J=0$, i.e.,
$C_{j}=\frac{1}{2\sqrt{2}}$ for $j=1,\cdots,8$. As $J$ is increased from $0$
and $B_{x}$ is decreased, the system rapidly changes to the final state, i.e.,
a linear combination of reduced bases. In Fig. 4 (c) the solution
$\Psi_{FF}(t)$ of TDSE in Eq.(14) has exactly reproduced the time-rescaled
ground state wave function.

In case of $N=3$ spin systems, we have obtained the regularization terms and
the fast-forward Hamiltonian without having recourse to the 3-body
interaction. Of course, we can see regularization terms which include the
3-body interaction: For a regular triangle we can have an extra solution
consisting of only the 3-body interaction ($\tilde{Q}$), and for the open
linear 3 spin system there can be solutions where $\tilde{Q}\neq 0$ and one of
$\tilde{W_{1}}$ and $\tilde{W_{2}}$ is non-vanishing. But these extra
solutions are less interesting from the viewpoint of searching for simpler
controls. In the case of $N=4$ spin systems in next Section, however, we
cannot proceed without the 3-body interaction, although it will play only a
subsidiary role.

##  V triangular pyramid, square, star graph and open linear 4 spin chain

Now we shall investigate regular spin clusters with $N=4$ spins, namely, a
triangular pyramid, square, star graph and open linear 4 spin chain in Fig. 2.
Their original (reference) and regularization Hamiltonians are already given
by Eq.(17) and Eq.(18), respectively, where we put $N=4$.

By using the spin configuration bases,
$\Ket{1}=\Ket{\uparrow\uparrow\uparrow\uparrow}$,
$\Ket{2}=\Ket{\uparrow\uparrow\uparrow\downarrow}$,
$\Ket{3}=\Ket{\uparrow\uparrow\downarrow\uparrow}$,
$\Ket{4}=\Ket{\uparrow\downarrow\uparrow\uparrow}$,
$\Ket{5}=\Ket{\downarrow\uparrow\uparrow\uparrow}$
$\Ket{6}=\Ket{\uparrow\uparrow\downarrow\downarrow}$,
$\Ket{7}=\Ket{\uparrow\downarrow\downarrow\uparrow}$,
$\Ket{8}=\Ket{\downarrow\downarrow\uparrow\uparrow}$,
$\Ket{9}=\Ket{\downarrow\uparrow\uparrow\downarrow}$,
$\Ket{10}=\Ket{\uparrow\downarrow\uparrow\downarrow}$,
$\Ket{11}=\Ket{\downarrow\uparrow\downarrow\uparrow}$,
$\Ket{12}=\Ket{\downarrow\downarrow\downarrow\uparrow}$,
$\Ket{13}=\Ket{\downarrow\downarrow\uparrow\downarrow}$,
$\Ket{14}=\Ket{\downarrow\uparrow\downarrow\downarrow}$,
$\Ket{15}=\Ket{\uparrow\downarrow\downarrow\downarrow}$, and
$\Ket{16}=\Ket{\downarrow\downarrow\downarrow\downarrow}$, the matrix form for
original Hamiltonian $H_{0}$ in Eq.(17) can be constructed.

###  V.1 Triangular pyramid

The eigenvalue of the ground state is
$E_{0}=\frac{1}{3}(-(\beta+\bar{\beta})+4J+\sqrt{3}i(\beta-\bar{\beta}))$,
where
$\beta=(35\,{J}^{3}-18{B_{x}}^{2}J+3i\sqrt{108{J}^{6}+309{B_{x}}^{2}{J}^{4}+3{B_{x}}^{4}{J}^{2}+3{B_{x}}^{6}})^{1/3}.$
For all regular clusters with $N=4$ spins in Fig. 2, as is the case of the
previous Section, we have numerically confirmed that there is no level
crossing between the ground and 1st excited states in the fast-forward time
range. So figures of 16 eigenvalues will be suppressed in this Section.

The components of the eigenvector of the ground state are:
$C_{1}=C_{16}=V_{1}\zeta$,
$C_{2}=C_{3}=C_{4}=C_{5}=C_{12}=C_{13}=C_{14}=C_{15}=V_{2}\zeta$, and
$C_{6}=C_{7}=C_{8}=C_{9}=C_{10}=C_{11}=V_{6}\zeta$. Here
$\zeta=(2+8V_{2}^{2}+6V_{6}^{2})^{-1/2}$, $V_{1}=1$,
$V_{2}=\frac{(\beta+\bar{\beta})+14J-\sqrt{3}i(\beta-\bar{\beta})}{6B_{x}}$,
and
$V_{6}=-{\frac{2\left(\beta^{2}+\bar{\beta}^{2}\right)-10J\left(\beta+\bar{\beta}\right)-\left(48J^{2}+15B_{x}^{2}\right)+i\sqrt{3}(2\left(\beta^{2}-\bar{\beta}^{2}\right)+10J\left(\beta-\bar{\beta}\right))}{27{B_{x}}^{2}}}$,
where the equality $|\beta|^{2}=13J^{2}+3B_{x}^{2}$ is used.

From $R$-derivative of the normalization
($2C_{1}^{2}+8C_{2}^{2}+6C_{6}^{2}=1$), we see

$C_{1}\frac{\partial C_{1}}{\partial R}+4C_{2}\frac{\partial C_{2}}{\partial
R}+3C_{6}\frac{\partial C_{6}}{\partial R}=0.$ (28)

If we suppress the 3-body interaction, the regularization Hamiltonian consists
of only one pairwise interaction $\tilde{W}\equiv\tilde{W}_{ij}^{yz}$, due to
the high symmetry of the triangular pyramid in Fig.2 (a). The corresponding
matrix for the regularization term can be written as

$\mathcal{\tilde{H}}=i\left(\begin{array}[]{cccccccccccccccc}0&-3\tilde{W}&-3\tilde{W}&-3\tilde{W}&-3\tilde{W}&0&0&0&0&0&0&0&0&0&0&0\\\
3\tilde{W}&0&0&0&0&-\tilde{W}&0&0&-\tilde{W}&-\tilde{W}&0&0&0&0&0&0\\\
3\tilde{W}&0&0&0&0&-\tilde{W}&-\tilde{W}&0&0&0&-\tilde{W}&0&0&0&0&0\\\
3\tilde{W}&0&0&0&0&0&-\tilde{W}&-\tilde{W}&0&-\tilde{W}&0&0&0&0&0&0\\\
3\tilde{W}&0&0&0&0&0&0&-\tilde{W}&-\tilde{W}&0&-\tilde{W}&0&0&0&0&0\\\
0&\tilde{W}&\tilde{W}&0&0&0&0&0&0&0&0&0&0&\tilde{W}&\tilde{W}&0\\\
0&0&\tilde{W}&\tilde{W}&0&0&0&0&0&0&0&\tilde{W}&0&0&\tilde{W}&0\\\
0&0&0&\tilde{W}&\tilde{W}&0&0&0&0&0&0&\tilde{W}&\tilde{W}&0&0&0\\\
0&\tilde{W}&0&0&\tilde{W}&0&0&0&0&0&0&0&\tilde{W}&\tilde{W}&0&0\\\
0&\tilde{W}&0&\tilde{W}&0&0&0&0&0&0&0&0&\tilde{W}&0&\tilde{W}&0\\\
0&0&\tilde{W}&0&\tilde{W}&0&0&0&0&0&0&\tilde{W}&0&\tilde{W}&0&0\\\
0&0&0&0&0&0&-\tilde{W}&-\tilde{W}&0&0&-\tilde{W}&0&0&0&0&3\tilde{W}\\\
0&0&0&0&0&0&0&-\tilde{W}&-\tilde{W}&-\tilde{W}&0&0&0&0&0&3\tilde{W}\\\
0&0&0&0&0&-\tilde{W}&0&0&-\tilde{W}&0&-\tilde{W}&0&0&0&0&3\tilde{W}\\\
0&0&0&0&0&-\tilde{W}&-\tilde{W}&0&0&-\tilde{W}&0&0&0&0&0&3\tilde{W}\\\
0&0&0&0&0&0&0&0&0&0&0&-3\tilde{W}&-3\tilde{W}&-3\tilde{W}&-3\tilde{W}&0\end{array}\right).$
(29)

Due to the symmetry of $\\{C_{j}\\}$, the number of independent equations in
Eq.(8) are three:

$\displaystyle-12\tilde{W}C_{2}$ $\displaystyle=$
$\displaystyle\hbar\frac{\partial C_{1}}{\partial R}$ $\displaystyle
3\tilde{W}C_{1}-3\tilde{W}C_{6}$ $\displaystyle=$
$\displaystyle\hbar\frac{\partial C_{2}}{\partial R}$ $\displaystyle
4\tilde{W}C_{2}$ $\displaystyle=$ $\displaystyle\hbar\frac{\partial
C_{6}}{\partial R}.$ (30)

While one of the above equations is trivial due to Eq.(28), we need one more
unknown variable to make meaningful the algebraic equations in Eq. (V.1). Here
we evaluate the contribution of the 3-body interaction. The geometrical
symmetry allows a universal 3-body interaction
$\tilde{Q}\equiv\tilde{Q}_{ijk}^{xyz}$, independent of all possible 3-body
configurations $(i,j,k)$. The inclusion of the 3-body interaction improves
some matrix elements of $\mathcal{\tilde{H}}$ in Eq.(29) as follows:

$\displaystyle\mathcal{\tilde{H}}_{1,j}$ $\displaystyle=$
$\displaystyle\mathcal{\tilde{H}}_{16,j}=-4i\tilde{Q}\quad\rm{for}\quad\textit{j}=6,\cdots,11,$
$\displaystyle\mathcal{\tilde{H}}_{i,1}$ $\displaystyle=$
$\displaystyle\mathcal{\tilde{H}}_{i,16}=4i\tilde{Q}\quad\rm{for}\quad\textit{i}=6,\cdots,11.$

After the above improvements, the algebraic equations in Eq.(V.1) are revised
as:

$\displaystyle-12\tilde{W}C_{2}-24\tilde{Q}C_{6}$ $\displaystyle=$
$\displaystyle\hbar\frac{\partial C_{1}}{\partial R}$ $\displaystyle
3\tilde{W}C_{1}-3\tilde{W}C_{6}$ $\displaystyle=$
$\displaystyle\hbar\frac{\partial C_{2}}{\partial R}$ $\displaystyle
8\tilde{Q}C_{1}+4\tilde{W}C_{2}$ $\displaystyle=$
$\displaystyle\hbar\frac{\partial C_{6}}{\partial R},$ (32)

where one of the above lines is again trivial because of Eq. (28). Equation
(V.1), whose coefficient matrix has the rank 2, gives the solution:

$\displaystyle\tilde{W}$ $\displaystyle=$
$\displaystyle\frac{\hbar\partial_{R}C_{2}}{3(C_{1}-C_{6})},$
$\displaystyle\tilde{Q}$ $\displaystyle=$
$\displaystyle\frac{\hbar\partial_{R}(C_{1}+3C_{6})}{24(C_{1}-C_{6})}.$ (33)

The fast-forward Hamiltonian is given by

$\displaystyle H_{FF}$ $\displaystyle=$ $\displaystyle
J(R(\Lambda(t)))\sum_{(i,j)\in
N.N.}\sigma_{i}^{z}\sigma_{j}^{z}-\frac{1}{2}B_{x}(R(\Lambda(t)))\sum_{i=1}^{4}\sigma_{i}^{x},$
(34) $\displaystyle+$ $\displaystyle v(t)\tilde{\mathcal{H}}(R(\Lambda(t)))$

with

$\displaystyle v\mathcal{\tilde{H}}$ $\displaystyle=$
$\displaystyle\sum_{(i,j)\in
all}v(t)\tilde{W}(R(\Lambda(t)))(\sigma_{i}^{y}\sigma_{j}^{z}+\sigma_{i}^{z}\sigma_{j}^{y})$
$\displaystyle+$ $\displaystyle\sum_{(i,j,k)\in
all}v(t)\tilde{Q}(R(\Lambda(t)))(\sigma_{i}^{x}\sigma_{j}^{y}+\sigma_{i}^{y}\sigma_{j}^{x})\cdot\sigma_{k}^{z}.$

In the triangular pyramid, $\sum_{(i,j)\in N.N.}$ is equivalent to
$\sum_{(i,j)\in all}$. The fast forward Hamiltonian guarantees the fast
forward of the adiabatic dynamics of the ground state wave function.

Figures 5 (a) and 6 (a) show the time dependence of regularization terms and
that of the wave function, respectively. The wave function starts from the
ground state with $J=0$, i.e., $C_{j}=\frac{1}{4}$ for $j=1,\cdots,16$. In
Fig. 6 (a) the solution $\Psi_{FF}(t)$ of TDSE in Eq.(14) has exactly
reproduced the time-rescaled ground state wave function during the fast-
forward time range $0\leq t\leq T_{FF}$.

(a)  (b)

(c)  (d)

Figure 5: The time dependence of regularization terms multiplied by $v(t)$ in
the fast-forward time range where we choose $J=R(\Lambda(t))$ and
$B_{x}=B_{0}-R(\Lambda(t))$ with $R(\Lambda(t))$ defined in Eq.(16).
$B_{0}=10$ and $\bar{v}=100$. $T_{FF}=0.1$ and $R_{0}=0$. (a) Triangular
pyramid. $v(t)\tilde{W}$ (dashed line) and $v(t)\tilde{Q}$ (solid line); (b)
Square. $v(t)\tilde{W}_{1}$ (dashed line), $v(t)\tilde{W}_{2}$ (dotted line)
and $v(t)\tilde{Q}$ (solid line) ; (c) Primary star graph. $v(t)\tilde{W}_{1}$
(dashed line), $v(t)\tilde{W}_{2}$ (dotted line) and $v(t)\tilde{Q}$ (solid
line) ; (d) Open linear 4 spins. $v(t)\tilde{W}_{1}$ (dashed line),
$v(t)\tilde{W}_{2}$ (dotted line), $v(t)\tilde{W}_{3}$ (dotted dashed line),
$v(t)\tilde{W}_{4}$ (double-dotted dashed line) and $v(t)\tilde{Q}$ (solid
line).

(a)  (b)

(c)  (d)

Figure 6: Time dependence of probability amplitudes $|C_{j}^{FF}|^{2}$ with
$j=1\sim 16$ for the solution $\Psi_{FF}(t)$ of TDSE in the fast-forward time
range, where we choose $J=R(\Lambda(t))$ and $B_{x}=B_{0}-R(\Lambda(t))$ with
$R(\Lambda(t))$ defined in Eq.(16). $B_{0}=10$ and $\bar{v}=100$. $T_{FF}=0.1$
and $R_{0}=0$. (a) Triangular pyramid. $j=1,16$ (dashed line), $j=2\sim 5$ and
$12\sim 15$ (dotted line), and $j=6\sim 11$ (solid line); (b) Square. $j=1,16$
(dotted dashed line), $j=2\sim 5$ and $12\sim 15$ (dotted line), $j=6\sim 9$
(dashed line), and $j=10,11$ (solid line); (c) Primary star graph. $j=1,16$
(dashed line), $j=2,3,5,12,13,15$ (dotted line) , $j=4,14$ (solid line), and
$j=6\sim 11$ (dotted dashed line); (d) Open linear 4 spins.$j=1,16$ (lower
solid line), $j=2,5,12,15$ (dotted dashed line), $j=3,4,13,14$ (dashed line),
$j=6,8$ (dotted line), $j=7,9$ (double-dotted dashed line), and
$j=10,11$(upper solid line).

###  V.2 Square

The eigenvalue of the ground state is $E_{0}=-\beta_{1}$, where
$\beta_{1}=\sqrt{8{J}^{2}+2{B_{x}}^{2}+2\beta_{2}}$ with
$\beta_{2}=\sqrt{16{J}^{4}+{B_{x}}^{4}}$. The components of the eigenvector of
the ground state are: $C_{1}=C_{16}=V_{1}\zeta$,
$C_{2}=C_{3}=C_{4}=C_{5}=C_{12}=C_{13}=C_{14}=C_{15}=V_{2}\zeta$,
$C_{6}=C_{7}=C_{8}=C_{9}=V_{6}\zeta$, and $C_{10}=C_{11}=V_{10}\zeta$ with
$\zeta=(8+2V_{1}^{2}+4V_{6}^{2}+2V_{10}^{2})^{-1/2}$. Here
$V_{1}=\frac{(\beta_{1}-4J)(4J^{2}-B_{x}^{2}+\beta_{2})}{8J^{2}B_{x}}$,
$V_{2}=1$, $V_{6}=\frac{(\beta_{1}^{2}-4\beta_{2})\beta_{1}}{16J^{2}B_{x}}$,
and $V_{10}=\frac{(\beta_{1}+4J)(4J^{2}-B_{x}^{2}+\beta_{2})}{8J^{2}B_{x}}$.
From $R$-derivative of the normalization
($2C_{1}^{2}+8C_{2}^{2}+4C_{6}^{2}+2C_{10}^{2}=1$), we see

$C_{1}\frac{\partial C_{1}}{\partial R}+4C_{2}\frac{\partial C_{2}}{\partial
R}+2C_{6}\frac{\partial C_{6}}{\partial R}+C_{10}\frac{\partial
C_{10}}{\partial R}=0$ (36)

The geometric symmetry of the square spin system in Fig.2 (b) allows two
candidates as regularization terms, which are
$\tilde{W}_{12}=\tilde{W}_{23}=\tilde{W}_{34}=\tilde{W}_{41}=\tilde{W}_{1}$
and $\tilde{W}_{31}=\tilde{W}_{42}=\tilde{W}_{2}$. $\tilde{W}_{1}$ and
$\tilde{W}_{2}$ correspond to N.N. and the second N.N. interactions,
respectively. The regularization matrix $\mathcal{\tilde{H}}$ is given in
Eq.(49). To add one more unknown variable, we include a contribution of the
universal 3-body interaction $\tilde{Q}\equiv\tilde{Q}_{ijk}^{xyz}$. This
inclusion requires the same improvement of some matrix elements of
$\mathcal{\tilde{H}}$ as in Eq. (V.1).

Due to the symmetry of $\\{C_{j}\\}$, the number of independent algebraic
equations are four:

$\displaystyle(-8\tilde{W}_{1}-4\tilde{W}_{2})C_{2}-16\tilde{Q}C_{6}-8\tilde{Q}C_{10}$
$\displaystyle=$ $\displaystyle\hbar\frac{\partial C_{1}}{\partial R}$
$\displaystyle(2\tilde{W}_{1}+\tilde{W}_{2})C_{1}+(-2\tilde{W}_{2})C_{6}+(-2\tilde{W}_{1}+\tilde{W}_{2})C_{10}$
$\displaystyle=$ $\displaystyle\hbar\frac{\partial C_{2}}{\partial R}$
$\displaystyle 8\tilde{Q}C_{1}+4\tilde{W}_{2}C_{2}$ $\displaystyle=$
$\displaystyle\hbar\frac{\partial C_{6}}{\partial R}.$ $\displaystyle
8\tilde{Q}C_{1}+(8\tilde{W}_{1}-4\tilde{W}_{2})C_{2}$ $\displaystyle=$
$\displaystyle\hbar\frac{\partial C_{10}}{\partial R}.$

Because of Eq.(36), one of the above equations is trivial. Ignoring the second
line for example, Eq.(V.2), whose coefficient matrix has the rank 3, gives the
solution:

$\displaystyle\tilde{W}_{1}$ $\displaystyle=$
$\displaystyle-\frac{\hbar}{8(3C_{1}-C_{10}-2C_{6})C_{2}}\times(C_{1}\partial_{R}C_{1}-4C_{2}\partial_{R}C_{2}$
$\displaystyle+$
$\displaystyle(C_{1}+C_{10})\partial_{R}C_{6}-(C_{1}-2C_{6})\partial_{R}C_{10}),$
$\displaystyle\tilde{W}_{2}$ $\displaystyle=$
$\displaystyle-\hbar\frac{C_{1}\partial_{R}C_{1}-(C_{1}-2C_{6}-C_{10})\partial_{R}C_{6}+C_{1}\partial_{R}C_{10}}{4(3C_{1}-C_{10}-2C_{6})C_{2}},$
$\displaystyle\tilde{Q}$ $\displaystyle=$
$\displaystyle\frac{\hbar\partial_{R}(C_{1}+2C_{6}+C_{10})}{8(3C_{1}-C_{10}-2C_{6})C_{2}}.$
(38)

The fast-forward Hamiltonian is given by Eq.(34), where
$v(t)\tilde{\mathcal{H}}(R(\Lambda(t)))$ is now replaced by:

$\displaystyle v\mathcal{\tilde{H}}$ $\displaystyle=$
$\displaystyle\sum_{(i,j)=(1,2),(2,3),(3,4),(4,1)}v(t)\tilde{W}_{1}(R(\Lambda(t)))(\sigma_{i}^{y}\sigma_{j}^{z}+\sigma_{i}^{z}\sigma_{j}^{y})$
$\displaystyle+$
$\displaystyle\sum_{(i,j)=(3,1),(4,2)}v(t)\tilde{W}_{2}(R(\Lambda(t)))(\sigma_{i}^{y}\sigma_{j}^{z}+\sigma_{i}^{z}\sigma_{j}^{y})$
$\displaystyle+$ $\displaystyle\sum_{(i,j,k)\in
all}v(t)\tilde{Q}(R(\Lambda(t)))(\sigma_{i}^{x}\sigma_{j}^{y}+\sigma_{i}^{y}\sigma_{j}^{x})\cdot\sigma_{k}^{z}.$

Figures 5 (b) and 6 (b) show the time dependence of regularization terms and
that of wave function, respectively. The wave function starts from the ground
state with $J=0$, i.e., $C_{j}=\frac{1}{4}$ for $j=1,\cdots,16$. In Fig. 6 (b)
the solution $\Psi_{FF}(t)$ of TDSE in Eq.(14) has exactly reproduced the
time-rescaled ground state wave function.

###  V.3 Primary star graph

The eigenvalue of the ground state is $E_{0}=-\beta$, where
$\beta=\sqrt{2{B_{x}}^{2}+5{J}^{2}+2\sqrt{{B_{x}}^{4}+{B_{x}}^{2}{J}^{2}+4{J}^{4}}}$.
The components of the eigenvector of the ground state are:
$C_{1}=C_{16}=V_{1}\zeta$,
$C_{2}=C_{3}=C_{5}=C_{12}=C_{13}=C_{15}=V_{2}\zeta$,
$C_{4}=C_{14}=V_{4}\zeta$, and
$C_{6}=C_{7}=C_{8}=C_{9}=C_{10}=C_{11}=V_{6}\zeta$ with
$\zeta=(6+2V_{1}^{2}+6V_{2}^{2}+2V_{4}^{2})^{-1/2}$. Here
$V_{1}=\frac{-J(7{B_{x}}^{2}+3J^{2})+\beta(4{B_{x}}^{2}+3\beta
J-{\beta}^{2}+J^{2})}{5J{B_{x}}^{2}}$,
$V_{2}=\frac{-2J(9J^{2}-4{B_{x}}^{2})+\beta(4{B_{x}}^{2}-2\beta
J-{\beta}^{2}+21J^{2})}{30J^{2}B_{x}}$,
$V_{4}=\frac{-2J(J^{2}+4{B_{x}}^{2})-\beta(4{B_{x}}^{2}-2\beta
J-{\beta}^{2}+J^{2})}{10J^{2}B_{x}}$, and $V_{6}=1$. From $R$-derivative of
the normalization ($2C_{1}^{2}+6C_{2}^{2}+2C_{4}^{2}+6C_{6}^{2}=1$), we see

$C_{1}\frac{\partial C_{1}}{\partial R}+3C_{2}\frac{\partial C_{2}}{\partial
R}+C_{4}\frac{\partial C_{4}}{\partial R}+3C_{6}\frac{\partial C_{6}}{\partial
R}=0.$ (40)

The geometric symmetry of the primary star-graph spin system in Fig.2(c)
allows two candidates as regularization terms, which are
$\tilde{W}_{12}=\tilde{W}_{23}=\tilde{W}_{24}=\tilde{W}_{1}$ and
$\tilde{W}_{14}=\tilde{W}_{13}=\tilde{W}_{34}=\tilde{W}_{2}$. $\tilde{W}_{1}$
and $\tilde{W}_{2}$ correspond to N.N. and the 2nd N.N. interactions,
respectively. The matrix for regularization term $\mathcal{\tilde{H}}$ can be
written in Eq.(50). To add one more unknown variable, we include a
contribution of the universal 3-body interaction
$\tilde{Q}\equiv\tilde{Q}_{ijk}^{xyz}$. This inclusion requires the same
improvement of some matrix elements of $\mathcal{\tilde{H}}$ as in Eq. (V.1).
One might have an idea to include two species of 3-body interactions with one
among N.N.s and another among the 2nd N.N.s. But this idea results in
incompatible equations in Eq.(8) and cannot be acceptable. Due to the symmetry
of $\\{C_{j}\\}$, the number of independent equations are four:

$\displaystyle(-6\tilde{W}_{2}-3\tilde{W}_{1})C_{2}+(-3\tilde{W}_{1})C_{4}-24\tilde{Q}C_{6}$
$\displaystyle=$ $\displaystyle\hbar\frac{\partial C_{1}}{\partial R}$
$\displaystyle(2\tilde{W}_{2}+\tilde{W}_{1})C_{1}+(-3\tilde{W}_{1})C_{6}$
$\displaystyle=$ $\displaystyle\hbar\frac{\partial C_{2}}{\partial R}$
$\displaystyle(3\tilde{W}_{1})C_{1}+(3\tilde{W}_{1}-6\tilde{W}_{2})C_{6}$
$\displaystyle=$ $\displaystyle\hbar\frac{\partial C_{4}}{\partial R}.$
$\displaystyle
8\tilde{Q}C_{1}+(3\tilde{W}_{1})C_{2}+(-\tilde{W}_{1}+2\tilde{W}_{2})C_{4}$
$\displaystyle=$ $\displaystyle\hbar\frac{\partial C_{6}}{\partial R}.$

Because of Eq.(40), one of the above 4 equations becomes trivial. Ignoring the
first line for example, Eq.(V.3), whose coefficient matrix has the rank 3,
gives the solution:

$\displaystyle\tilde{W}_{1}$ $\displaystyle=$
$\displaystyle\hbar\frac{C_{1}\partial_{R}C_{4}+3C_{6}\partial_{R}C_{2}}{3(C_{1}-C_{6})(C_{1}+3C_{6})},$
$\displaystyle\tilde{W}_{2}$ $\displaystyle=$
$\displaystyle\hbar\frac{3(C_{1}+C_{6})\partial_{R}C_{2}-(C_{1}-3C_{6})\partial_{R}C_{4}}{6(C_{1}-C_{6})(C_{1}+3C_{6})},$
$\displaystyle\tilde{Q}$ $\displaystyle=$
$\displaystyle\frac{\hbar}{24C_{1}(C_{1}-C_{6})(C_{1}+3C_{6})}$
$\displaystyle\times$
$\displaystyle\bigl{(}3(C_{1}^{2}+2C_{1}C_{6}-3C_{6}^{2})\partial_{R}C_{6}$
$\displaystyle-$ $\displaystyle 3(3C_{2}C_{6}+C_{1}C_{4})\partial_{R}C_{2}$
$\displaystyle-$
$\displaystyle(3C_{1}C_{2}-2C_{1}C_{4}+3C_{4}C_{6})\partial_{R}C_{4}\bigr{)}.$

The fast-forward Hamiltonian is given by Eq.(34), where
$v(t)\tilde{\mathcal{H}}(R(\Lambda(t)))$ is replaced by:

$\displaystyle v\mathcal{\tilde{H}}$ $\displaystyle=$
$\displaystyle\sum_{(i,j)=(1,2),(2,3),(2,4)}v(t)\tilde{W}_{1}(R(\Lambda(t)))(\sigma_{i}^{y}\sigma_{j}^{z}+\sigma_{i}^{z}\sigma_{j}^{y})$
$\displaystyle+$
$\displaystyle\sum_{(i,j)=(1,4),(1,3),(3,4)}v(t)\tilde{W}_{2}(R(\Lambda(t)))(\sigma_{i}^{y}\sigma_{j}^{z}+\sigma_{i}^{z}\sigma_{j}^{y})$
$\displaystyle+$ $\displaystyle\sum_{(i,j,k)\in
all}v(t)\tilde{Q}(R(\Lambda(t)))(\sigma_{i}^{x}\sigma_{j}^{y}+\sigma_{i}^{y}\sigma_{j}^{x})\cdot\sigma_{k}^{z}.$

Figures 5 (c) and 6 (c) show the time dependence of regularization terms and
that of wave function, respectively. The wave function starts from the ground
state with $J=0$, i.e., $C_{j}=\frac{1}{4}$ for $j=1,\cdots,16$. In Fig. 6 (c)
the solution $\Psi_{FF}(t)$ of TDSE in Eq.(14) has exactly reproduced the
time-rescaled ground state wave function.

###  V.4 Open linear 4 spin chain

The eigenvalue of the ground state is $E_{0}=-\frac{\beta_{2}}{\sqrt{3}}$,
where $\beta_{2}=\sqrt{2(\beta_{1}+\bar{\beta_{1}})+11J^{2}+4B_{x}^{2}}$ with
$\beta_{1}=\bigl{(}64{J}^{6}+15{J}^{4}{B_{x}}^{2}+21{B_{x}}^{4}{J}^{2}+8{B_{x}}^{6}+3\sqrt{3}J^{2}B_{x}i\sqrt{128{J}^{6}+93{J}^{4}{B_{x}}^{2}+51{B_{x}}^{4}{J}^{2}+25{B_{x}}^{6}}\bigr{)}^{1/3}$
and $|\beta_{1}|^{2}=4{B_{x}}^{4}+7{B_{x}}^{2}J^{2}+16J^{4}$. The components
of the eigenvector of the ground state are: $C_{1}=C_{16}=V_{1}\zeta$,
$C_{2}=C_{5}=C_{12}=C_{15}=V_{2}\zeta$,
$C_{3}=C_{4}=C_{13}=C_{14}=V_{3}\zeta$, $C_{6}=C_{8}=V_{6}\zeta$,
$C_{7}=C_{9}=V_{7}\zeta$, and $C_{10}=C_{11}=V_{10}\zeta$ with
$\zeta=(2+4V_{2}^{2}+4V_{3}^{2}+2V_{6}^{2}+2V_{7}^{2}+2V_{10}^{2})^{-1/2}$.
Here $V_{1}=1$,
$V_{2}=-\frac{\sqrt{3}{J}^{2}\beta_{2}\left(180{B_{x}}^{2}+144{J}^{2}\right)-\sqrt{3}\beta_{2}^{3}\left(12{B_{x}}^{2}+33{J}^{2}\right)+\sqrt{3}\beta_{2}^{5}-162{J}^{5}}{162{J}^{4}B_{x}}$,
$V_{3}=\frac{\sqrt{3}{J}^{2}\beta_{2}\left(180{B_{x}}^{2}+198J^{2}\right)-\sqrt{3}\beta_{2}^{3}\left(12{B_{x}}^{2}+33{J}^{2}\right)+\sqrt{3}\beta_{2}^{5}+324{J}^{5}}{162{J}^{4}B_{x}}$,
$V_{6}=\frac{-\sqrt{3}{J}^{2}\beta_{2}(144{B_{x}}^{2}+81{J}^{2})-J\beta_{2}^{2}(36{B_{x}}^{2}+90{J}^{2})+\sqrt{3}\beta_{2}^{3}(12{B_{x}}^{2}+30{J}^{2})+3J\beta_{2}^{4}-\sqrt{3}\beta_{2}^{5}+243{J}^{5}+648{B_{x}}^{2}{J}^{3}}{216{B_{x}}^{2}{J}^{3}}$,
$V_{7}=-\frac{\sqrt{3}{J}^{2}\beta_{2}\left(144{B_{x}}^{2}+81{J}^{2}\right)-J\beta_{2}^{2}\left(36{B_{x}}^{2}+90{J}^{2}\right)-\sqrt{3}\beta_{2}^{3}\left(12{B_{x}}^{2}+30J^{2}\right)+3J\beta_{2}^{4}+\sqrt{3}\beta_{2}^{5}+243{J}^{5}+324{J}^{3}B_{x}^{2}}{108{B_{x}}^{2}{J}^{3}}$,
and
$V_{10}=-\frac{\sqrt{3}J^{2}\beta_{2}\left(144{B_{x}}^{2}-9{J}^{2}\right)+J\beta_{2}^{2}\left(108{B_{x}}^{2}+54{J}^{2}\right)+6\sqrt{3}\beta_{2}^{3}\left(2{B_{x}}^{2}+J^{2}\right)-9J\beta_{2}^{4}-\sqrt{3}\beta_{2}^{5}+648{B_{x}}^{2}{J}^{3}-81{J}^{5}}{648{B_{x}}^{2}{J}^{3}}$.
Since $\beta_{2}$ is real, all components of the ground state are also real.
From $R$-derivative of the normalization
($2C_{1}^{2}+4C_{2}^{2}+4C_{3}^{2}+2C_{6}^{2}+2C_{7}^{2}+2C_{10}^{2}=1$), we
see

$C_{1}\frac{\partial C_{1}}{\partial R}+2C_{2}\frac{\partial C_{2}}{\partial
R}+2C_{3}\frac{\partial C_{3}}{\partial R}+C_{6}\frac{\partial C_{6}}{\partial
R}+C_{7}\frac{\partial C_{7}}{\partial R}+C_{10}\frac{\partial
C_{10}}{\partial R}=0$ (44)

In case of the open linear 4 spin system in Fig.2(d), the symmetry
consideration allows 4 regularization terms which consist of
$\tilde{W}_{12}=\tilde{W}_{34}=\tilde{W}_{1},\tilde{W}_{23}=\tilde{W}_{2}$,
$\tilde{W}_{13}=\tilde{W}_{24}=\tilde{W}_{3}$, and
$\tilde{W}_{14}=\tilde{W}_{4}$. The regularization Hamiltonian
$\mathcal{\tilde{H}}$ is given in Eq.(51). To add one more unknown variable,
we include a contribution of the universal 3-body interaction
$\tilde{Q}\equiv\tilde{Q}_{ijk}^{xyz}$. This inclusion requires the same
improvement of some matrix elements of $\mathcal{\tilde{H}}$ as in Eq. (V.1).
The idea to include plural species of 3-body interactions results in
incompatible equations in Eq.(8) and can not be employed. Due to the symmetry
of $\\{C_{j}\\}$, the number of independent equations are six:

$\displaystyle(-2\tilde{W}_{1}-2\tilde{W}_{4}-2\tilde{W}_{3})C_{2}+(-2\tilde{W}_{2}-2\tilde{W}_{1}-2\tilde{W}_{3})C_{3}-8\tilde{Q}C_{6}-8\tilde{Q}C_{7}-8\tilde{Q}C_{10}$
$\displaystyle=$ $\displaystyle\hbar\frac{\partial C_{1}}{\partial R}$
$\displaystyle(\tilde{W}_{1}+\tilde{W}_{4}+\tilde{W}_{3})C_{1}+(-\tilde{W}_{2}+\tilde{W}_{1}-\tilde{W}_{3})C_{6}+(-\tilde{W}_{1}+\tilde{W}_{4}-\tilde{W}_{3})C_{7}+(-\tilde{W}_{1}-\tilde{W}_{2}+\tilde{W}_{3})C_{10}$
$\displaystyle=$ $\displaystyle\hbar\frac{\partial C_{2}}{\partial R}$
$\displaystyle(\tilde{W}_{1}+\tilde{W}_{2}+\tilde{W}_{3})C_{1}+(\tilde{W}_{1}-\tilde{W}_{4}-\tilde{W}_{3})C_{6}+(-\tilde{W}_{1}+\tilde{W}_{2}-\tilde{W}_{3})C_{7}+(-\tilde{W}_{1}-\tilde{W}_{4}+\tilde{W}_{3})C_{10}$
$\displaystyle=$ $\displaystyle\hbar\frac{\partial C_{3}}{\partial R}$
$\displaystyle
8\tilde{Q}C_{1}+(2\tilde{W}_{2}-2\tilde{W}_{1}+2\tilde{W}_{3})C_{2}+(-2\tilde{W}_{1}+2\tilde{W}_{4}+2\tilde{W}_{3})C_{3}$
$\displaystyle=$ $\displaystyle\hbar\frac{\partial C_{6}}{\partial R}$
$\displaystyle
8\tilde{Q}C_{1}+(2\tilde{W}_{1}-2\tilde{W}_{2}+2\tilde{W}_{3})C_{3}+(2\tilde{W}_{1}-2\tilde{W}_{4}+2\tilde{W}_{3})C_{2}$
$\displaystyle=$ $\displaystyle\hbar\frac{\partial C_{7}}{\partial R}$
$\displaystyle
8\tilde{Q}C_{1}+(2\tilde{W}_{1}+2\tilde{W}_{2}-2\tilde{W}_{3})C_{2}+(2\tilde{W}_{1}+2\tilde{W}_{4}-2\tilde{W}_{3})C_{3}$
$\displaystyle=$ $\displaystyle\hbar\frac{\partial C_{10}}{\partial R}.$ (45)

The constraint in Eq.(44) renders one of the above 6 equations trivial, and
Eq.(V.4), whose coefficient matrix has the rank 5, gives the following
solution:

$\displaystyle\tilde{W}_{1}$ $\displaystyle=$
$\displaystyle\hbar\frac{\partial_{R}(C_{7}+C_{10})}{4(C_{2}+C_{3})}+\kappa,$
$\displaystyle\tilde{W}_{2}$ $\displaystyle=$
$\displaystyle-\gamma_{1}+\gamma_{2},$ $\displaystyle\tilde{W}_{3}$
$\displaystyle=$
$\displaystyle\hbar\frac{\partial_{R}(C_{6}+C_{7})}{4(C_{2}+C_{3})}+\kappa,$
$\displaystyle\tilde{W}_{4}$ $\displaystyle=$
$\displaystyle\gamma_{1}+\gamma_{2},$ $\displaystyle\tilde{Q}$
$\displaystyle=$
$\displaystyle\frac{\hbar}{8(3C_{1}-C_{6}-C_{7}-C_{10})(C_{1}+C_{6}+C_{7}+C_{10})}$
$\displaystyle\times$
$\displaystyle\bigl{(}4(C_{2}-C_{3})\partial_{R}(C_{2}-C_{3})$
$\displaystyle+$
$\displaystyle(C_{1}+C_{6}+C_{7}+C_{10})\partial_{R}(C_{1}+C_{6}+C_{7}+C_{10})\bigr{)},$

with

$\displaystyle\kappa$ $\displaystyle\equiv$
$\displaystyle\frac{\hbar}{2(C_{2}+C_{3})(-3C_{1}+C_{6}+C_{7}+C_{10})}$
$\displaystyle\times$
$\displaystyle\bigl{(}C_{1}\partial_{R}C_{1}+(C_{2}-C_{3})\partial_{R}(C_{2}-C_{3})$
$\displaystyle+$ $\displaystyle
C_{1}\partial_{R}(C_{6}+C_{7}+C_{10})\bigr{)},$ $\displaystyle\gamma_{1}$
$\displaystyle\equiv$
$\displaystyle\hbar\frac{\partial_{R}(C_{2}-C_{3})}{2(C_{1}+C_{6}+C_{7}+C_{10})},$
$\displaystyle\gamma_{2}$ $\displaystyle\equiv$
$\displaystyle\hbar\frac{\partial_{R}(C_{6}+C_{10})}{4(C_{2}+C_{3})}+\kappa.$

The fast-forward Hamiltonian is given by Eq.(34), where
$v(t)\tilde{\mathcal{H}}(R(\Lambda(t)))$ is replaced by:

$\displaystyle v\mathcal{\tilde{H}}$ $\displaystyle=$
$\displaystyle\sum_{(i,j)=(1,2),(3,4)}v(t)\tilde{W}_{1}(R(\Lambda(t)))(\sigma_{i}^{y}\sigma_{j}^{z}+\sigma_{i}^{z}\sigma_{j}^{y})$
$\displaystyle+$
$\displaystyle\sum_{(i,j)=(2,3)}v(t)\tilde{W}_{2}(R(\Lambda(t)))(\sigma_{i}^{y}\sigma_{j}^{z}+\sigma_{i}^{z}\sigma_{j}^{y})$
$\displaystyle+$
$\displaystyle\sum_{(i,j)=(1,3),(2,4)}v(t)\tilde{W}_{3}(R(\Lambda(t)))(\sigma_{i}^{y}\sigma_{j}^{z}+\sigma_{i}^{z}\sigma_{j}^{y})$
$\displaystyle+$
$\displaystyle\sum_{(i,j)=(1,4)}v(t)\tilde{W}_{4}(R(\Lambda(t)))(\sigma_{i}^{y}\sigma_{j}^{z}+\sigma_{i}^{z}\sigma_{j}^{y})$
$\displaystyle+$ $\displaystyle\sum_{(i,j,k)\in
all}v(t)\tilde{Q}(R(\Lambda(t)))(\sigma_{i}^{x}\sigma_{j}^{y}+\sigma_{i}^{y}\sigma_{j}^{x})\cdot\sigma_{k}^{z}.$

Figures 5 (d) and 6 (d) show the time dependence of regularization terms and
that of wave function, respectively. The wave function starts from the ground
state with $J=0$, i.e., $C_{j}=\frac{1}{4}$ for $j=1,\cdots,16$. In Fig. 6 (d)
the solution $\Psi_{FF}(t)$ of TDSE in Eq.(14) has reproduced the time-
rescaled ground state wave function, which means the perfect fidelity during
the fast-forward time range $0\leq t\leq T_{FF}$.

In this Section, the number of independent equations to determine the pair-
wise interactions ($\tilde{W}_{i}$) is varied depending on the symmetry of
clusters. To make Eq.(8) solvable, however, these equations always require
only one extra unknown 3-body interaction, whose contribution to
$\mathcal{\tilde{H}}$ is commonly given in Eq.(V.1) for all spin clusters with
$N=4$ spins. Therefore the 3-body interaction ($\tilde{Q}$) here is geometry-
independent and played a subsidiary role.

##  VI Summary and discussions

The fast forward is the quasi-adiabatic dynamics guaranteed by regularization
terms added to the reference Hamiltonian, followed by a rescaling of time with
use of a large scaling factor. Assuming the regularization terms consisting of
pair-wise and 3-body interactions, we applied the core formula in Eq.(8) to
regular spin clusters with various geometries, e.g., regular triangle and open
linear chain for $N=3$ spin systems, and triangular pyramid, square, primary
star graph and open linear chain for $N=4$ spin systems. The geometry-induced
symmetry greatly decreases the rank of coefficient matrix of the linear
algebraic equation for regularization terms, namely, the rank is determined by
the geometric symmetry of the regular spin cluster. Choosing a transverse
Ising Hamiltonian as a reference, we find:
(1) for $N=3$ spin clusters, the driving interaction consists of only the
geometry-dependent pair-wise interactions and there is no need for the 3-body
interaction. The regular triangle and open linear 3 spins require,
respectively, one and 2 species of the pair-wise driving interactions; (2) for
$N=4$ spin clusters, the main part of the driving interaction again consists
of pair-wise interactions. The triangular pyramid and open linear 4 spins
require, respectively, one and 4 species of the pair-wise driving
interactions. On the other hand, two species of the pair-wise driving
interactions are necessary for the square and primary star graph. For $N=4$
spin clusters, besides these geometry-dependent pair-wise interactions, we
need a common geometry-independent 3-body interaction just to make the core
equation in Eq.(8) solvable. The 3-body interaction here plays a subsidiary
role. The geometric symmetry of regular spin clusters determines the number of
independent species of pair-wise driving interactions, and the clusters with
the highest symmetry have only one species of pair-wise driving interaction.
Our fast-forward scheme provides a flexible method in designing the practical
driving interaction in accelerating the adiabatic quantum dynamics of
structured regular spin clusters. The scheme may also be useful in our
inventing a variational method for treating much bigger regular clusters.

###### Acknowledgements.

We are grateful to S. Masuda for valuable discussions in the early stage of
the present work. The work is supported by Hibah Disertasi Doktor
Kemenristekdikti 2018. The work of B.E.G is supported by PUPT Ristekdikti-ITB
2017-2018.

##  Appendix A Regularization matrix $\mathcal{\tilde{H}}$ without
contributions due to the 3-body interaction

###  A.1 Square

The matrix for regularization term can be written as

$\mathcal{\tilde{H}}=i\left(\begin{array}[]{cccccccccccccccc}0&-A_{1}&-A_{1}&-A_{1}&-A_{1}&0&0&0&0&0&0&0&0&0&0&0\\\
A_{1}&0&0&0&0&-A_{2}&0&0&-A_{2}&-A_{3}&0&0&0&0&0&0\\\
A_{1}&0&0&0&0&-A_{2}&-A_{2}&0&0&0&-A_{3}&0&0&0&0&0\\\
A_{1}&0&0&0&0&0&-A_{2}&-A_{2}&0&-A_{3}&0&0&0&0&0&0\\\
A_{1}&0&0&0&0&0&0&-A_{2}&-A_{2}&0&-A_{3}&0&0&0&0&0\\\
0&A_{2}&A_{2}&0&0&0&0&0&0&0&0&0&0&A_{2}&A_{2}&0\\\
0&0&A_{2}&A_{2}&0&0&0&0&0&0&0&A_{2}&0&0&A_{2}&0\\\
0&0&0&A_{2}&A_{2}&0&0&0&0&0&0&A_{2}&A_{2}&0&0&0\\\
0&A_{2}&0&0&A_{2}&0&0&0&0&0&0&0&A_{2}&A_{2}&0&0\\\
0&A_{3}&0&A_{3}&0&0&0&0&0&0&0&0&A_{3}&0&A_{3}&0\\\
0&0&A_{3}&0&A_{3}&0&0&0&0&0&0&A_{3}&0&A_{3}&0&0\\\
0&0&0&0&0&0&-A_{2}&-A_{2}&0&0&-A_{3}&0&0&0&0&A_{1}\\\
0&0&0&0&0&0&0&-A_{2}&-A_{2}&-A_{3}&0&0&0&0&0&A_{1}\\\
0&0&0&0&0&-A_{2}&0&0&-A_{2}&0&-A_{3}&0&0&0&0&A_{1}\\\
0&0&0&0&0&-A_{2}&-A_{2}&0&0&-A_{3}&0&0&0&0&0&A_{1}\\\
0&0&0&0&0&0&0&0&0&0&0&-A_{1}&-A_{1}&-A_{1}&-A_{1}&0\end{array}\right)$ (49)

where $A_{1}$ = $2\tilde{W}_{1}+\tilde{W}_{2}$, $A_{2}$ = $\tilde{W}_{2}$,
$A_{3}$ = $2\tilde{W}_{1}-\tilde{W}_{2}$.

###  A.2 Primary star graph

The matrix for regularization term can be written as

$\mathcal{\tilde{H}}=i\left(\begin{array}[]{cccccccccccccccc}0&-A_{1}&-A_{1}&-A_{2}&-A_{1}&0&0&0&0&0&0&0&0&0&0&0\\\
A_{1}&0&0&0&0&-A_{3}&0&0&-A_{3}&-A_{3}&0&0&0&0&0&0\\\
A_{1}&0&0&0&0&-A_{3}&-A_{3}&0&0&0&-A_{3}&0&0&0&0&0\\\
A_{2}&0&0&0&0&0&A_{4}&A_{4}&0&A_{4}&0&0&0&0&0&0\\\
A_{1}&0&0&0&0&0&0&-A_{3}&-A_{3}&0&-A_{3}&0&0&0&0&0\\\
0&A_{3}&A_{3}&0&0&0&0&0&0&0&0&0&0&-A_{4}&A_{3}&0\\\
0&0&A_{3}&-A_{4}&0&0&0&0&0&0&0&A_{3}&0&0&A_{3}&0\\\
0&0&0&-A_{4}&A_{3}&0&0&0&0&0&0&A_{3}&A_{3}&0&0&0\\\
0&A_{3}&0&0&A_{3}&0&0&0&0&0&0&0&A_{3}&-A_{4}&0&0\\\
0&A_{3}&0&-A_{4}&0&0&0&0&0&0&0&0&A_{3}&0&A_{3}&0\\\
0&0&A_{3}&0&A_{3}&0&0&0&0&0&0&A_{3}&0&-A_{4}&0&0\\\
0&0&0&0&0&0&-A_{3}&-A_{3}&0&0&-A_{3}&0&0&0&0&A_{1}\\\
0&0&0&0&0&0&0&-A_{3}&-A_{3}&-A_{3}&0&0&0&0&0&A_{1}\\\
0&0&0&0&0&A_{4}&0&0&A_{4}&0&A_{4}&0&0&0&0&A_{2}\\\
0&0&0&0&0&-A_{3}&-A_{3}&0&0&-A_{3}&0&0&0&0&0&A_{1}\\\
0&0&0&0&0&0&0&0&0&0&0&-A_{1}&-A_{1}&-A_{2}&-A_{1}&0\end{array}\right)$ (50)

where $A_{1}$ = $2\tilde{W}_{2}+\tilde{W}_{1}$, $A_{2}$ = $3\tilde{W}_{1}$,
$A_{3}$ = $\tilde{W}_{1}$, $A_{4}$ = $\tilde{W}_{1}-2\tilde{W}_{2}$.

###  A.3 Open linear 4 spins

The matrix for regularization term can be written as

$\mathcal{\tilde{H}}=i\left(\begin{array}[]{cccccccccccccccc}0&-A_{1}&-A_{2}&-A_{2}&-A_{1}&0&0&0&0&0&0&0&0&0&0&0\\\
A_{1}&0&0&0&0&-A_{3}&0&0&-A_{4}&-A_{5}&0&0&0&0&0&0\\\
A_{2}&0&0&0&0&-A_{6}&-A_{7}&0&0&0&-A_{8}&0&0&0&0&0\\\
A_{2}&0&0&0&0&0&-A_{7}&-A_{6}&0&-A_{8}&0&0&0&0&0&0\\\
A_{1}&0&0&0&0&0&0&-A_{3}&-A_{4}&0&-A_{5}&0&0&0&0&0\\\
0&A_{3}&A_{6}&0&0&0&0&0&0&0&0&0&0&A_{6}&A_{3}&0\\\
0&0&A_{7}&A_{7}&0&0&0&0&0&0&0&A_{4}&0&0&A_{4}&0\\\
0&0&0&A_{6}&A_{3}&0&0&0&0&0&0&A_{3}&A_{6}&0&0&0\\\
0&A_{4}&0&0&A_{4}&0&0&0&0&0&0&0&A_{7}&A_{7}&0&0\\\
0&A_{5}&0&A_{8}&0&0&0&0&0&0&0&0&A_{8}&0&A_{5}&0\\\
0&0&A_{8}&0&A_{5}&0&0&0&0&0&0&A_{5}&0&A_{8}&0&0\\\
0&0&0&0&0&0&-A_{4}&-A_{3}&0&0&-A_{5}&0&0&0&0&A_{1}\\\
0&0&0&0&0&0&0&-A_{6}&-A_{7}&-A_{8}&0&0&0&0&0&A_{2}\\\
0&0&0&0&0&-A_{6}&0&0&-A_{7}&0&-A_{8}&0&0&0&0&A_{2}\\\
0&0&0&0&0&-A_{3}&-A_{4}&0&0&-A_{5}&0&0&0&0&0&A_{1}\\\
0&0&0&0&0&0&0&0&0&0&0&-A_{1}&-A_{2}&-A_{2}&-A_{1}&0\end{array}\right)$ (51)

where $A_{1}$ = $\tilde{W}_{1}+\tilde{W}_{3}+\tilde{W}_{4}$, $A_{2}$ =
$\tilde{W}_{1}+\tilde{W}_{2}+\tilde{W}_{3}$, $A_{3}$ =
$\tilde{W}_{2}-\tilde{W}_{1}+\tilde{W}_{3}$, $A_{4}$ =
$\tilde{W}_{1}-\tilde{W}_{4}+\tilde{W}_{3}$, $A_{5}$ =
$\tilde{W}_{1}+\tilde{W}_{2}-\tilde{W}_{3}$, $A_{6}$ =
$-\tilde{W}_{1}+\tilde{W}_{4}+\tilde{W}_{3}$, $A_{7}$ =
$\tilde{W}_{1}-\tilde{W}_{2}+\tilde{W}_{3}$, $A_{8}$ =
$\tilde{W}_{1}+\tilde{W}_{4}-\tilde{W}_{3}$.

## References

  * (1) H. R. Lewis and W. B. Riesenfeld, J. Math. Phys. A 10, 1458 (1969).
  * (2) X. Chen, A. Ruschhaupt, A. del Campo, D. Guery-Odelin and J. G. Muga, Phys. Rev. Lett. 104, 063002 (2010).
  * (3) E. Torrontegui, M. Ibanez, M. Martinez-Garaot, M. Modugmo, A. del Campo, D. Guery-Odelin, A. Ruschhaupt, X. Chen and J. G. Muga, Adv.At. Mol. Opt. Phys 62, 117 (2013).
  * (4) M. Demirplak and S. A. Rice, J. Phys. Chem. A 89, 9937 (2003).
  * (5) M. Demirplak and S. A. Rice, J. Phys. Chem. B 109, 6838 (2005).
  * (6) M. V. Berry, J. Phys. A 42, 365303 (2009).
  * (7) S. Masuda and K. Nakamura, Phys. Rev. A 78, 062108 (2008).
  * (8) S. Masuda and K. Nakamura, Proc. R. Soc A 466, 1135 (2010).
  * (9) S. Masuda and K. Nakamura, Phys. Rev. A 84, 043434 (2011).
  * (10) T. Opatrnỳ and K. Mølmer, New. J Phys. 16, 015025 (2014).
  * (11) D. Sels and A. Polkovnikov, Proc. Natl. Acad. Sci. 114, 3909 (2017).
  * (12) A. Hartmann and W. Lechner, New J. Phys. 21, 043025 (2019).
  * (13) K Nakamura, A. Khujakulov, S. Avazbaev and S. Masuda, Phys. Rev. A 95, 062108 (2017).
  * (14) G. Babajanova, J. Matrasulov and K. Nakamura, Phys. Rev. E 97, 042104 (2018).
  * (15) A. Patra and C. Jarzynski, New J. Phys. 19, 125009 (2017).
  * (16) I. Setiawan, B. E. Gunara, S. Masuda and K. Nakamura, Phys. Rev. A 96, 052106 (2017).
  * (17) X. Chen, E. Torrontegui, and J. G. Muga, Phys. Rev. A 83, 062116 (2011).
  * (18) K. Paul and A.K. Sarma, Phys. Rev. A 94, 052303 (2016).
  * (19) D. Stefanatos and E. Paspalakis, Phys. Rev. A 99, 022327 (2019).
  * (20) K. Takahashi, Phys. Rev. A 89, 042113 (2014).
  * (21) K. Takahashi, J. Phys. Soc. Jpn. 88, 061002 (2019).
  * (22) A. del Campo, M.M. Rams and W.H. Zurek, Phys. Rev. Lett. 109, 115703 (2012).
  * (23) H. Saberi, T. Opatrnỳ, K. Mølmer and A del Campo, Phys. Rev. A 90, 060301(R) (2014).
  * (24) B. Damski, J. Stat. Mech. P12109 (2014).
  * (25) V. Mukherjee, S. Montangero and R. Fazio, Phys. Rev. A 93, 062108 (2016).
  * (26) S. Campbell, G. De Chiara, M. Paternostro, G.M. Palma, and R. Fazio, Phys. Rev. Lett. 114, 177206 (2015).