# Transferring Multiple Policies to Hotstart Reinforcement Learning in an Air
Compressor Management Problem

Hélène Plisnier<EMAIL_ADDRESS>
Vrije Universiteit Brussel Denis Steckelmacher<EMAIL_ADDRESS>
Vrije Universiteit Brussel Jeroen Willems<EMAIL_ADDRESS>
FlandersMake Bruno Depraetere<EMAIL_ADDRESS>
FlandersMake Ann Nowé<EMAIL_ADDRESS>
Vrije Universiteit Brussel

###### Abstract

Many instances of similar or almost-identical industrial machines or tools are
often deployed at once, or in quick succession. For instance, a particular
model of air compressor may be installed at hundreds of customers. Because
these tools perform distinct but highly similar tasks, it is interesting to be
able to quickly produce a high-quality controller for machine $N+1$ given the
controllers already produced for machines $1..N$. This is even more important
when the controllers are learned through Reinforcement Learning, as training
takes time, energy and other resources. In this paper, we apply Policy
Intersection, a Policy Shaping method, to help a Reinforcement Learning agent
learn to solve a new variant of a compressors control problem faster, by
transferring knowledge from several previously learned controllers. We show
that our approach outperforms loading an old controller, and significantly
improves performance in the long run.

##  1 Introduction

A Reinforcement Learning algorithm takes an input, predicts an output, and
then receives a scalar grading, or _reward_ , indicating how well it did; the
objective of the Reinforcement learner is to maximize the rewards it gets.
Reinforcement Learning (RL) is attractive when the solution to a problem is
unknown or too difficult to implement by hand, but that it is possible to
determine whether a solution is optimal or not. It has already been proven
useful in a wide range of applications, such as industrial and healthcare
problems: order dispatching Kuhnle et al. (2019), datacenter cooling Lazic et
al. (2018), personalized pharmacological anemia management Gaweda et al.
(2005), etc. In this paper, we apply Soft Actor-Critic Haarnoja et al. (2018)
(SAC), a well-known RL algorithm, to learn a controller in an air compressor
management problem, which setup is detailed in Figure 1. Three piston
compressors are expelling air in a tank; air is going out through a valve
which progressively opens or closes following a demand curve. The compressors
must provide a satisfactory pressure in between 3 and 5 bars, while conserving
energy. The SAC Reinforcement Learning agent learns to control the three
motors, each activating one of the piston compressor, while observing the
current speed of the motors and the current pressure; the demand curve is
unknown to the agent.

In addition, we consider the following problem: given that several controllers
have already been learned, each in a different variant of the air compressor
management problem, can a new controller be learned faster by exploiting
already learned controllers? Variants of the air compressor management problem
are generated by varying the volume of the tank, and the compressing power of
the three compressors. To intelligently reuse the knowledge acquired by the
previously learned controllers, Transfer Reinforcement Learning techniques can
be used.

Transfer Learning Taylor & Stone (2009) has the potential to make RL agents
much faster at mastering new tasks, by allowing the reuse of knowledge
acquired in previous tasks. A Transfer Learning setting often involves a
source task and a target task; first, the agent learns the source task, then
the knowledge acquired in the source task is intelligently leveraged by the
agent while tackling the target task, using a Transfer Learning algorithm
Taylor & Stone (2009); Zhu et al. (2020). Transferred knowledge can be the
learned controller of the agent Taylor et al. (2007); Fernández & Veloso
(2006); Brys (2016); learned skills Andre & Russell (2002); Ravindran & Barto
(2003); Konidaris & Barto (2007), some general enough skill (like walking) to
fit a large set of target tasks Tang & Haarnoja (2017); parts of a modular
neural network in which each module deals with a different aspect of the task
Devin et al. (2017); Mirowski et al. (2018). Methods to effectively transfer
knowledge include reward shaping Brys (2016), controller reuse methods, which
shape the agent’s exploration strategy Fernández & Veloso (2006); Griffith et
al. (2013), initializing a controller Taylor et al. (2007) using information
from the source task, and initializing parts of the target network with the
source network Devin et al. (2017); Mirowski et al. (2018); Chaplot et al.
(2016), to only cite a few.

The Transfer Learning methods we evaluate in this paper is Policy Intersection
(which we review in Section 2.3), a Policy Shaping method using the previously
learned controller to guide the exploration of the agent while it learns the
new controller García & Fernández (2015). We apply Policy Intersection to
effectively distill multiple transferred controllers in the learning process
of a SAC agent on a new variant of the air compressor management problem. We
empirically show that Policy Intersection outperforms simply loading a
transferred controller in the new task, be it with one or multiple transferred
controllers, and that leveraging multiple controllers performs significantly
better than leveraging only one transferred controller.

Figure 1: Description of the setup. Three piston air compressors are filling a
tank of varying size. The valve progressively opens or closes to simulate a
demand.

##  2 Background

In this section, we introduce the Reinforcement Learning framework, the Soft
Actor-Critic algorithm and Policy Intersection, which we use in our
experiments.

###  2.1 Reinforcement Learning

Reinforcement Learning techniques learn a controller that observes states of
an environment, executes actions and receives rewards from the environment. An
environment can be formulated using a Markov Decision Process (MDP) defined by
the tuple $\langle S,A,R,T\rangle$ Bellman (1957), with the time discretized
in time-steps. At each time-step $t$, the agent takes an action $a_{t}\in A$
based on the current state $s_{t}\in S$, receives a new state from the
environment $s_{t+1}\in S$ and a scalar reward $r_{t}$, returned by a reward
function $R(s_{t},a_{t},s_{t+1})\in\mathcal{R}$. For each state transition, a
transition function $T(s_{t+1}|s_{t},a_{t})\in[0,1]$ taking as input a state-
action pair $(s_{t},a_{t})$ returns a probability distribution over new states
$s_{t+1}$; $T$ is generally unknown to the agent. Both the actions set $A$ and
states set $S$ can be finite or infinite.

The amount of rewards received in the long run is formally defined as the
discounted return $\mathcal{R}_{t}=\sum_{t}\gamma^{t}r_{t}$, where
$\gamma\in[0,1]$ is called the discount factor. The closer $\gamma$ is to 1,
the more the agent will prefer long-term rewards to short-terms ones. At
acting time, the agent select an action based on its controller, or policy
$\pi$, which maps a state to a probability distribution over actions, with
$\pi(a_{t}|s_{t})\in[0,1]$ denoting the probability of taking action $a_{t}$
given state $s_{t}$. After having executed action $a_{t}$ in state $s_{t}$,
the environment returns a reward $r_{t}$, and the newt state $s_{t+1}$; the
agent stores the data sample, or _experience tuple_
$(s_{t},a_{t},r_{t},s_{t+1})$ in a buffer. This transaction between the agent
and the environment, to go from state $s_{t}$ to $s_{t+1}$, occurs during one
_time-step_. At learning time, the agent updates its policy based on the
experience tuples collected during acting time. The objective of an RL
algorithm is to progressively modify $\pi$ until the optimal policy $\pi^{*}$
is found, which maximizes $E_{\pi^{*}}[\mathcal{R}_{t}]$, the expected
discounted return (Sutton et al., 2000, ch. 3).

We focus on episodic tasks, in which time is broken down into episodes; in the
problem we tackle in this paper, an episode ends and a new one starts once a
fixed amount of time-steps have passed. At the beginning of each new episode,
the agent is put back in an initial state, and must re-perform the task from
the beginning. The performance of an RL agent, and ultimately of the algorithm
it executes, is evaluated mainly through looking at its learning curve. A
learning curve shows, for each episode, the sum of all rewards collected
during the episode. The higher the curve goes, the better the policy learned
is. In addition, an experiment often consists in several _runs_ of an RL
algorithm learning a task from beginning to end, for a number of episodes.

###  2.2 Soft Actor-Critic

Soft Actor-Critic (SAC) is an off-policy actor-critic Reinforcement Learning
algorithm, originally tailored for environments with continuous actions
Haarnoja et al. (2018). A way to categorize an RL algorithm is by whether it
consists in an on-policy or off-policy method: on-policy algorithms update the
policy from which actions are executed in the environment, generating data
samples; off-policy algorithms update another policy, different from the one
generating the samples Sutton & Barto (2018). In contrast to on-policy
algorithms, off-policy algorithms tend to be more efficient at pursuing the
optimal policy $\pi^{*}$, and are less prone to fall in local optima.
Furthermore, Reinforcement Learning algorithms either learn a $Q$ (_quality_)
function, or a critic, from which a controller is indirectly derived, a policy
$\pi$, or actor, which directly acts as a controller, or both. SAC is of the
latter category.

SAC is based on the principle of maximum entropy: the agent maximises
$E_{\pi}[\mathcal{R}_{t}+\alpha\mathcal{H}(\pi)]$, hence not only the expected
discounted return, but also the entropy of its policy. This approach results
in a policy acting as randomly as possible all throughout learning, which
allows the agent to intensively explore its environment. Continuous actions
are challenging for Reinforcement Learning, and not every algorithm is
compatible with them. For instance, almost all RL algorithms for continuous
actions need an explicit actor: there are very few critic-only algorithms for
continuous actions Engel et al. (2005); Antos et al. (2007). In SAC, $\pi$ is
implemented as a Gaussian distribution, parameterized by a mean $\mu$ and a
standard deviation $\sigma$, these two parameters being output by a neural
network given a state $s_{t}$. At each time-step, an action $a_{t}$ is sampled
from the Gaussian policy with mean $\mu$ and standard deviation $\sigma$.

In the next section, we review how a learned policy $\pi$ can be directly
influenced by an external advisory policy to guide its exploration.

###  2.3 Policy Shaping

Policy Shaping lets an external advisory policy $\pi_{A}$ alter or determine
the policy of the agent at acting time, either sporadically or continuously
throughout learning. This general approach, shared with the Safe RL and the
learning from human interactions RL subfields García & Fernández (2015);
Alshiekh et al. (2018); Griffith et al. (2013), can easily be used for
Transfer Learning purposes. In this section, we review Policy Intersection
(Griffith et al., 2013; Cederborg et al., 2015, originally called “Policy
Shaping"), a method to shape an RL agent’s policy. The first work we know of
that introduced the term “Policy Shaping” is Griffith et al. (2013). However,
the method presented (which we detail below) is very specific, while the term
Policy Shaping could apply to a wide variety of methods MacGlashan et al.
(2017); Harutyunyan et al. (2014). Since Griffith et al. (2013) introduces an
element-wise multiplication between discrete probability distributions, and
that this operation represents taking the intersection between the
distributions, we rebaptized Griffith’s formula Policy Intersection in this
paper for clarity. Similarly, a Policy Union algorithm can be achieved by
summing the probability distributions, although the qualification and
evaluation of this approach is outside the scope of this paper.

####  2.3.1 Policy Intersection

This second approach at Policy Shaping multiplies the $\pi_{L}$ and $\pi_{A}$
vectors together then normalizes the resulting policy vector at each timestep:

$a_{t}\sim\pi_{L}\times\pi_{A}=\frac{\overbrace{\pi_{L}(s_{t})\,\pi_{A}(s_{t})}^{\text{element-
wise product}}}{\underbrace{\pi_{L}(s_{t})\cdot\pi_{A}(s_{t})}_{\sum_{a\in
A}\pi_{L}(a|s_{t})\pi_{A}(a|s_{t})}}$ (1)

where $\pi_{L}(s_{t})\cdot\,\pi_{A}(s_{t})$ is the dot product of the two
policies. At acting time, the agent samples an action $a_{t}$ from the mixture
$\pi_{L}\times\pi_{A}$ of the agent’s current learned policy $\pi_{L}(s_{t})$
and the external advisory policy $\pi_{A}(s_{t})$, instead of sampling only
from $\pi_{L}(s_{t})$. This product of probability distribution vectors
amounts to taking the intersection between what the current agent’s policy
wants to do, and what the transferred policy would do in that state. As a
result, although this formula was first introduced by Griffith et al. (2013)
as “Policy Shaping", we re-name it Policy Intersection in this paper for
clarity. Policy Intersection allows $\pi_{L}$ and $\pi_{A}$ to more
cooperatively select actions, with $\pi_{A}$ able to increase or decrease the
probability of actions, but without fully determining the action. However,
Equation 1 can only be applied to tasks for which there is a finite set of
actions, excluding environments with a continuous action space.

The Actor-Advisor Plisnier et al. (2019) is the first attempt made at using
the Policy Intersection formula in Equation 1 to achieve Transfer Learning.
Their main contribution is to directly influence a Policy Gradient agent’s
policy with some off-policy external advice $\pi_{A}$ without convergence
issue. However, that approach is also limited to discrete actions.

##  3 Related Work

Transferring knowledge in Reinforcement Learning potentially improves sample-
efficiency, as it allows an agent to exploit relevant past knowledge while
tackling a new task, instead of learning the new task from scratch (Taylor &
Stone, 2009). Usually, we consider that the valuable knowledge to be
transferred in Reinforcement Learning is the actual output of a reinforcement
learner: a $Q$ function or a policy $\pi$ (Brys, 2016, p.34). Some work also
consider reusing skills, or _options_ (Sutton et al., 1999), as a transfer of
knowledge across tasks (Andre & Russell, 2002; Ravindran & Barto, 2003;
Konidaris & Barto, 2007). We focus on Transfer Learning methods directly
reusing the policy learned in the source task. In this section, we sort
previous work in categories related to the _way_ the source policy is
transferred into the agent, and look at what is allowed to be different
between the source task and the target task. The two predominant ways in which
knowledge can be transferred are i) the source policy serves as a guide during
exploration, ii) the source policy is used to train or initialize the agent,
so that the agent actively imitates it.

###  3.1 Learning

Some techniques have proposed to let the source policy “teach” the agent to
perform the target task, either by dynamic teaching, or by straightforward
initialization. Imitation learning aims at allowing a student agent to learn
the policy of a demonstrator, out of data that it has generated Hussein et al.
(2017). Similarly, policy distillation Bucila et al. (2006) can be applied to
RL to train a fresh agent with one or several expert policies, hence resulting
in one, smaller, potentially multi-task RL agent Rusu et al. (2015).

Imitation learning and policy distillation are somewhat related to transfer
learning (Hussein et al., 2017, p.24), although imitation and distillation
assume that the source and target tasks are the same, while transfer does not.
The Actor-Mimic (Parisotto et al., 2015) uses several DQN policies (each
expert in a different source task) to train a multi-task student network, by
minimizing the cross-entropy loss between the student and experts’ policies.
To perform transfer, the resulting multi-task expert policy is used to
initialize yet another DQN network, which learns the target task. The Actor-
Mimic assumes that the source and target tasks share the same observation and
action spaces, with different reward and transition functions. In _Q-value
reuse_ (Taylor et al., 2007, Section 5.5), a Q-Learner uses $Q_{source}$ to
kickstarts its learning of the target task, while also learning a new action-
value function $Q_{target}$ to compensate $Q_{source}$’s irrelevant knowledge.
In Taylor et al. (2007), the agent learns to play Keepaway games, and is
introduced to a game with more players, resulting in more actions and state
variables. Brys et al. (2015) transfers the source policy to a Q-Learning
agent through reward shaping; the differences in the action and state spaces
between source and target tasks are solved using a provided translation
function.

Finally, Warm Start Reinforcement Learning (WSRL) Cheng et al. (2018); Zhu &
Liao (2017) aims at initializing the policy of the agent with another policy
pre-trained on the same task. Domain knowledge, i.e., information about the
environment known by the designers but not initially known by the agent, can
be used to kickstart learning, either through imitation learning on expert
demonstrations Cheng et al. (2018), directly encoding it via propositional
rules in the neural network architecture of the agent Silva & Gombolay (2021),
or actively learning to imitate a transferred policy Wexler et al. (2022).

Policy Intersection Griffith et al. (2013), the method we evaluate in this
paper, belongs to the category of Transfer Learning methods reviewed in the
next section.

###  3.2 Exploration

Policy Shaping Griffith et al. (2013), also referred to as guiding the
exploration strategy of the agent, transfers a policy in a fast and effective
way. Altering the exploration strategy is a popular technique in the safe RL
domain, and consists in biasing or determining the actions taken by the agent
at action selection time García & Fernández (2015). Such exploration requires
the agent to be able to learn from _off-policy_ experiences, as it is the case
with Soft Actor-Critic. Some existing work applies guided exploration to
Transfer Learning Fernández & Veloso (2006); Taylor & Stone (2007); Madden &
Howley (2004), and illustrates how this technique allows the agent to
outperform the performance of the source policy. Regarding the components of
the source and target tasks that are allowed to differ, Fernández & Veloso
(2006) considers different goal placements (hence, different reward
functions), Madden & Howley (2004) uses symbolically learned knowledge to
tackle states that are seen by the agent for the first time, and Taylor &
Stone (2007) assumes similar state variables and actions, but a different
reward function. In our setting, the environmental dynamics from the source
task to the target task are completely disturbed and lead to wildly different
outcomes following the actions of the agent, while the goal remains the same
across tasks.

As our air compressor management problem can be translated into a
Reinforcement Learning problem with continuous actions, in the next section,
we propose a solution to extend Policy Intersection to continuous action
spaces.

##  4 Policy Intersection for Continuous Actions

Equation 1 expresses the product of two state-dependent probability
distributions: the actor’s $\pi_{L}(s_{t})$, and the advisor’s
$\pi_{A}(s_{t})$. In the discrete actions case, $\pi_{L}(s_{t})$ and
$\pi_{A}(s_{t})$ are both vectors of size $|A|$, where $A$ is a finite set of
actions available to the agent. Multiplying those two vectors amounts to
taking the intersection between those two probability distributions. In other
words, Policy Intersection samples actions that both the actor and the advisor
“agree on”, as these actions have a non-zero probability in both
$\pi_{L}(s_{t})$ and $\pi_{A}(s_{t})$ 111Note that if no agreement can be
made, i.e., the intersection between which action the actor and the advisor
want to choose is empty, it can be arbitrarily decided that either the advisor
or the actor has the last word, depending on the problem.. Unfortunately, when
the action space is continuous, finding actions in the intersection between
the actor’s and the advisor’s policies is less straightforward.

Algorithm 1 Policy Intersection for Continuous Actions

$\pi_{L}$ is the currently learning actor, and $\pi_{A}$ is the frozen actor
used as advisor

$A^{L}=$ set of actions sampled from $\pi_{L}(s_{t})$

$\forall a^{L}_{i}\in A^{L}:$ get probability density
$\pi_{A}(a^{L}_{i}|s_{t})$

Compute the Categorical Distribution $f(\pi_{A}(a^{L}_{i}|s_{t}))$

if $a^{L}_{i}\sim f(\pi_{A})\neq None$ then

Execute action $a^{L}_{i}$

end if

Execute action $a\sim\pi_{L}(s_{t})$

In Reinforcement Learning algorithms that can be applied to environments with
continuous actions, such as SAC, individual actions can directly be sampled
from the actor. Unfortunately, it is difficult to access a probability
distribution over all possible actions, as the number of possible actions is
infinite. However, the actor can provide, given a particular action, the
probability density of that action. Assuming that the advisor can similarly be
sampled, our approach to sampling the intersection between which actions are
allowed by the actor and those allowed by the advisor is the following:

  1. 1.

If we use multiple advisors, one advisor is randomly picked from the set of
advisors at each time-step.

  2. 2.

At action selection time, we sample a large amount of actions $A^{L}$ from the
learned policy, or actor (where the superscript “$L$” stands for _learned_).

  3. 3.

We then submit $A^{L}$ to the advisor $\pi_{A}$, which returns probability
densities $\pi_{A}(s_{t})$. For each action $a^{L}_{i}\in A^{L}$, we now have
a corresponding density $\pi_{A}(a^{L}_{i}|s_{t})$.

  4. 4.

We compute a Categorical distribution $f(\pi_{A}(a^{L}_{i}|s_{t}))$ over the
discrete set of actions $A^{L}$, parameterized by the probability densities
$\pi_{A}(a^{L}_{i}|s_{t})$.

  5. 5.

We sample an action $a^{L}_{i}$ from $f(\pi_{A}(a^{L}_{i}|s_{t}))$. If no
action can be sampled, then the policy of the actor $\pi_{L}$ is sampled.

This method implements an “AND” between the actor and the advisor’s policies.
While our sampling technique allows to compute the intersection between an
actor and an advisor, with no assumption about their distributions (we do not
assume that they are Gaussian, for instance), we acknowledge that alternative
approaches could have been devised, such as creating a new stochastic policy
resulting of the point-wise multiplication of the advisor’s probability
density function with the actor’s probability density function, sampled for a
large number of actions.

##  5 Reinforcement Learning Environment

A piston compressor is a type of industrial compressor, which increases the
pressure of air enclosed in a cylinder. Air enters the cylinder through a
valve, then is expelled under pressure by a piston operated by an eternal
motor, through a second valve. In our particular setting, three compressors,
each operated by its own motor, supply air to a tank. The air flows through a
valve which is progressively opened or closed to simulate a demand. The demand
curve changes at each episode. Episodes last for 250 time-steps; at each time-
step, the agent chooses the rotary speed of each of the three motors operating
the compressors. The action space is continuous, and ranges from 400
revolutions per minute (RPM) to 600 RPM, with the possibility to completely
stop the motor if the agent chooses to. When restarted, the motor goes from 0
directly to 400 RPM. At each time-step, the agent receives an immediate reward
consisting in the negative amount of energy (in joules) that was consumed to
power the motors during the time-step. The agent must keep the air under a
pressure in between 3 and 5 bars within the system; going above 5 bars does
not bring any added value; going under 3 bars triggers a backup policy. The
backup policy is a naive controller that sets all three motors to their
maximum speed for one time-step, which results in a very negative reward for
the agent whenever the backup policy takes over. The agent observes the
current speed of the motors, as well as the current pressure in bars, however,
it does not know the demand curve. Although the reward is dense and highly
informative, this setting consists in a challenging environment with
continuous actions and partially observation states.

We generate several variants of this setting by randomly altering the size of
the tank, as well as the compressors. The parameters determining the amount of
air coming out of a compressor given $x$ RPM are defined at the beginning of
an experiment, using the following formula:
$\rho\times(1.0+\lambda\times\zeta\sim\mathcal{R}\in[0,1))$, where $\rho$ is a
compressor parameter, $\lambda\in[0,1]$ is the percentage of alterations
applied to $\rho$, and $\zeta$ is a random float yielded by a generator
simulating a uniform distribution. The tank volume is defined as $50+10\zeta$,
in liters. The random float generator $\mathcal{R}$ takes an integer called
_seed_ $s$ as parameter; we produce multiple variants of the environments to
transfer across by varying the value of $s$ (from 1 to 17).

This simulated setup is modelled as an MDP as follows.

###  5.1 Action space

The action space is continuous and consists of 3 real values, ranging from -1
to 1. It is considered best practice [stable-baselines] to have the action and
state spaces be centered around 0, and of a range of as close to 1 as
possible. Each of the 3 real values allows to set the target RPM of its
corresponding compressor, linearly interpolated between 0 (off, for an action
value of -1) to 100% (for an action value of 1).

The action space is discontinuous in two places: for every compressor, the
actual effect of the action depends on the target RPM value it defines:

  * •

Below 20 RPM: the compressor is turned off completely

  * •

Between 20 RPM and the minimum RPM of the compressor: the target RPM is
adjusted to the minimum RPM of the compressor

  * •

Above the minimum RPM of the compressor: the action is left untouched

The effect is that the agent can produce low actions to indicate that a
compressor should work at its lowest RPM, and an even lower action to indicate
that the compressor should be turned off. This allows the agent to control the
on/off status of the compressors, in addition to their target speed, with a
single real value.

###  5.2 Observation space

The observation space consists of several real values that measure past tank
pressures and RPMs. For every time-step, 4 real values are logged: the current
pressure (in bars) in the tank, and the current RPMs of the 3 compressors.
When producing observations, the environment looks $N$ time-steps in the past
to produce $N$ real values corresponding to the tank pressures at these past
$N$ time-steps (so, this is a history of past tank pressures). The environment
also looks $M$ time-steps in the past to produce $3M$ real values
corresponding to RPMs of the compressors during these past $M$ time-steps. $N$
and $M$ can be distinct, and in our experiment, we use $N=5$ past tank
pressures and $M=3$ past RPMs.

By observing past tank pressures and RPMs, the agent gets a feel of:

  * •

How fast is the tank depleting, which allows it to approximate the derivative
of the tank pressure (that only looks at the past), thereby adjusting the
future RPMs.

  * •

What is the demand of air in the recent past (by combining how the tank
pressure changes and what the RPMs were), which may help the agent guess what
the demand will be in the future if there are time-specific patterns in the
demand.

This information is still not enough for optimal control, as it does not
contain information about the future demand, but observing a history of past
sensor readings has been shown to be one of the best approaches to learn in
Partially Observable MDPs [citation], and is easy to implement.

###  5.3 Reward function

The reward function is the change in cost that occurs after a given time-step
executes (so, after 60 seconds of simulated time after an action has been
applied to the physical or simulated system). More specifically, the reward
given after a time-step consists of two components:

  1. 1.

Minus the amount of kilojoules consumed during the time-step. In the simulated
setup, the power consumption of the compressors is approximated in a lookup
table (in watts). In simulation, we assume that a compressor instantly reaches
its target RPM and its corresponding power consumption, so the amount of
kilojoules is $60s\times P\times 0.001$, with P the power (in watts) obtained
from the lookup table.

  2. 2.

A turn on penalty when a compressor turns on. In addition to the point above
(the power consumption during the whole time-step), a compressor that goes
from off to on at the start of a time-step is considered to incur a cost equal
to its maximum (max RPM) power consumption during 60 seconds. For instance, if
the third compressor powers on, a reward of $-60s\times 2000W\times
0.001=-120kJ$ is given to the agent.

##  6 Experiments

We set $\lambda$, the percentage of alteration applied to the parameters of
the compressors, to 0.5. We train 16 different advisory policies, each trained
on the environment with a different seed $s\in[1,...16]$. We consider two
settings in which Policy Intersection for continuous actions can be used: i)
(_single_) a single advisor, with a different seed than the advisee, is
exploited, ii) (_multiple_) all available advisors are exploited, except the
one with the same seed as the advisee. To leverage multiple advisors, at each
time-step, one advisor is randomly picked from the set of advisors. We compare
transferring advisory policies using Policy Intersection to simply loading an
advisor of a different seed and letting it resume learning.

Figure 2: Comparison between transferring from 16 advisors, 1 advisor, loading
one advisor at a time, or not using any transfer at all. Using Policy
Intersection, either with a single or multiple advisors, outperforms loading
an advisor and letting it resume learning. However, there is a significant dip
in the learning curve shortly after learning begins when Policy Intersection
is used.

At first glance, in both Figure 2 and Figure 3, a significant dip can be
noticed in the learning curve, occurring only a few episodes after the
beginning of learning. Moreover, this dip only occurs when using Policy
Intersection; it is not present when loading an advisor. This decrease in
performance is a property of off-policy algorithms, such as Soft Actor-Critic,
when their learning is kickstarted by an already pre-trained policy, as in
this experiment. At the beginning of learning, the policy of the advisee
$\pi_{L}$ is random, while the advisor $\pi_{A}$ is not, and shows actions to
the advisee that are generating good rewards. At this point, the critic of
Soft Actor-Critic believe that all actions are uniformly good, since the
advisee has only seen good actions, thanks to the advisor’s guidance. Due to
the inner randomness of neural networks, the policy of the advisee then
arbitrarily concentrates its probability on one action, which is not
necessarily good, producing the dip in the curve. It takes some exploration
time for the advisee to realize the suboptimality of that action, and to learn
to choose better ones, hence the recovering occurring a few episodes later. We
do not propose a technique to mitigate this particular problem in this paper,
but solutions have been investigated Wexler et al. (2022).

Policy Intersection, leveraging either a single or multiple advisors,
outperforms loading an advisor and letting it resume learning. In addition, we
observe in Figure 2 that exploiting multiple advisors instead of a single one
significantly improve performance, especially when recovering from the dip in
performance. This phenomenon is also noticeable in Figure 3, in which we
compare leveraging 1 advisor, 4 advisors, and 16 advisors. However, there is a
performance ceiling, reached by all settings in the long run, that does not
seem possible to exceed by increasing the amount of advisors.

Figure 3: There is a significant difference in performance between using 4 and
16 advisors, especially when recovering from the dip in the curve. This
difference lessens in the long run.

##  7 Conclusion

We applied Soft Actor-Critic to learn to control three motors, each activating
a piston compressor in an air compressor management problem. In this problem,
air is expelled by the piston compressors in a tank, while air is going out
through a valve which progressively opens or closes following a demand curve.
The goal of the compressors is to provide a satisfactory pressure in between 3
and 5 bars, while conserving energy. Moreover, we considered the problem of
learning a new controller based on previously learned ones on different
variants of the air compressor management problem. To tackle this Transfer
Reinforcement Learning problem, we used Policy Intersection, a Policy Shaping
method, allowing one or multiple already acquired policies to distill their
knowledge in a new policy, through guiding the exploration strategy of the
agent. We empirically showed that this approach outperforms simply loading a
transferred controller, and that exploiting multiple transferred controllers
instead of a single one can significantly improve performance. In future work,
we will focus on finding and comparing solutions to mitigate the degradation
of performance occurring a few episodes after the beginning of learning, a
phenomenon known as _Warm Start Reinforcement Learning Degradation_ in Wexler
et al. (2022).

## Acknowledgment

The first author is funded by the Science Foundation of Flanders (FWO,
Belgium) as 1SA6619N Applied Researcher.

## References

  * Alshiekh et al. (2018) Mohammed Alshiekh, Roderick Bloem, Rüdiger Ehlers, Bettina Könighofer, Scott Niekum, and Ufuk Topcu.  Safe reinforcement learning via shielding.  In _Thirty-Second AAAI Conference on Artificial Intelligence_ , 2018\.
  * Andre & Russell (2002) David Andre and Stuart J Russell.  State abstraction for programmable reinforcement learning agents.  In _AAAI/IAAI_ , pp. 119–125, 2002.
  * Antos et al. (2007) András Antos, Csaba Szepesvári, and Rémi Munos.  Fitted q-iteration in continuous action-space mdps.  _Advances in neural information processing systems_ , 20, 2007.
  * Bellman (1957) Richard Bellman.  A Markovian decision process.  _Journal Of Mathematics And Mechanics_ , 6:679–684, 1957\.
  * Brys (2016) Tim Brys.  _Reinforcement Learning with Heuristic Information_.  PhD thesis, PhD thesis, Vrije Universitet Brussel, 2016.
  * Brys et al. (2015) Tim Brys, Anna Harutyunyan, Halit Bener Suay, Sonia Chernova, Matthew E. Taylor, and Ann Nowé.  Reinforcement learning from demonstration through shaping.  In _Proceedings of the 24th International Conference on Artificial Intelligence_ , IJCAI’15, pp. 3352–3358. AAAI Press, 2015.  ISBN 978-1-57735-738-4.  URL http://dl.acm.org/citation.cfm?id=2832581.2832716.
  * Bucila et al. (2006) C Bucila, R Caruana, and A Niculescu-Mizil.  Model compression: Making big, slow models practical.  In _International Conference on Knowledge Discovery and Data Mining_ , 2006.
  * Cederborg et al. (2015) Thomas Cederborg, Ishaan Grover, Charles Lee Isbell, and Andrea Lockerd Thomaz.  Policy shaping with human teachers.  In _IJCAI_ , 2015.
  * Chaplot et al. (2016) Devendra Singh Chaplot, Guillaume Lample, Kanthashree Mysore Sathyendra, and Ruslan Salakhutdinov.  Transfer deep reinforcement learning in 3d environments: An empirical study.  In _NIPS Deep Reinforcemente Leaning Workshop_ , 2016.
  * Cheng et al. (2018) Ching-An Cheng, Xinyan Yan, Nolan Wagener, and Byron Boots.  Fast policy learning through imitation and reinforcement.  _arXiv preprint arXiv:1805.10413_ , 2018.
  * Devin et al. (2017) Coline Devin, Abhishek Gupta, Trevor Darrell, Pieter Abbeel, and Sergey Levine.  Learning modular neural network policies for multi-task and multi-robot transfer.  In _2017 IEEE International Conference on Robotics and Automation (ICRA)_ , pp. 2169–2176. IEEE, 2017.
  * Engel et al. (2005) Yaakov Engel, Shie Mannor, and Ron Meir.  Reinforcement learning with gaussian processes.  In _Proceedings of the 22nd international conference on Machine learning_ , pp. 201–208, 2005.
  * Fernández & Veloso (2006) Fernando Fernández and Manuela M. Veloso.  Probabilistic policy reuse in a reinforcement learning agent.  In _International Conference on Autonomous Agents and Multiagent Systems_ , 2006.
  * García & Fernández (2015) Javier García and Fernando Fernández.  A comprehensive survey on safe reinforcement learning.  _Journal of Machine Learning Research_ , 2015.
  * Gaweda et al. (2005) Adam E Gaweda, Mehmet K Muezzinoglu, George R Aronoff, Alfred A Jacobs, Jacek M Zurada, and Michael E Brier.  Individualization of pharmacological anemia management using reinforcement learning.  _Neural Networks_ , 18(5-6):826–834, 2005.
  * Griffith et al. (2013) Shane Griffith, Kaushik Subramanian, Jonathan Scholz, Charles L Isbell, and Andrea L Thomaz.  Policy shaping: Integrating human feedback with reinforcement learning.  In C. J. C. Burges, L. Bottou, M. Welling, Z. Ghahramani, and K. Q. Weinberger (eds.), _Advances in Neural Information Processing Systems 26_ , pp. 2625–2633. Curran Associates, Inc., 2013.  URL http://papers.nips.cc/paper/5187-policy-shaping-integrating-human-feedback-with-reinforcement-learning.pdf.
  * Haarnoja et al. (2018) Tuomas Haarnoja, Aurick Zhou, Kristian Hartikainen, George Tucker, Sehoon Ha, Jie Tan, Vikash Kumar, Henry Zhu, Abhishek Gupta, Pieter Abbeel, et al.  Soft actor-critic algorithms and applications.  _arXiv preprint arXiv:1812.05905_ , 2018.
  * Harutyunyan et al. (2014) Anna Harutyunyan, Tim Brys, Peter Vrancx, and Ann Nowé.  Off-policy shaping ensembles in reinforcement learning.  _arXiv preprint arXiv:1405.5358_ , 2014.
  * Hussein et al. (2017) Ahmed Hussein, Mohamed Medhat Gaber, Eyad Elyan, and Chrisina Jayne.  Imitation learning: A survey of learning methods.  _ACM Computing Surveys (CSUR)_ , 50(2):1–35, 2017\.
  * Konidaris & Barto (2007) George Konidaris and Andrew G Barto.  Building portable options: Skill transfer in reinforcement learning.  In _IJCAI_ , volume 7, pp. 895–900, 2007.
  * Kuhnle et al. (2019) Andreas Kuhnle, Nicole Röhrig, and Gisela Lanza.  Autonomous order dispatching in the semiconductor industry using reinforcement learning.  _Procedia CIRP_ , 79:391–396, 2019.  ISSN 2212-8271.  doi: https://doi.org/10.1016/j.procir.2019.02.101.  URL https://www.sciencedirect.com/science/article/pii/S2212827119302185.  12th CIRP Conference on Intelligent Computation in Manufacturing Engineering, 18-20 July 2018, Gulf of Naples, Italy.
  * Lazic et al. (2018) Nevena Lazic, Craig Boutilier, Tyler Lu, Eehern Wong, Binz Roy, MK Ryu, and Greg Imwalle.  Data center cooling using model-predictive control.  _Advances in Neural Information Processing Systems_ , 31, 2018.
  * MacGlashan et al. (2017) James MacGlashan, Mark K. Ho, Robert Loftin, Bei Peng, Guan Wang, David L. Roberts, Matthew E. Taylor, and Michael L. Littman.  Interactive learning from policy-dependent human feedback.  In Doina Precup and Yee Whye Teh (eds.), _Proceedings of the 34th International Conference on Machine Learning_ , volume 70 of _Proceedings of Machine Learning Research_ , pp. 2285–2294, International Convention Centre, Sydney, Australia, 06–11 Aug 2017. PMLR.  URL http://proceedings.mlr.press/v70/macglashan17a.html.
  * Madden & Howley (2004) Michael G Madden and Tom Howley.  Transfer of experience between reinforcement learning environments with progressive difficulty.  _Artificial Intelligence Review_ , 21(3-4):375–398, 2004.
  * Mirowski et al. (2018) Piotr Mirowski, Matt Grimes, Mateusz Malinowski, Karl Moritz Hermann, Keith Anderson, Denis Teplyashin, Karen Simonyan, Andrew Zisserman, Raia Hadsell, et al.  Learning to navigate in cities without a map.  In _Advances in Neural Information Processing Systems_ , pp. 2419–2430, 2018.
  * Parisotto et al. (2015) Emilio Parisotto, Jimmy Lei Ba, and Ruslan Salakhutdinov.  Actor-mimic: Deep multitask and transfer reinforcement learning.  _arXiv preprint arXiv:1511.06342_ , 2015.
  * Plisnier et al. (2019) Hélène Plisnier, Denis Steckelmacher, Diederik M. Roijers, and Ann Nowé.  The actor-advisor: Policy gradient with off-policy advice.  _arXiv_ , abs/1902.02556, 2019.
  * Ravindran & Barto (2003) Balaraman Ravindran and Andrew G Barto.  Relativized options: Choosing the right transformation.  In _Proceedings of the 20th International Conference on Machine Learning (ICML-03)_ , pp. 608–615, 2003.
  * Rusu et al. (2015) Andrei A Rusu, Sergio Gomez Colmenarejo, Caglar Gulcehre, Guillaume Desjardins, James Kirkpatrick, Razvan Pascanu, Volodymyr Mnih, Koray Kavukcuoglu, and Raia Hadsell.  Policy distillation.  _arXiv preprint arXiv:1511.06295_ , 2015.
  * Silva & Gombolay (2021) Andrew Silva and Matthew Gombolay.  Encoding human domain knowledge to warm start reinforcement learning.  In _Proceedings of the AAAI Conference on Artificial Intelligence_ , volume 35, pp. 5042–5050, 2021.
  * Sutton et al. (2000) Richard Sutton, D McAllester, Satinder Singh, and Yishay Mansour.  Policy Gradient Methods for Reinforcement Learning with Function Approximation.  _Neural Information Processing Systems (NIPS)_ , pp. 7, 2000.
  * Sutton & Barto (2018) Richard S. Sutton and Andrew G. Barto.  _Reinforcement Learning: An Introduction_.  MIT Press, Cambridge, 2018.
  * Sutton et al. (1999) Richard S Sutton, David A McAllester, Satinder P Singh, Yishay Mansour, et al.  Policy gradient methods for reinforcement learning with function approximation.  In _Neural Information Processing Systems (NeurIPS)_ , volume 99, pp. 1057–1063, 1999.
  * Tang & Haarnoja (2017) Haoran Tang and Tuomas Haarnoja.  Learning diverse skills via maximum entropy deep reinforcement learning.  _URL http://bair. berkeley. edu/blog/2017/10/06/soft-q-learning_ , 2017.
  * Taylor & Stone (2007) Matthew E Taylor and Peter Stone.  Cross-domain transfer for reinforcement learning.  In _Proceedings of the 24th international conference on Machine learning_ , pp. 879–886. ACM, 2007.
  * Taylor & Stone (2009) Matthew E Taylor and Peter Stone.  Transfer learning for reinforcement learning domains: A survey.  _Journal of Machine Learning Research_ , 2009.
  * Taylor et al. (2007) Matthew E Taylor, Peter Stone, and Yaxin Liu.  Transfer learning via inter-task mappings for temporal difference learning.  _Journal of Machine Learning Research_ , 8(Sep):2125–2167, 2007.
  * Wexler et al. (2022) Benjamin Wexler, Elad Sarafian, and Sarit Kraus.  Analyzing and overcoming degradation in warm-start reinforcement learning.  2022\.
  * Zhu & Liao (2017) Feiyun Zhu and Peng Liao.  Effective warm start for the online actor-critic reinforcement learning based mhealth intervention.  _arXiv preprint arXiv:1704.04866_ , 2017.
  * Zhu et al. (2020) Zhuangdi Zhu, Kaixiang Lin, and Jiayu Zhou.  Transfer learning in deep reinforcement learning: A survey, 2020.


11institutetext: Department of Physics, Indian Institute of Technology,
Guwahati, Assam - 781039, India

# Evolution of holographic Fermi surface from non-minimal couplings

Wadbor Wahlang _Email:_<EMAIL_ADDRESS>
(Received: date / Revised version: date)

###### Abstract

We study a holographic toy model by considering a probe fermion of finite
charge density in an anisotropic background. By computing the fermionic
spectral function numerically, we observed that the system exhibits some
interesting behaviours in the nature of the Fermi surface (FS) and its
evolution when tuning the controlling parameters. We introduced non-minimal
interaction terms in the action for holographic fermions along with a complex
scalar field but neglecting the backreaction of the fermions field on the
background. Suppression in the spectral weight and deformation of FS is
observed, which are reminiscent of the results seen in various condensed
matter experiments in real materials.

##  1 Introduction

The study of strongly correlated condensed matter systems using holographic
approach is one of the active field of research. It provides the necessary
tool in understanding the condensed matter systems from classical gravity
theories. The earlier promising results of holographic method were those of
its application in quark-gluon plasma Sadeghi:2013zma  and in holographic
superconductors. The well-known holographic principle Maldacena:1997re ;
Witten:1998qj  has been applied as an alternative framework to understand
various complex phenomena; such as non-linear hydrodynamics
Bhattacharyya:2008jc , non-Fermi liquid behavior Faulkner:2009wj ;
Cubrovic:2009ye , transport phenomena Herzog:2007ij , and high temperature
superconductors Hartnoll:2008vx ; Hartnoll:2008kx ; Horowitz:2010gk  to point
out a few. Since then, several applications have been found out to study the
emergence of Fermi and non-Fermi liquid behaviours in the fermionic systems
Liu:2009dm ; Lee:2008xf ; Faulkner:2009am ; Guarrera:2011my ; Faulkner:2011tm
, and later this method was also implemented as a tools to study the
conductivities and phase transition in high $T_{c}$ superconductor, the
pseudogap phase, Mott transition Hartnoll:2008vx ; Langley:2015exa ;
Vanacore:2015poa ; Edalati:2010ww ; PhysRevD.90.126013  and many more
applications in the field of high energy and nuclear physics. In the recent
years, new materials were discovered and they belong to a class of topological
insulators, and the so-called Dirac and Weyl semimetals. These new compounds
have many interesting features. Some of which have posted as a challenge to
the condensed matter research community.

Some of the results seen in condensed matter experiments on these new
materials, such as those from Angle-resolved photoemission spectroscopy
(ARPES) Damascelli:2003bi ; 2011PhRvL.106l7005K  were quite a challenge for a
condensed matter theorist. The study of electronic properties in these new
materials and high-$T_{c}$ superconductors, including the band structures,
quasiparticle excitations and the Fermi surface (FS) can be done through ARPES
experiment. However, ARPES shows the presence of an unconventional pseudo-gap
phase and a disconnected Fermi surface in the spectral function. This
partially destructed Fermi surface is known in literature as the Fermi Arc
1998Natur.392..157N ; 2006PhRvB..74v4510Y ; Cremonini:2018xgj ; Seo:2018hrc ;
PhysRevB.99.161113 . The appearance of Fermi arc has also been observed in the
ARPES experiments in different condensed matter systems such as in the
topological insulators, Dirac and Weyl semi-metals PhysRevB.99.161113 ; YangL
; Brillaux:2020chh ; Su-Yang ; Su-Yang1 ; Noam ; Peng ; Mehdi ; MZHazan ;
Binghai ; PhysRevB.83.205101 . A detailed review on this topic can be found in
BaiqingLv . In all such mentioned systems, Fermi arcs appear as the surface
states. It was speculated that the presence of these surface states might be
caused by some unknown underlying symmetries or structures in the bulk of the
material. Since these systems are strongly correlated, it is difficult to pin
point the exact mechanisms that cause these unconventional electronic
properties using conventional approaches. To address these problems, many have
started using AdS/CFT correspondence as a tool that allow us to study the
strongly correlated systems. In the recent years, lots of experiments and
theories were performed and modelled to shed some light in understanding of
the underlying mechanism behind these strange properties of the Fermi surface
from the perspective of condensed matter physics PhysRevB.76.174501 ;
PhysRevB.73.174501 ; PhysRevB.86.115118 ; PhysRevB.74.125110  and other
approaches like the holographic method Hartnoll:2009sz ; Herzog:2009xv ;
McGreevy:2009xe ; Zaanen:2015oix ; Hartnoll:2016apf . In the recent pioneering
works Vanacore:2015poa ; Edalati:2010ww ; Edalati:2010ge ; Vegh:2010fc ;
Benini:2010qc , they have investigated Mott transition and the evolution of
Fermi arc-like structures, taking various classical gravity background as the
bulk theory. The authors in Edalati:2010ww ; Edalati:2010ge , considered a non
minimal coupling of the form $p\,\bar{\psi}\,\not{F}\psi$, where the gauge
field is coupled with the fermions field $\psi$. They observed a transfer of
spectral weight between energy bands as a strength of the coupling $p$ is
increased and also beyond certain critical value of $p$, a gap emerges in the
spectral function which are the key features of Mott physics in the condensed
matter systems. Later in Vanacore:2015poa , by exploring the system symmetries
they proposed a similar modification of the coupling mentioned earlier and the
Fermi-arcs features were seen.

Motivated by the features seen in these works Vanacore:2015poa ;
Edalati:2010ww ; Edalati:2010ge , in this paper, we will study a holographic
toy model similar to the one considered in our previous work
Chakrabarti:2021qie , where we observed an evolution of a pair of Fermi arcs.
Here, we will essentially extend by modifying our previous couplings with a
different one Chakrabarti:2019gow , where the bulk dipole coupling is modified
to break the rotational and Lorentz symmetries at the boundary. Our main
objective here is to further study the evolution of the Fermi surface and
their band structure in presence of the two couplings given in section 3.

In this work, we will consider the Q-lattice Donos:2013eha  as our holographic
lattice background which were constructed with a complex scalar field that
breaks the translational symmetry in the bulk. While in Chakrabarti:2021qie ,
we have studied the fermionic spectral function in the Q-lattice background
with couplings of the form $p\,\bar{\psi}\,\not{F}\psi$ and
$p\,\bar{\psi}\,\not{F}|\phi|^{2}\psi$. Given the anisotropic background it
will be interesting to study the Fermi surface evolution due to Fermions bulk
couplings parameters given in equation 7 below.

We will organise the discussion in this paper as follows: In section 2, we
have a brief discussion on the important aspects of the Q-lattice background
solution, then in section 3, we write down the Fermion’s action with our
coupling terms and equations of motion, leading to the spectral function
$A\,(\omega,\,\vec{k})$ which is the most relevant quantity for our
discussion. Thus is followed by section 4, where we present the numerical
results with discussion on each of them. In the last section, we summarize the
results and conclude with some futures directions.

##  2 The Q-lattice geometry

In this section, we will briefly discuss about the Q-lattice since we had put
a detail discussion in Chakrabarti:2021qie , we will highlight only few
important points here. Here, the set up for the Q-lattice background will be a
$(3+1)-$dimensional gravity theory in the bulk that could be mapped to a dual
theory at the boundary in $(2+1)-$ dimensions. The full action consists of
Einstein-Maxwell field and a complex scalar field given by:

$\displaystyle\mathcal{S}=\frac{1}{\kappa^{2}}\int
d^{4}x\sqrt{-g}\left[\mathcal{R}+\frac{6}{L^{2}}-\frac{1}{4}F^{2}-|\partial\phi|^{2}-m^{2}|\phi|^{2}\right]$
(1)

where $\mathcal{R}$ is the Ricci scalar, Maxwell’s field strength $F=dA$.
Finally, $\kappa^{2}=16\pi G$ and $L$ are the effective reduced gravitational
constant and the AdS radius respectively which we set to be unity later. As
mentioned before, we consider a complex scalar field $\phi$, which will break
the translational invariance of the boundary field theory. From the above
action (1) we have the following equations of motions

$\displaystyle R_{\mu\nu}=$ $\displaystyle
g_{\mu\nu}\left(-3+\frac{m^{2}}{2}|\phi|^{2}\right)+\partial_{(\mu}\phi\partial_{\nu)}\phi^{*}+\frac{1}{4}\left(2F_{\mu\nu}^{2}-\frac{1}{2}g_{\mu\nu}F^{2}\right)$
$\displaystyle\nabla_{\mu}F^{\mu\nu}=0\;,\;\;\left(\nabla^{2}-m^{2}\right)\phi=0.$
(2)

To evaluate these equations, we shall take the following ansatz for the metric
and the complex scalar field

$\displaystyle
ds^{2}=-g_{tt}(z)\;dt^{2}+g_{zz}(z)\;dz^{2}+g_{xx}(z)\;dx^{2}+g_{yy}(z)\;dy^{2}.$
(3) $\displaystyle\text{where},$ $\displaystyle
g_{tt}(z)=\frac{U(z)}{z^{2}};\;g_{zz}(z)=\frac{U(z)^{-1}}{z^{2}};\;g_{xx}(z)=\frac{V_{1}(z)}{z^{2}};$
$\displaystyle g_{yy}(z)=\frac{V_{2}(z)}{z^{2}},\;\;A=(1-z)\;a(z)dt,$
$\displaystyle\;U(z)\,=\,(1-z)\;u(z),\,\,\phi=e^{i\,k_{1}\,x+i\,k_{2}\,y}\;\chi(z)\,.$
(4)

In particular, the ansatz for the scalar field is chosen to incorporate the
breaking of translational symmetry by the same scalar field in both $x$ and
$y$ directions. Here, $u,V_{1},V_{2},a,\chi$ are all unknown functions which
depends on radial coordinate $z$, and $k_{1},\,k_{2}$ are constants
interpreted as wave numbers in the lattice. From equation 2 above, when
combined with the ansatz, we get four second order coupled ODEs for
$V_{1},\,V_{2},\,a,\,\chi$ and one first order for $u$. In general, for none
zero scalar mass $m_{\phi}^{2}$, the leading behaviour of the complex scalar
field near the AdS boundary ($z\longrightarrow 0$) is given by

$\displaystyle\chi(z)\,=\,z^{\alpha_{-}}\,\chi^{(1)}\,+\,z^{\alpha_{+}}\,\chi^{(2)}\,+...$
(5)

where, $\alpha_{\pm}=3/2\pm\sqrt{9/4+m_{\phi}^{2}}$ . The leading term
$\chi^{(1)}$ is associated with the source of the dual scalar operator in the
boundary theory, whose dimension is $\Delta=3-\alpha_{-}=\alpha_{+}$. To this
end we study in detail the case when, scalar field mass $m_{\phi}^{2}=0$,
which corresponds to $\alpha_{-}=0$ and the corresponding marginal dual scalar
operator in the (2+1) dimensional boundary theory. To construct a numerical
black hole solution, we solve these equations as boundary value problems with
one end at the horizon and the other at the AdS boundary. At the horizon
$z=1$, we use the regularity conditions by expanding the fields near the
horizon, and at the AdS boundary or UV, we assume the following leading
expansion,

$\displaystyle u=1+O(z)\,\,,\,\,V_{1}=1+O(z)\,\,,\,\,V_{2}=1+O(z)\;,$
$\displaystyle a=\mu+O(z)$ (6)

Once we obtained the boundary conditions, we linearize the ODEs by
discretizing simultaneously both the equations of motions and the boundary
conditions following Trefethen ; Boyd  and then use the iterative Newton-
Raphson’s (NR) method following the techniques given in Andrade:2017jmt ;
Krikun:2018ufr . Further, for fixed mass $m^{2}_{\phi}$, we found that the
solutions are specified in terms of three dimensionless parameters namely,
$T/\mu\,,\,k/\mu$ and $\chi^{(1)}/\mu^{\alpha_{-}}$, where $T$ is the
temperature of black hole. We showed some plots of the background solutions in
our previous work Chakrabarti:2021qie  and we will use the same numerical
background for analysis of the fermion spectral function later.

##  3 Fermions: Action and the spectral function.

As mentioned in the introduction, we will consider the probe fermions in the
anisotropic background solutions discussed above along with two non minimal
couplings and the fermionic action is given by

$\displaystyle\mathcal{S}_{\psi}=\int
d^{4}x\sqrt{-g}i\bar{\psi}\bigg{(}\not{D}-m_{\psi}-i\,p_{1}\,|\phi|^{2}\not{F}-i\,p_{2}\,|\phi|^{2}{\bf{\Gamma}}\not{F}\bigg{)}\psi$
(7)

Here, $\phi$ is the same complex scalar field discussed in the lattice
solution and $m_{\psi}$ is the fermion mass. Expanded expression of the
symbols given in 7 are as follows:

$\displaystyle{\bf{\Gamma}}=\Gamma^{\underline{z}}\Gamma^{\underline{t}}(\hat{n}.\vec{\Gamma})\;,\;\;\;\;\;\;$
$\displaystyle\not{D}=e^{\mu}_{c}\Gamma^{c}\left(\partial_{\mu}+\frac{1}{4}\omega_{\mu}^{ab}\Gamma_{ab}-iqA_{\mu}\right)$
$\displaystyle\not{F}=\frac{1}{2}\Gamma^{ab}e^{\mu}_{a}e^{\nu}_{b}F_{\mu\nu}\;,\;\;$
$\displaystyle\vec{\Gamma}\equiv(\Gamma^{\underline{x}},\Gamma^{\underline{y}}).$

where, $e^{\mu}_{a},\omega_{\mu}^{ab}$ are the vielbeins and spin connection,
$q$ is the charge of the bulk fermions. The first coupling with controlling
parameter $p_{1}$ is the same to what we studied in Chakrabarti:2021qie ,
however we include here to study the combine effects. While the second
coupling parametrized by $p_{2}$ is similar the one studied in the evolution
of Fermi Arc from Mott insusator Vanacore:2015poa  that breaks rotational and
Lorentz symmetries of the boundary theory. We studied this type of coupling
term in our previous work Chakrabarti:2019gow , involving a real scalar field
that controls the transition from circular to arc like Fermi surface
controlled by the scalar condensation in the bulk at low temperature. In this
paper, we will also look at the effects of these couplings on fermionic
spectrum. The Dirac equation from the above action is given by:

$\displaystyle\bigg{(}\not{D}-m_{\psi}-i\,p_{1}\,|\phi|^{2}\not{F}-i\,p_{2}\,|\phi|^{2}{\bf{\Gamma}}\not{F}\bigg{)}\psi=0.$
(8)

In order to further simplify the Dirac equation, we choose the following gamma
matrices

$\displaystyle\Gamma^{\underline{z}}=\left(\begin{array}[]{cc}-\sigma^{3}&0\\\
0&-\sigma^{3}\end{array}\right),\;\;\Gamma^{\underline{t}}=\left(\begin{array}[]{cc}i\sigma^{1}&0\\\
0&i\sigma^{1}\end{array}\right),$ (13)
$\displaystyle\Gamma^{\underline{x}}=\left(\begin{array}[]{cc}-\sigma^{2}&0\\\
0&\sigma^{2}\end{array}\right)\;\;,\;\;\Gamma^{\underline{y}}=\left(\begin{array}[]{cc}0&\sigma^{2}\\\
\sigma^{2}&0\end{array}\right)$ (18)

Now, we can expand the Dirac equation (8) together with rescaling of the field
$\psi=(g_{tt}g_{xx}g_{yy})^{-\frac{1}{4}}e^{-i\omega
t+ik_{x}x+ik_{y}y}\xi\,(z,\vec{k})$, where the vector
$\vec{k}\equiv(-\omega,k_{x},k_{y})$, we have

$\displaystyle\bigg{(}$
$\displaystyle\frac{1}{\sqrt{g_{zz}(z)}}\;\Gamma^{\underline{z}}\;\partial_{z}+\frac{1}{\sqrt{-g_{tt}(z)}}\;\left(\Gamma^{\underline{t}}\;(-i\omega)-iqA_{t}\right)+$
$\displaystyle\frac{1}{\sqrt{g_{xx}(z)}}\;\Gamma^{\underline{x}}\;(ik_{x})+\frac{1}{\sqrt{g_{yy}(z)}}\;\Gamma^{\underline{y}}\;(ik_{y})-m_{\psi}\;+$
$\displaystyle\frac{\partial_{z}A_{t}}{\sqrt{-g_{zz}(z)g_{tt}(z)}}\left(-i\,p_{1}\,|\phi|^{2}\;\Gamma^{\underline{z}\;\underline{t}}-i\,p_{2}\;|\phi|^{2}\,{\bf{\Gamma}}\;\Gamma^{\underline{z}\;\underline{t}}\right)\bigg{)}\,\xi(z,\textbf{k})=0$
(19)

Now using the basis (13) and by splitting the spinors
$\xi=(\xi_{1},\xi_{2})^{T}$, and $\xi_{j}=(\beta_{j},\alpha_{j})^{T}$, with
$j$ taking values $j=1,2$, we get the following coupled radial equations

$\displaystyle\left(\frac{1}{\sqrt{g_{zz}}}\partial_{z}\pm
m_{\psi}\right)\left(\begin{matrix}\beta_{1}\cr\alpha_{1}\end{matrix}\right)\mp\frac{(\omega+qA_{t})}{\sqrt{-g_{tt}(z)}}\left(\begin{matrix}\alpha_{1}\cr\beta_{1}\end{matrix}\right)+\frac{k_{x}}{\sqrt{g_{xx}}}\left(\begin{matrix}\alpha_{1}\cr\beta_{1}\end{matrix}\right)$
$\displaystyle-\frac{k_{y}}{\sqrt{g_{xx}}}\left(\begin{matrix}\alpha_{2}\cr\beta_{2}\end{matrix}\right)+\frac{\partial_{z}A_{t}}{\sqrt{-g_{zz}g_{tt}}}\;(p_{1}\,|\phi|^{2}-p_{2}\;|\phi|^{2})\left(\begin{matrix}\alpha_{1}\cr\beta_{1}\end{matrix}\right)=0$
(20) $\displaystyle\left(\frac{1}{\sqrt{g_{zz}}}\partial_{z}\pm
m_{\psi}\right)\left(\begin{matrix}\beta_{2}\cr\alpha_{2}\end{matrix}\right)\mp\frac{(\omega+qA_{t})}{\sqrt{-g_{tt}(z)}}\left(\begin{matrix}\alpha_{2}\cr\beta_{2}\end{matrix}\right)-\frac{k_{x}}{\sqrt{g_{xx}}}\left(\begin{matrix}\alpha_{2}\cr\beta_{2}\end{matrix}\right)$
$\displaystyle-\frac{k_{y}}{\sqrt{g_{xx}}}\left(\begin{matrix}\alpha_{1}\cr\beta_{1}\end{matrix}\right)+\frac{\partial_{z}A_{t}}{\sqrt{-g_{zz}g_{tt}}}\;(p_{1}\,|\phi|^{2}+p_{2}\;|\phi|^{2})\left(\begin{matrix}\alpha_{2}\cr\beta_{2}\end{matrix}\right)=0$
(21)

From now on, we follow the standard holographic procedure for extracting the
Green’s function. One can expand these equations near the horizon ($z=1$) to
obtain the in-falling boundary condition given by

$\left(\begin{matrix}\beta_{j}(z,\vec{k})\cr\alpha_{j}(z,\vec{k})\end{matrix}\right)\sim
c_{j}(1-z)^{-\frac{i\omega}{4\pi T}}.$ (22)

Now to extract the boundary Green’s function, one needs the asymptotic
behaviour of the Dirac equations near the AdS boundary $(z\rightarrow 0)$ to
identify the source and the expectations values. In this case, the leading
behaviour at the boundary is given by

$\left(\begin{matrix}\beta_{j}\cr\alpha_{j}\end{matrix}\right){\approx}\;a_{j}z^{m_{\psi}}\left(\begin{matrix}1\cr
0\end{matrix}\right)+b_{j}z^{-m_{\psi}}\left(\begin{matrix}0\cr
1\end{matrix}\right).$ (23)

We then followed the usual prescription used in Faulkner:2009am ;
Guarrera:2011my ; Vanacore:2015poa  to obtain the Green’s function by
extracting the coefficients $a_{j}$ and $b_{j}$ and because the couplings
considered here, we have a mixing of various spinorial components, thus we use
two different sets of linearly independent boundary conditions that can be
written in the form $B=SA$ and expanded as

$\displaystyle\left(\begin{array}[]{cc}\beta^{I}_{1}&\beta^{II}_{1}\\\
\beta^{I}_{2}&\beta^{II}_{2}\end{array}\right)=\left(\begin{array}[]{cc}s_{11}&s_{12}\\\
s_{21}&s_{22}\end{array}\right)\left(\begin{array}[]{cc}\alpha^{I}_{1}&\alpha^{II}_{1}\\\
\alpha^{I}_{2}&\alpha^{II}_{2}\end{array}\right),$ (30)

Now the retarded Green’s function is defined as

$\displaystyle
G^{R}(\omega,k_{x},k_{y})=-i\left(\begin{array}[]{cc}s_{11}&s_{12}\\\
s_{21}&s_{22}\end{array}\right)\cdot\gamma^{t}$ (33)

with gamma matrices in our basis as $\gamma^{t}=i\sigma_{1}$. Now from the
retarded Green’s function we can get quantity of interest, i.e., the spectral
function $A\,(\omega,k_{x},k_{y})$ given by

$\displaystyle
A(\omega\,,\vec{k})=\text{Im}\left[\text{Tr}\,G^{R}(\omega\,,\vec{k})\right].$
(34)

where $\vec{k}\,\equiv\,(k_{x},k_{y})$. In the section that follows we will
study the properties of $A\,(\omega\,,\vec{k})$. The importance of this
spectral function is that it directly relates to real condensed matter
experiments.

##  4 Numerical Results and Discussion

In this section, we are going to look at the qualitative properties of the
spectral function $A(\omega,\vec{k})$ defined in (34) above, which reflect on
the nature of Fermi surface (FS) and the dispersion spectrum by varying the
coupling $p_{1}\,,p_{2}$ and also the background parameters. For our numerical
purposes, we defined the Fermi level with small offset from $\omega=0$. In our
previous work Chakrabarti:2021qie , we have explored the affects of
temperature and the source $\chi^{(1)}$ on the Fermi surface with and without
couplings, also more analysis on fermions spectral function due to the
Q-lattice can be found in these papers Ling2014 ; Iliasov:2019pav . Here we
show the results only for scenarios when coupling parameters are non-
vanishing.

As discussed above the complex scalar field can break translational invariance
in both $x$ and $y$ directions, depending on $k_{1}$ and $k_{2}$. Let us first
study for $k_{2}=0$ and consider only $k_{1}$. Fixing the background scalar
mass $m_{\phi}^{2}=0$, then the leading constant $\chi^{(1)}$ which represents
the source of dual boundary operator of the bulk complex scalar field can be
varied and we have seen in Chakrabarti:2021qie  and references therein, as we
increase the strength of the source, the weight of spectral function reduces
along $k_{x}$ where translational symmetry is broken. Since the spectrum of
our first couplings control by $p_{1}$ is known from our previous discussion
in Chakrabarti:2021qie , we directly show the combined effects of $p_{1}$ and
$p_{2}$ in Figure 1.

Figure 1: Density plot of $A\,(k_{x},k_{y})$ by varying the value of $p_{1}$.
Here, we fixed $p_{2}=0.3$ and $m_{\psi}=0\;,q=1$. Left to right: are for
$p_{1}=-0.2\,,-0.3,\,-0.4,$ . The top, middle and bottom panel correspond to
$T/\mu=0.02,\,0.009,\,0.001$ with $m_{\phi}^{2}=0$ and source $\chi^{(1)}=2$
respectively. Figure 2: Energy-momentum dispersion with $p_{1}$, $p_{2}$ set
to non-zero and $m_{\psi}=0,\;q=1$, $T/\mu=0.009$, $m_{\phi}^{2}=0$,
$k_{1}/\mu=0.8$,and $k_{2}/\mu=0$. Top and middle panel are along $\omega-
k_{x}\;(k_{y}=0)$ and $\omega-k_{y}\;(k_{x}=0)$ plane, while the bottom panel
is along $\omega-k_{y}\;(k_{x}=-1)$. From left to right the parameters [
$(p_{2}\,,p_{1})\,]\,=\,[\,(0.3,-0.2),(0.3,-0.3),(0.3,-0.4)]$ respectively.

One can immediately notice in Figure 1 that for low temperature (bottom panel)
the combine effects yields two arcs like spectrum, and the peak of FS is more
sharp in comparison to that of higher temperature (top panel) where the inner
arc vanishes. Further, the stretched Fermi surface that appeared in the
spectral function because of the anisotropic background for large $\chi^{(1)}$
values, now its amplitude is suppressed and more prominent along negative
$k_{x}-$direction specially at low temperature, due to the second coupling.
However, from previous analysis at lower temperature, the effects of $p_{1}$
induces a second Fermi surface that is partially destroyed along $x-$direction
which is the signature of second coupling. Whereas, along $k_{y}$ direction,
the spectral function indicates the existence of sharp Fermi surface with a
large density of states. The sharp peak in the spectral function, essentially
indicate the presence of stable quasi-particle with longer lifetime.
Similarly, from the energy momentum dispersion shown in Figure 2, we see the
presence of gapless spectra which is anisotropic in nature and the surface
state induces by $p_{2}$ connecting the lower and upper band.

From the emergence of these interesting behaviour in the spectral function
from our couplings $p_{1}$ and $p_{2}$, firstly we can think of the effects of
the source $\chi^{(1)}$ as a doping parameter in the pseudogap region and
secondly, for both the couplings, we know from the work in Vanacore:2015poa ;
Edalati:2010ww , the appearance of these Fermi arcs can be understood in the
context of pole-zero duality PhysRevD.90.126013 . In terms of the two diagonal
entries of the boundary fermions Green’s function $G_{ij}(\omega,\vec{k})$,
the FS appears at two poles which exist at $G_{11}(\omega=0,|k_{f}|)$ and
$G_{22}(\omega=0,-|k_{f}|)$. Now the presence of $p_{2}$ coupling, whenever
the pole/zero of $G_{11}$ coincides with zero/pole of $G_{22}$, we get a
reduced spectral weight along $-k_{x}$ or $+k_{x}$ depending on the sign of
$p_{2}$ that appears as Fermi arcs. The presence of these pairs of FS in the
spectral function indicates that the scalar field plays a crucial role in not
mixing the individual spectra of $p_{1}$ and $p_{2}$. Hence we eventually have
two Fermi surfaces on changing $p_{1},p_{2}$. Notice that in our plots, we
choose the mass of scalar field to be $m_{\phi}=0$, because for non-zero
masses, the way our scalar field manifest itself in the coupling terms, its
effects will be highly suppressed. However, we plotted for non-zero fermion
mass in Figure 4. Again here for same charge $q$ and background parameter we
see a similar spectrum to that in the case of zero mass shown above.

So far we have consider only $k_{1}$, now let us turn on $k_{2}$. As shown in
our previous work Chakrabarti:2021qie , depending on whether $k_{1}>k_{2}$ or
$k_{1}<k_{2}$, the fermionic spectrum is gapped along $k_{x}$ or $k_{y}$. We
consider the case when $k_{2}>k_{1}$ and the plots are shown in Figure 3. In
the top panel we fixed $p_{1}=1$ and vary boundary source $\chi^{(1)}$ of the
bulk scalar field $\phi$. As the source $\chi^{(1)}$ is increased, in addition
to being stretch from circular shape to an elliptical shape, we see an
emergence of a second FS of smaller peaks. Some similarity is seen when we
vary $p_{2}$ instead and fixing $\chi^{(1)}=2$. Here also we observed a new
kind of spectrum which is different from that observed in Vanacore:2015poa ;
Chakrabarti:2019gow  with similar couplings. It was earlier shown in
Gubser:2012yb  the presence of multiple FS in holographic model with no Fermi
arcs and then recently in Cremonini:2019fzz ; Balm:2019dxk , the Fermi
features were seen triggered by the effective mass term and the lattice
effects in their holographic models. It is to be noted here that though both
our studies and theirs are somewhat similar in terms of the features in
suppression of the spectral function and the Fermi surfaces, our couplings and
the one considered in Cremonini:2019fzz  should correspond to a different
boundary field theory with a different kind of underlying mechanism. Moreover,
the evolution of these Fermi arcs without including superconductivity as
discussed in Vegh:2010fc ; Benini:2010qc , it could provide an interesting
framework for studying the Fermi arcs in the pseudogap phase and cuprates.

Up till now, we have discussed only the spectra produced by the two couplings.
Let us try to relate these results to real condensed matter systems. Firstly,
the emergence of these pair of Fermi surfaces are mostly our interesting
features out of our numerics as these kind of spectra have been observed in
the ARPES experiments measuring the spectral function in real condensed matter
systems such as those of Topological insulators (TI), Dirac and Weyl semi-
metals PhysRevB.99.161113 ; YangL ; MZHazan . Also, for superconductivity,
these types of interactions terms provides us a holographic model in the
pseudogap region in the phase diagram where these Fermi arcs are seen. They
are connected through the suppression in the weight spectral function. Similar
to our observation, the presence of these kind of arcs in pairs, was also
discussed in a theoretical model in Kun-Yang . While in the case of Dirac and
Weyl semi-metals, Fermi arcs emerges out as surface states which connect the
two Dirac points in three-dimensional bulk crystals. From the energy band
dispersion shown in Figure 2 in our models, we can see the presence of non-
trivial gapless surface states that were also observed in experiments. Details
discussion on these topics can be found in these articles Su-Yang ; Su-Yang1 ;
Noam ; Peng ; MZHazan  and references therein. Since our holographic system at
the boundary is ($2+1$) dimensions, we can think of these pairs of Fermi arcs
as the surface states of a ($3+1$) dimensional bulk material. Thus the
observations from our holographic models are reminiscent to those of real
materials in terms of the band spectrum and surface states known as Fermi
arcs. There have been other holographic models also, such as those in
Rodgers:2021azg ; Landsteiner:2019kxb ; Oh:2020cym , which attempted to
address these interesting and rich phenomena observed in topological
insulators and semi-metals.

Figure 3: Here we plotted the spectral density with fermion mass $m_{\psi}=0$
and charge $q=1$. In the top panel, we fixed $T/\mu=\,0.009$, coupling
$p_{2}=1.$ and vary source $\chi^{(1)}=(\,0.5,\,1.0,\,1.5,\,1.8\,)$ with
$k_{1}/\mu=0.2$ , $k_{2}/\mu=0.8$, and $m_{\phi}^{2}=0$ respectively. In the
bottom panel, we vary the coupling $p_{2}$ taking values
$p_{2}=0.1,\,0.5,\,0.8,\,1.2$ while we fixed $\chi^{(1)}=2$. Figure 4: Here we
plotted the spectral density for non-zero fermion mass $m_{\psi}=1/4$ and
charge $q=1$ with the background parameters $T/\mu=0.02,\,0.009,\,0.001$ (top
to bottom), $k_{1}/\mu=0.8$, $k_{2}/\mu=0$, $m_{\phi}^{2}=0$ and
$\chi^{(1)}=2$ respectively. We fixed the coupling parameters $p_{2}=0.3$ and
vary $p_{1}=-0.2,\,-0.3,\,-0.4$.

##  5 Conclusion

In this paper, we have studied the fermion spectral function
$A(\omega,\vec{k})$ in anisotropic Q-lattice background where, the probe
fermion is coupled with two non-minimal dipole-type couplings. We discussed
some of the interesting features produced by the couplings. Our observation of
the spectral function from the numerics of our holographic models seems to
encode the unconventional properties that were observed in real condensed
matter systems, such as the gapless spectra in the energy-momentum dispersion
and the presence of Fermi arcs, also the appearance of doubled FS connecting
the two nodes seen in Dirac -Weyl semimetals and topological insulators.
Though we are not in anyway claiming that our models to represent the exact
match with actual condensed matter systems, however we hope that in future,
this kind of holographic approach might give us some hints in understanding
the strongly coupled systems.

## Acknowledgements

The author would like to thank Dr S. Chakrabarti and Dr D. Maity for their
illuminating discussions and comments on the draft of this work.

## References

  * (1) J. Sadeghi, B. Pourhassan and S. Heshmatian, Adv. High Energy Phys. 2013 (2013), 759804
  * (2) J. M. Maldacena, _The Large $N$ limit of superconformal field theories and supergravity_, _Int. J. Theor. Phys._ 38 (1999) 1113 [_Adv. Theor. Math. Phys._ 2 (1998) 231].
  * (3) E. Witten, _Anti-de Sitter space and holography_ , _Adv. Theor. Math. Phys._ 2 (1998) 253,
  * (4) S. Bhattacharyya, V. E. Hubeny, S. Minwalla and M. Rangamani, _Nonlinear Fluid Dynamics from Gravity_ , _JHEP_ 02 (2008) 045.
  * (5) T. Faulkner, H. Liu, J. McGreevy and D. Vegh, _Emergent quantum criticality, Fermi surfaces, and AdS(2)_ , _Phys. Rev. D._ 83 (2011) 125002.
  * (6) M. Cubrovic, J. Zaanen and K. Schalm, _String Theory, Quantum Phase Transitions and the Emergent Fermi-Liquid_ , _Science_ 325 (2009) 439.
  * (7) C. P. Herzog, P. Kovtun, S. Sachdev and D. T. Son, _Quantum critical transport, duality, and M-theory_ , _Phys. Rev. D._ 75 (2007) 085020.
  * (8) S. A. Hartnoll, C. P. Herzog and G. T. Horowitz, _Building a Holographic Superconductor_ , _Phys. Rev. Lett._ 101 (2008) 031601.
  * (9) S. A. Hartnoll, C. P. Herzog and G. T. Horowitz, _Holographic Superconductors_ , _JHEP_ 12 (2008) 015,
  * (10) G. T. Horowitz, _Introduction to Holographic Superconductors_ , _Lect. Notes Phys._ 828 (2011) 313.
  * (11) H. Liu, J. McGreevy and D. Vegh, _Non-Fermi liquids from holography_ , _Phys. Rev. D_ 83 (2011) 065029.
  * (12) S. S. Lee, _A Non-Fermi Liquid from a Charged Black Hole: A Critical Fermi Ball_ , _Phys. Rev. D_ 79 (2009) 086006.
  * (13) T. Faulkner, G. T. Horowitz, J. McGreevy, M. M. Roberts and D. Vegh, _Photoemission ‘experiments’ on holographic superconductors_ , _JHEP_ 03 (2010) 121.
  * (14) D. Guarrera and J. McGreevy, _Holographic Fermi surfaces and bulk dipole couplings_ , arXiv:1102.3908 [hep-th].
  * (15) T. Faulkner, N. Iqbal, H. Liu, J. McGreevy and D. Vegh, _Holographic non-Fermi liquid fixed points_ , _Phil. Trans. Roy. Soc. A_ 369 (2011) 1640.
  * (16) B. W. Langley, G. Vanacore and P. W. Phillips, JHEP 10, 163 (2015) doi:10.1007/JHEP10(2015)163
  * (17) G. Vanacore, S. T. Ramamurthy and P. W. Phillips, _Evolution of Holographic Fermi Arcs from a Mott Insulator_ , _JHEP_ 1809 (2018) 009.
  * (18) M. Edalati, R. G. Leigh and P. W. Phillips, _Dynamically Generated Mott Gap from Holography_ , _Phys. Rev. Lett._ 106 (2011) 091602.
  * (19) M. Edalati, R. G. Leigh, K. W. Lo and P. W. Phillips, _Dynamical Gap and Cuprate-like Physics from Holography_ , Phys. Rev. D 83 (2011)
  * (20) J. Alsup, E. Papantonopoulos, G. Siopsis, K. Yeter, _Duality between zeroes and poles in holographic systems with massless Fermions and a dipole coupling_ ,_Phys. Rev. D_ 90 (2014) .
  * (21) A. Damascelli, Z. Hussain and Z.-X. Shen, _Angle-resolved photoemission studies of the cuprate superconductors_ , _Rev. Mod. Phys._ 75 (2003) 473.
  * (22) P. D. C. King, J. A. Rosen, W. Meevasana, A. Tamai, E. Rozbicki, R. Comin, G. Levy, D. Fournier, Y. Yoshida, H. Eisaki, K. M. Shen, N. J. C. Ingle, A. Damascelli and F. Baumberger, _Structural origin of apparent Fermi surface pockets in angle-resolved photoemission of Bi 2Sr2-xLaxCuO6+δ_, _Phys. Rev. Lett._ 106 (2011) 127005.
  * (23) M. R. Norman, H. Ding, M. Randeria, J. C. Campuzano, T. Yokoya, T. Takeuchi, T. Takahashi, T. Mochiku, K. Kadowaki, P. Guptasarma and D. G. Hinks, _Destruction of the Fermi surface in underdoped high- $T_{c}$ superconductors_, _Nature_ 392 (1998) 157.
  * (24) T. Yoshida, X. J. Zhou, K. Tanaka, W. L. Yang, Z. Hus- sain, Z. X. Shen, A. Fujimori, S. Sahrakorpi, M. Lin- droos, R. S. Markiewicz, A. Bansil, S. Komiya, Y. Ando, H. Eisaki, T. Kakeshita and S. Uchida, _Doping Evolution of the Underlying Fermi Surface in La 2-xSrxCuO4_, _Phys. Rev. B_ 74 (2006) 224510.
  * (25) S. Cremonini, L. Li and J. Ren, _Holographic Fermions in Striped Phases,_ _JHEP_ 12 (2018) 080.
  * (26) Y. Seo, G. Song, Y. H. Qi and S. J. Sin, _Mott transition with Holographic Spectral function,_ _JHEP_ 08 (2018) 077.
  * (27) Y. Wu, N. H. Jo, L-L Wang, C. A. Schmidt, K. M. Neilson, B. Schrunk, P. Swatek, A. Eaton, S. L. Bud’ko, P. C. Canfield and A. Kaminski, _Fragility of Fermi arcs in Dirac semimetals_ , _Phys. Rev. B_ 99 (2019).
  * (28) Yang, L., Liu, Z., Sun, Y. et al. _Weyl semimetal phase in the non-centrosymmetric compound TaAs_. Nature Phys 11, 728-732 (2015).
  * (29) E. Brillaux and A. A. Fedorenko, Phys. Rev. B 103 (2021) doi:10.1103/PhysRevB.103.L081405
  * (30) Su-Yang Xu et al., _Discovery of a Weyl Fermion semimetal and topological Fermi arcs ,_ _Science_ 349, (2015) 613.
  * (31) Su-Yang Xu et al., _Observation of Fermi arc surface states in a topological metal ,_ _Science_ 347, (2015) 10.1126/science.1256742.
  * (32) Noam Morali et al., _Fermi-arc diversity on surface terminations of the magnetic Weyl semimetal ,_ _Science_ 365, (2019) 10.1126/science.aav2334.
  * (33) Peng Zhang et al., _Observation of topological superconductivity on the surface of an iron-based superconductor ,_ _Science_ 360, (2018) 10.1126/science.aan4596.
  * (34) M. Kargarian, M. Randeria and Y-Mi Lu, _Are the surface Fermi arcs in Dirac semimetals topologically protected? ,_ _PNAS_ 113 (2016) 8648.
  * (35) M. Z. Hasan and C. L. Kane, _Colloquium: Topological insulators_ , _Rev. Mod. Phys._ 82 (2010) 3045.
  * (36) B. Yan and C. Felser, _Topological Materials: Weyl Semimetals,_ _Ann. Rev. Cond. Matt. Phys._ 8 (2017) 337.
  * (37) X. Wan, A. M. Turner, A. Vishwanath, and S. Y. Savrasov, _Topological semimetal and Fermi-arc surface states in the electronic structure of pyrochlore iridates,__Phys. Rev. B_ 83 (2011) 205101.
  * (38) B. Lv, T. Qian and H. Ding, _Angle-resolved photoemission spectroscopy and its application to topological materials,_ _Nat Rev Phys_ 1 (2019) 609-626.
  * (39) M. R. Norman, A. Kanigel, M. Randeria, U. Chatterjee and J. C. Campuzano, _Modeling the Fermi arc in underdoped cuprates_ , _Phys. Rev. B_ 76 (2007) 174501.
  * (40) K.-Y. Yang, T. M. Rice and F.-C. Zhang, _Phenomenological theory of the pseudogap state_ , _Phys. Rev. B_ 73 (2006) 174501.
  * (41) S. Hong and P. Phillips, _Towards the standard model for Fermi arcs from a Wilsonian reduction of the Hubbard model_ , _Phys. Rev. B_ 86 (2012) 115118.
  * (42) T. D. Stanescu and G. Kotliar, _Fermi arcs and hidden zeros of the Green’s function in the pseudogap state_ , _Phys. Rev. B_ 74 (2006) 125110.
  * (43) S. A. Hartnoll, _Lectures on holographic methods for condensed matter physics_ , _Class. Quant. Grav._ 26 (2009) 224002, [arXiv:0903.3246 [hep-th]].
  * (44) C. P. Herzog, _Lectures on Holographic Superfluidity and Superconductivity_ , _J. Phys. A_ 42 (2009) 343001, [arXiv:0904.1975 [hep-th]].
  * (45) J. McGreevy, _Holographic duality with a view toward many-body physics_ , _Adv. High Energy Phys._ 2010 (2010) 723105, [arXiv:0909.0518 [hep-th]].
  * (46) J. Zaanen, Y-W Sun, and Y. Liu and K. Schalm, _Holographic duality in Condensed Matter Physics_ ; _Cambridge Univ. Press_ (2015)
  * (47) S. A. Hartnoll, A. Lucas and S. Sachdev, _Holographic Quantum Matter_ ; _MIT Press_ (2018).
  * (48) D. Vegh, _Fermi arcs from holography_ , arXiv:1007.0246 [hep-th].
  * (49) F. Benini, C. P. Herzog and A. Yarom, _Holographic Fermi arcs and a $d$-wave gap_, _Phys. Lett. B_ 701 (2011) 626.
  * (50) A. Donos and J. P. Gauntlett, _Holographic Q-lattices_ , JHEP 04, 040 (2014).
  * (51) L. N. Trefethen, _Spectral methods in MATLAB_ , _SIAM_ , Philadelphia, (2000).
  * (52) J. P. Boyd, _Chebyshev and Fourier spectral methods,_ Lecture Notes in Engineering, _S_ pringer (1989).
  * (53) T. Andrade, _Holographic Lattices and Numerical Techniques_ , arXiv:1712.00548 [hep-th].
  * (54) A. Krikun, _Numerical Solution of the Boundary Value Problems for Partial Differential Equations. Crash course for holographer_ , arXiv:1801.01483 [hep-th].
  * (55) S. Chakrabarti, D. Maity and W. Wahlang, _Studying the holographic Fermi surface in the scalar induced anisotropic background_ , Phys. Lett. B 827 (2022)
  * (56) S. Chakrabarti, D. Maity and W. Wahlang, _Probing the Holographic Fermi Arc with scalar field: Numerical and analytical study_ , _JHEP_ 07 (2019) 037.
  * (57) Y. Ling , P. Liu , C. Niu ,J-P. Wu and, Z-Y. Xian, _Holographic Fermionic system with dipole coupling on Q-lattice_ , _JHEP_ 12 (2014) 149.
  * (58) A. Iliasov, A. A. Bagrov, M. I. Katsnelson and A. Krikun, _Anisotropic destruction of the Fermi surface in inhomogeneous holographic lattices,_ _JHEP_ 01 (2020) 065.
  * (59) S. S. Gubser and J. Ren, _Analytic fermionic Green’s functions from holography_ , Phys. Rev. D 86 (2012), 046004 doi:10.1103/PhysRevD.86.046004
  * (60) S. Cremonini, L. Li and J. Ren, _Spectral Weight Suppression and Fermi Arc-like Features with Strong Holographic Lattices_ , JHEP 09 (2019), 014 doi:10.1007/JHEP09(2019)014
  * (61) F. Balm, A. Krikun, A. Romero-Bermúdez, K. Schalm and J. Zaanen, _Isolated zeros destroy Fermi surface in holographic models with a lattice_ , JHEP 01 (2020), 151 doi:10.1007/JHEP01(2020)151
  * (62) Su-Yang Xu et al., _Exactly solvable model of Fermi arcs and pseudogap,_ _Phys. Rev. B_ 103, 024529 (2021).
  * (63) R. Rodgers, E. Mauri, U. Gürsoy and H. T. C. Stoof, JHEP 11 (2021), 191 doi:10.1007/JHEP11(2021)191
  * (64) K. Landsteiner, Y. Liu and Y. W. Sun, Sci. China Phys. Mech. Astron. 63 (2020) no.5, 250001 doi:10.1007/s11433-019-1477-7
  * (65) E. Oh, Y. Seo, T. Yuk and S. J. Sin, JHEP 01 (2021), 053 doi:10.1007/JHEP01(2021)053