# An Incremental Update Framework for Online Recommenders with Data-Driven
Prior

Chen Yang  JD.com<EMAIL_ADDRESS>,  Jin Chen🖂 UESTC
<EMAIL_ADDRESS>,  Qian Yu  JD.com<EMAIL_ADDRESS>,  Xiangdong Wu
JD.com<EMAIL_ADDRESS>,  Kui Ma  JD.com<EMAIL_ADDRESS>,  Zihao Zhao
JD.com<EMAIL_ADDRESS>,  Zhiwei Fang  JD.com<EMAIL_ADDRESS>,  Wenlong
Chen  JD.com<EMAIL_ADDRESS>,  Chaosheng Fan  JD.com
<EMAIL_ADDRESS>,  Jie He  JD.com<EMAIL_ADDRESS>,  Changping Peng
JD.com<EMAIL_ADDRESS>,  Zhangang Lin  JD.com<EMAIL_ADDRESS>and
Jingping Shao  JD.com<EMAIL_ADDRESS>
(2018; 2023)

###### Abstract.

Online recommenders have attained growing interest and created great revenue
for businesses. Given numerous users and items, incremental update becomes a
mainstream paradigm for learning large-scale models in industrial scenarios,
where only newly arrived data within a sliding window is fed into the model,
meeting the strict requirements of quick response. However, this strategy
would be prone to overfitting to newly arrived data. When there exists a
significant drift of data distribution, the long-term information would be
discarded, which harms the recommendation performance. Conventional methods
address this issue through native model-based continual learning methods,
without analyzing the data characteristics for online recommenders. To address
the aforementioned issue, we propose an incremental update framework for
online recommenders with Data-Driven Prior (DDP), which is composed of Feature
Prior (FP) and Model Prior (MP). The FP performs the click estimation for each
specific value to enhance the stability of the training process. The MP
incorporates previous model output into the current update while strictly
following the Bayes rules, resulting in a theoretically provable prior for the
robust update. In this way, both the FP and MP are well integrated into the
unified framework, which is model-agnostic and can accommodate various
advanced interaction models. Extensive experiments on two publicly available
datasets as well as an industrial dataset demonstrate the superior performance
of the proposed framework.

CTR Prediction, Incremental Update, Data-Driven Prior

††copyright: acmcopyright††journalyear: 2018††doi:
XXXXXXX.XXXXXXX††conference: Make sure to enter the correct conference title
from your rights confirmation emai; June 03–05, 2018; Woodstock, NY††price:
15.00††isbn: 978-1-4503-XXXX-X/18/06††journalyear: 2023††copyright:
acmlicensed††conference: Proceedings of the 32nd ACM International Conference
on Information and Knowledge Management; October 21–25, 2023; Birmingham,
United Kingdom††booktitle: Proceedings of the 32nd ACM International
Conference on Information and Knowledge Management (CIKM ’23), October 21–25,
2023, Birmingham, United Kingdom††price: 15.00††doi:
10.1145/3583780.3615456††isbn: 979-8-4007-0124-5/23/10††ccs: Information
systems Recommender systems

##  1\. Introduction

With the rapid development of Web applications, recommender systems have
become ubiquitous to solve the serious information overload problem, with the
aim of locating potentially preferred items among numerous items for users
(Chen et al., 2022c, a, b; Lian et al., 2020). Many e-commerce companies have
generated significant revenue from recommender systems, where a higher click-
through rate corresponds to a higher probability of greater benefit. Thus, CTR
prediction plays a vital role in nowadays online recommendations.

In recent years, in order to better fit in the large-scale target data,
expressive models are proposed to capture the complex interactions between
multiple features (Song et al., 2019; Liu et al., 2020), where the models tend
to go more complex to fully capture the multiple high-order combinatorial
features. However, these models require massive computational resources under
the tremendous number of users and items, preventing quick online deployment
and updates for industrial online recommendations. A mainstream framework is
to feed models with the incremental data, i.e., newly arrived data within the
sliding window, to continuously train the latest model rather than training
from scratch (Wang et al., 2020a; Cai et al., 2022). This strategy
significantly reduces time overhead and adapts to dynamics in online data
distribution.

However, when the distribution of online data changes significantly, this
framework is prone to overfitting recent data. For example, considering the
scenarios of large promotions, such as Double 11 shopping carnival and Black
Friday, the specific commodities will receive a lot of exposure in a short
period, resulting in a different distribution from previously collected
feedback. The training framework would suffer from this and be inclined to
these recent exposures. As a result, the model pays too much attention to the
newly arrived data and gradually ignores the long-term information about user
interests, limiting the recommendation performance. Moreover, due to the
relatively decreased exposure ratio of long-tail items, the model pays less
attention to these items, which aggravates the long-tail effect.

Existing studies apply continual learning to alleviate this issue for online
recommenders. IncCTR (Wang et al., 2020a) adopts distillation logit to guide
the latest model while ASMG (Peng et al., 2021) incorporates the meta-learners
to fit in recent models. To summarize, these studies tackle the issue by
utilizing model-based prior to directly predicting CTR for each instance.
However, these studies adopt strategies inspired by conventional continual
learning, without analyzing the data characteristic in online recommendations.
The extreme data sparsity and feature diversity are two prominent
characteristics. Under the numerous numbers of users and items, user clicks
are extremely sparse, which makes it difficult to accurately estimate user
preference. Recent research (Wu et al., 2021) suggests that each item requires
around 10,000 impressions for convergence. Limited impressions hinder robust
estimation during incremental updates. The recent success of CTR prediction
can be attributed to the use of complex features, where tail items with
similar features to popular items obtain more accurate estimates.
Additionally, considering that features are also the most important factor
affecting the model effect (Nadler and Coifman, 2005; Wang et al., 2022), we
are motivated to explicitly incorporate the feature prior to enhancing the
performance.

To this end, we propose a robust and unified incremental learning framework
with data-driven prior(DDP) to improve the performance under the nowadays
mainstream training framework. It incorporates the feature prior in an end-to-
end manner and provides a more theoretically provable model prior. More
concretely, the Feature Prior is intended to explicitly estimate the average
CTR of the specific feature value. The distribution of CTR value at the
feature granularity presents more stable than the distribution at the instance
level, since the data are more aggregated for the features. FP finnaly acts as
auxiliary feature information and provides a more stable learning for model
updates, thus benefitting the optimization for long-tail items. Furthermore,
depending on the Bayes rules, we develop the Model Prior, which approximates
the posterior on the complete data by maximizing the likelihood function on
the incremental data and lowering the distance to the prior model.
Accordingly, the output from previous models can be easily integrated into the
framework to achieve the model prior, where the output of the previous model
is used to supervise the current model.

Our main contribution can be summarized as follows:

  * •

We propose a robust and unified incremental update framework with data-driven
prior to alleviate the overfitting problem on recent data, which is model-
agnostic and can be optimized in an end-to-end manner.

  * •

FP is designed to estimate the average CTR value for features, benefitting
from a more stable distribution at the feature-level. This module can help CTR
model more stable training and more accurate CTR estimation on long-tail
items.

  * •

MP approximates the posterior on the complete data by optimizing the
likelihood function on the incremental data and reducing the distance to the
previous model.

  * •

Exhaustive experiments are conducted on two public dataset to demonstrate the
superiority of the proposed framework. Both offline and online experiments are
conducted on an industrial dataset schow the effectiveness in real scenarios.

##  2\. Related Work

###  2.1. CTR Prediction Models

Click-Through Rate (CTR) prediction plays an important role for online
recommenders, such as online advertising, and numerous studies have
investigated it in recent decade (Cheng et al., 2016; Wang et al., 2017; Guo
et al., 2017; Pan et al., 2018; Song et al., 2019). These methods mainly vary
in the way of capturing the feature interactions. Wide&Deep (Cheng et al.,
2016) firstly combines the shallow linear layers and deep neural networks to
simultaneously capture feature expression at different levels of features.
Motived by Wide&Deep, DeepFM (Guo et al., 2017) replaces the wide part with
the factorization machine models to better capture the second-order
interactions between features. DCN (Wang et al., 2017) explicitly constructs
high-level combinatorial relations of features through Cross Network. FwFM
(Pan et al., 2018) effectively captures the heterogeneity of field on second-
order interaction features. AutoInt (Song et al., 2019) automatically learn
the high-order feature interactions of input features with a multi-head self-
attention neural network to explicitly model feature interactions in a low-
dimensional space. Another mainstream direction is to model the historical
behavior of users to better figure out user interests. DIN (Zhou et al., 2018)
obtains the user’s interest in the target item by modeling the interest
distribution of a user’s browsing sequence under a given target item. Based on
DIN, DIEN (Zhou et al., 2019) investigates how user interests fluctuate over
time and improves user interest capture by simulating the evolution of
interests over time in the user sequence.

###  2.2. Incremental Update for Online Recommenders

Limited by the computational time and resources, the models are trained with
streaming data, resulting in the incremental update for online recommendation.
This incremental training framework has been widely used for industrial
scenarios and attracted growing interest in recent years (Wang et al., 2020a;
Cai et al., 2022; Peng et al., 2021; Guan et al., 2022; Mi et al., 2020; Xia
et al., 2022).

Here we briefly introduce a few papers that are most relevant to our work.
IncCTR (Wang et al., 2020a) uses the previous model as a teacher and
supervises the learning of the current model through the commonly-used
distillation loss. Motivated by (Wang et al., 2020a), ReLoop (Cai et al.,
2022) uses the hinge loss to guide the current model to perform better than
the previous model by treating the performance of the previous model as the
margin value. ASMG (Peng et al., 2021) draws the idea of meta-learning and
generates a better serving model from a sequence of historical models via a
meta generator, so as to overcome the problem of overfitting and forgetting.

##  3\. Methodology

###  3.1. Preliminaries

Assume each instance $(\mathcal{X},y)$ records a piece of impression, where
$\mathcal{X}$ denotes features with $M$ fields, and $y\in\\{0,1\\}$ indicates
whether the user clicks the item. After one-hot or multi-hot encoding for the
features, each instance can be converted into
$\mathbf{x}=[\mathbf{x}_{1},...,\mathbf{x}_{M}]$ where $\mathbf{x}_{j}$
denotes the encoding vector for the $j$-th field. The CTR-prediction model
then learns a mapping $\hat{y}=f_{\theta}(\mathbf{x})$ to estimate the
probability of user click. Previous models can be summarized with three parts:
embedding module, interaction module and loss function. The embedding module
convert encoding into embedding vector
$\mathbf{e}=[\mathbf{e}_{1},...,\mathbf{e}_{M}]$, where
$\mathbf{e}_{j}\in\mathbb{R}^{d}$ is obtained through the embedding matrix
$\bm{E}\in\mathbb{R}^{N\times d}$. $N$ denotes the number of sparse feature
values. The interaction module attempts to capture the complex relationships
between features and to predict the probability of the user click through the
sigmoid function, i.e.,
$\hat{y}=f_{\theta}(\mathbf{x})=\sigma\left(\text{Interaction\\_Module}(\mathbf{e})\right)$.
$\theta$ denotes the set of parameters and $f(\cdot)$ summarizes all
calculations. Finally, the model is optimized through the binary cross entropy
loss $\mathcal{L(\mathcal{D};\theta)}$.

Considering the industrial scenarios under large scales, the incremental
update becomes a popular framework, where the model is updated only with newly
arrived data $\mathcal{D}_{t}$. $\mathcal{D}_{t}$ denotes the data collected
from the period $t$, whereby a period can be a certain length of time (e.g.,
an hour, a day).
$\mathcal{H}_{t}=\mathcal{D}_{1}\cup\mathcal{D}_{2},...,\cup\mathcal{D}_{t}$
denotes the historical training data up to the period $t$. Correspondingly,
$\theta_{t-1}$ denotes model parameters updated in time $t-1$. The incremental
update receives the newly arrived data $\mathcal{D}_{t}$ and previous model
$\theta_{t-1}$, updating the model parameters, i.e.,
$\min\mathcal{L}(\mathcal{D}_{t},\theta_{t-1};\theta)$. It is obvious that the
models are prone to overfitting to recent data when there is a drift of online
data, and long-term user interest would be discarded under the loop of the
incremental update.

Figure 1. The Architecture of DDP.

###  3.2. Framework Overview

We propose an unified framework, namely incremental update framework for
online recommenders with Data-Driven Prior (DDP), which consist of two
important components, as shown in Figure 1:

(1) Feature Prior (FP) estimates the average CTR of the specific value of each
feature field, motivated by a more stable distribution at the feature level
rather than the frequent dynamics at the instance level. The objective of FP
is to help model learn more robust and have more accurate CTR estimates with
long-tail data.

(2) Model Prior (MP) provides a more robust update depending on the Bayes
rules, which acts as a regularizer to minimize the distance between the
incremental update and the training with whole data. The purpose of MP is to
assist the model in learning stably and avoiding over-fitting on incremental
data.

These two data-driven parts can be easily integrated into existing advanced
models, resulting in a model-agnostic and general framework. Furthermore, the
framework can be updated in an end-to-end manner, showing its feasibility for
online recommenders.

Figure 2. KL-Distance w.r.t. statistical CTR of seven consecutive days. The
KL-Distance measures the distance between the CTR value within the day and
average value.

###  3.3. Feature Prior (FP)

Previous studies adopt the native continual learning methods, by implementing
the model-based information to predict the CTR for each instance. A major
concern is the extreme data sparsity aggravated by incremental update. Model
parameters are considerably sensitive to these data, where sparse data raises
the instability of models, resulting in the overfitting to recent data.

Intuitively, the feature data among the instances are more frequent, which
acts more stable than instances. To further demonstrate it, we summarize the
change of CTR values within seven consecutive days at the feature-level and
instance-level in Figure 2. Specifically, we select three features and
corresponding instances. Feature C is sparser than Feature A and Feature B.
The KL-distance is calculated with
$KL=\sum_{i=0}^{1}q(i)\log\frac{q(i)}{p(i)}$, where $q(i)$ and $p(i)$ denote
the statistical CTR of features (instances) within the day and average value
respectively. As shown in Figure 2, the change of CTR with respect to the
instances behaves more frequently than that of features, which brings
difficulties for the update. Thus, we are motivated to design a module to
estimate the CTR values for the features, which is fed into CTR model as
stable and useful information, with the aim of improving the performance of
recommenders. This feature-level value can generalize to tailed items,
allowing CTR models to estimate long-tail characteristics more accurately. To
this end, we propose the Feature Prior (FP), which can maintain long-term
prior information for each feature, enabling a more stable expression of long-
tail characteristics.

As aforementioned, each instance is represented with the one-hot encoding or
multi-hot encoding:
$\mathbf{x}=[\mathbf{x}_{1},\mathbf{x}_{2},...,\mathbf{x}_{M}]$ where $M$
denotes feature fields and $\mathbf{x}_{i}$ denotes the feature encoding of
the field $i$. Notice all features are transformed into one-hot vectors here
for simplicity. Apart from the dense high-dimensional embedding matrix, we
design a feature prior layer to estimate the average CTR of each feature value
as follows:

$\hat{s}_{i}=\sigma(\mathbf{C}^{\top}\mathbf{x}_{i})$

where $\mathbf{C}\in\mathbb{R}^{N\times 1}$ denotes the feature prior
embedding vector of all sparse features, $N$ denotes the number of sparse
feature values and $\sigma$ is the sigmoid function. A binary cross entropy
loss is applied to update the embedding vectors by maximum likelihood
estimation:

(1)
$\footnotesize\mathcal{L}(\mathbf{x},y;\Phi)=-\mathbb{E}_{{\mathbf{x},y}\sim\mathcal{D}_{t}}\left[\frac{1}{M}\sum_{i}^{M}y\log(\hat{s}_{i})+(1-y)\log(1-\hat{s}_{i})\right]$

where $y$ denotes the label of the instance, $\mathcal{D}_{t}$ denotes
incremental data and $\Phi$ denotes the parameters of the feature prior layer.
Based on Eq (1), Feature Prior Loss, we can estimate the CTR values for each
feature, which further aids the robust training.

Thus, we obtain a continuous estimated CTR value for each specific value. In
order to integrate with the general embedding layers, i.e, the original
feature values, the discretization strategy is adopted here, i.e.,
$\mathbf{s}_{i}=\mathrm{One\\_Hot}(\lfloor B\cdot\sqrt{\hat{s_{i}}}\rfloor)$,
where $B$ denotes the number of bins with equal width for discretization.
Since the statistics CTR value is closer to 0 in real scenarios, the square
root function is used to lead the model to focus more on smaller values.

After discretizing the estimated values, another embedding matrix
$\bm{U}_{i}\in\mathbb{R}^{B\times d}$ is used to get the encoding vector for
the $i$-th feature field. Specifically,
$\bm{o}_{i}=\bm{U}_{i}^{\top}\mathbf{s}_{i}$. Thus, the low-dimensional dense
embedding for the feature prior follows as:

$\mathbf{o}=[\mathbf{o}_{1},\mathbf{o}_{2},...,\mathbf{o}_{M}]$

By concatenating vectors $\mathbf{e}$ and $\mathbf{o}$, we obtain the final
embedding representation of features and their prior:

$\mathbf{e}^{\prime}=\mathrm{Concat}([\mathbf{e},\mathbf{o}])=[\mathbf{e}_{1},...,\mathbf{e}_{M},\mathbf{o}_{1},...,\mathbf{o}_{M}]$

where $\mathrm{Concat}(\cdot)$ denotes concat function. In this way, the
feature prior is well integrated with the original embedding module, followed
by any interaction module to capture the interactions between different
features.

###  3.4. Model Prior (MP)

Empirically, the main reason for the model parameters overfitting to the
incremental data distribution is that the model parameter optimization only
considers the likelihood of the model parameter distribution on the
incremental data, while neglecting the influence of previous data. Therefore,
we conduct an in-depth analysis of the model optimization process, and finally
simplify it to make our parameters fully learn the complete data distribution
$\mathcal{H}_{t}=\\{\mathcal{H}_{t-1},\mathcal{D}_{t}\\}$ by fitting the
likelihood on $\mathcal{D}_{t}$ and constraining the distance between the
output of current model and the output of previous model $\theta_{t-1}$, which
is optimized by posterior with $\mathcal{H}_{t-1}$.

From a probabilistic perspective, our optimization goal is to optimize
posterior parameter distribution $\theta_{t}$ given the complete data
$\mathcal{H}_{t}$:

$\displaystyle\theta_{t}$ $\displaystyle=\mathop{\mathrm{argmax}}_{\theta}\log
p(\theta|\mathcal{H}_{t})$
$\displaystyle=\mathop{\mathrm{argmax}}_{\theta}\left(\log
p(\mathcal{H}_{t}|\theta)+\log p(\theta)-\log p(\mathcal{H}_{t})\right)$
$\displaystyle=\mathop{\mathrm{argmax}}_{\theta}\Big{(}\log
p(\mathcal{D}_{t}|\theta)+\log p(\mathcal{H}_{t-1}|\theta)$
$\displaystyle\qquad\qquad\qquad+\log p(\theta)-\log p(\mathcal{H}_{t-1})-\log
p(\mathcal{D}_{t})\Big{)}$
$\displaystyle=\mathop{\mathrm{argmax}}_{\theta}\left(\log
p(\mathcal{D}_{t}|\theta)+\log p(\theta|\mathcal{H}_{t-1})-\log
p(\mathcal{D}_{t})\right)$ (2)
$\displaystyle=\mathop{\mathrm{argmax}}_{\theta}\left(\log
p(\mathcal{D}_{t}|\theta)+\log p(\theta|\mathcal{H}_{t-1})\right)$

According to Eq (2), the objective is to maximize the likelihood $\log
p(\mathcal{D}_{t}|\theta)$ and the posterior estimation $\log
p(\theta|\mathcal{H}_{t-1})$ given the historical training data. Next we
introduce the solutions respectively. Regarding the likelihood
$p(\mathcal{D}_{t}|\theta)$, by assuming that the user click follows a
binomial distribution with the probability of $f_{\theta}(\mathbf{x})$, the
objective function deduces to the binary cross-entropy loss:

(3)
$\mathcal{L}_{l}=-\mathbb{E}_{{\mathbf{x},y}\sim\mathcal{D}_{t}}\left[y\log(f_{\theta}(\mathbf{x}))+(1-y)\log(1-f_{\theta}(\mathbf{x}))\right]$

where $y\in\\{0,1\\}$ denotes the click label and $f$ summarizes the model
output. $f_{\theta}$ denotes the output parameterized by $\theta$. This
objective function guides the model to fully fit the incremental data
$\mathcal{D}_{t}$.

Regarding the posterior $p(\theta|\mathcal{H}_{t-1})$, it is intractable to
directly obtain the solution. EWC (Kirkpatrick et al., 2017) adopts the
Laplace approximation to regularize the distance between parameter space
$\theta$ and the optimal parameter space $\theta_{t-1}$ obtained given the
historical training data $\mathcal{H}_{t-1}$, which acts as a surrogate loss
to optimize $\log p(\theta|\mathcal{H}_{t-1})$. MRNFS (Benjamin et al., 2018)
further simplifies the parameter space into the output of the models, with the
aim of more stable learning to alleviate the strong constraints at the
parameter level, achieving better performances. Thus, we follow these studies
and get the following objective function:

(4) $\displaystyle\mathcal{L}_{p}$
$\displaystyle=\mathbb{E}_{\mathbf{x}\sim\mathcal{D}_{t}}|f_{\theta}(\mathbf{x})-f_{\theta_{t-1}}(\mathbf{x})|^{2}$

where $f_{\theta}$ stands for current learning model and $f_{\theta_{t-1}}$
denotes the previous well-learned model given $\mathcal{H}_{t-1}$.

Finally, the objective function follows:

(5)
$\mathcal{L}(\mathbf{x},\mathbf{s},y,\theta_{t-1};\Theta)=\mathcal{L}_{l}+\frac{\lambda}{2}\mathcal{L}_{p}$

where $\lambda$ is the coefficient to regulate the relationship between the
two losses, $\mathbf{s}=[\mathbf{s}_{1},\mathbf{s}_{2},...,\mathbf{s}_{M}]$
denotes the output of feature prior layer and $\Theta$ denotes the other
parameters except feature prior layer. By integrating with Eq (1) of the
Feature Prior (FP), we can update the whole framework in an end-to-end manner,
without changing the structure of the important interaction modules. The whole
process is detailed in Alg 1. Note that if we optimize Eq (1) with the Adam
optimizer (Kingma and Ba, 2015), the feature prior layer will overfit within a
short period due to the gradient vanish (Wilson et al., 2017), which can not
capture the change of data distribution continuously. Thus, we use the SGD
optimizer to update the feature prior layer while using Adam optimizer for
other parts.

Input: Incremental Data $\mathcal{D}_{t}=\\{\mathbf{x},y\\}$, The latest model
parameters $\theta_{t-1}$

Output: $\theta=\\{\Phi,\Theta\\}$

// $\Phi$ is the parameter of feature prior layer and $\Theta$ is the other
parameters

1 $\theta=\theta_{t-1}$;

2 for  _$\\{\mathbf{x},y\\}$ in $\mathcal{D}_{t}$_ do

3 Get $\mathbf{s}=[\mathbf{s}_{1},\mathbf{s}_{2},...,\mathbf{s}_{M}]$
according to the Feature Prior Layer;

4 Obtain $\mathcal{L}(\mathbf{x},y;\Phi)$ through Eq (1);

5 Obtain $\mathcal{L}(\mathbf{x},\mathbf{s},y,\theta_{t-1};\Theta)$ through Eq
(5);

6 Optimize $\Phi$ using $\mathcal{L}(\mathbf{x},y;\Phi)$ with SGD;

7 Optimize $\Theta$ using
$\mathcal{L}(\mathbf{x},\mathbf{s},y,\theta_{t-1};\Theta)$ with Adam;

8

Algorithm 1 DDP

##  4\. Experiments on Public Datasets

###  4.1. Experimental Settings

####  4.1.1. Datasets

Two datasets, Criteo and CIKM2019, are adopted here for comparisons, whose
statistics are summarized in Tabel 1.

  * •

Criteo111https://www.kaggle.com/c/criteo-display-ad-challenge, which is widely
used for CTR prediction, is a collection of user traffic logs from Criteo for
seven consecutive days.

  * •

CIKM2019222https://tianchi.aliyun.com/competition/entrance/231721/introduction?lang=en-
us is a public dataset from an e-commerce company. We follow the previous work
(Zhu et al., 2021) to transform the original label into binary values, where
1/0 indicates whether a user has bought an item or not. It consists of
behavior logs for 16 consecutive days.

To simulate the online settings of online industrial scenarios, the public
data are fed into models in a streaming way. More concretely, within the logs
of $T$ days, the proceeding data of $w$ days is used to warm up the model
where the model is incremental updated for the next consecutive days with the
time period as a day. Finally, the performance is tested on the last day after
$T-w-1$ trails. For Criteo, $T=7$ and $w=3$. For CIKM2019 dataset, $T=16$ and
$w=8$.

Table 1. Statistics of datasets. M represents a million. Dataset | #Instances | #Field | Time Duration
---|---|---|---
Criteo | 45.84M | 39 | 7 days
CIKM2019 | 58.75M | 8 | 16 days
Table 2. Overall performance with Baselines on the Criteo and CIKM2019
datasets. FP represents the feature prior. Notice DeepFM is chosen as the
interaction module in DDP. Boldface denotes the highest score and underline
indicates the best result of the baselines. $\star$ represents significance
level $p$-value ¡ 0.05 of comparing with the best baselines.

Dataset | Criteo | CIKM2019
---|---|---
Data Split | All | All | Long Tail
Metric | AUC $\uparrow$ | LogLoss $\downarrow$ | AUC $\uparrow$ | LogLoss $\downarrow$ | AUC $\uparrow$ | LogLoss $\downarrow$
DNN | 0.7985 | 0.4546 | 0.7588 | 0.1234 | 0.7248 | 0.1212
DCN | 0.7982 | 0.4549 | 0.7588 | 0.1234 | 0.7250 | 0.1212
W&D | 0.7985 | 0.4546 | 0.7587 | 0.1234 | 0.7249 | 0.1212
DeepFM | 0.7996 | 0.4537 | 0.7589 | 0.1234 | 0.7248 | 0.1212
FwFM | 0.7985 | 0.4546 | 0.7589 | 0.1234 | 0.7249 | 0.1212
AutoInt | 0.7986 | 0.4546 | 0.7587 | 0.1234 | 0.7247 | 0.1212
MECP | 0.8004 | 0.4531 | 0.7594 | 0.1235 | 0.7278 | 0.1209
IncCTR | 0.8007 | 0.4528 | 0.7597 | 0.1236 | 0.7275 | 0.1210
RLP | 0.8005 | 0.4531 | 0.7595 | 0.1234 | 0.7273 | 0.1209
ASMG | 0.8020 | 0.4517 | 0.7599 | 0.1234 | 0.7231 | 0.1213
FP | 0.8031 | 0.4507 | 0.7602 | 0.1232⋆ | 0.7325⋆ | 0.1204⋆
DDP | 0.8038⋆ | 0.4500⋆ | 0.7608⋆ | 0.1232⋆ | 0.7317 | 0.1205
% Improv. | 0.22% | 0.38% | 0.12% | 0.16% | 0.65% | 0.41%

####  4.1.2. Baselines

We choose the following competing methods as baselines for comparison: IncCTR
(Wang et al., 2020a), RLP (Cai et al., 2022), ASMG (Peng et al., 2021) and
MECP (Ling et al., 2017). Moreover, since the proposed DDP is a model-agnostic
framework, we apply DDP to six different deep CTR models: DNN, DCN (Wang et
al., 2017), W&D (Cheng et al., 2016), DeepFM (Guo et al., 2017), FwFM (Pan et
al., 2018) and AutoInt (Song et al., 2019) to verify the compatibility of DDP
with various interaction module.

####  4.1.3. Metric and Implementation Details

We use the commonly used metrics in CTR prediction, AUC and LogLoss, for
evaluation. All the experiments are repeated 5 times by changing the random
seeds. The models are trained within mini batch, where the learning rate is
tuned over $\\{1e-6,5e-5,...,1e-2\\}$. The weight of $L_{2}$ regularization is
tuned over $\\{1e-6,5e-5,...,1e-3\\}$. The embedding size is set to 16 and the
hidden layers for the deep networks is fixed to $[200,200,200]$. All of these
models select the parameters with best performances in terms of AUC.

###  4.2. Overall Performance

We compare the performance of different deep CTR models for incremental update
and various embedding-based deep CTR models. FP represents that only Feature
Prior is included under the DeepFM interaction module. The results of our
proposed approaches and baselines on two public datasets are shown in Table 2,
from which we have the following observations:

  * •

Our methods: FP and DDP consistently outperform other different deep CTR
methods for incremental update and various embedding-based deep CTR models.
Compared with the best basline ASMG, our method DDP achieves a relative
improvement of 0.22% and 0.12% with respect to AUC on Criteo and CIKM2019
respectively. Notice, even a slight 0.1% improvement on the CTR prediction
task is significant (Guo et al., 2017; Wang et al., 2017).

  * •

Introducing the feature prior raises more improvements on the long-tail items
than the short-hot items. We split the items into short-hot and long-tail
according to the occurrence frequency of items. The 20% items with higher
frequency are defined as short hot, and the remaining 80% are defined as long
tail. The final ratio of short-hot and long-tail items of instances is about
8:2. In general, compared with DeepFM, our DDP improves AUC 0.95% on long-tail
items and 0.11% on short hot items respectively. The reason lies in that the
FP performs more stable and obtains more general prior information, which is
more helpful to provide a more accurate estimation on tailed items with
limited training instances. This finding is consistent with the previous
conclusion that the use of prior information is a very effective method to
solve the long tail problem (Cao et al., 2020, 2021; Wang et al., 2020b).

(a) AUC (b) LogLoss

Figure 3. Performances of Consecutive Days under Incremental Update on
CIKM2019 Dataset

(a) AUC on Criteo (b) LogLoss on Criteo

Figure 4. Varying Interaction Modules

###  4.3. Performances on Consecutive Increments

Considering that the real scenarios, where models are updated with the newly
arrived data in loops, we mimic the online setting with consecutive days to
explore the performance during the training loops. Experiments are carried out
on the CIKM2019 dataset, where we train a warmed-up model using data from the
preceding 8 days. The model is incrementally updated till the data before the
test day, then tested on test day. For example, in the tenth day, the data
before the tenth day is fed into the model incrementally and the test data is
the tenth day.

As shown in Figure 3, DDP consistently outperforms the competing baseline
IncCTR and RLP on the all seven tested days in terms of both AUC and LogLoss.
This indicates the superiority of adopting the data-driven prior, which
achieves a more robust update for recommenders. In the early stages, there is
a large gap between our method and other baselines, which is primarily due to
the fact that our FP can assist to better express the feature when the
quantity of each feature is limited.

###  4.4. Varying Interaction Modules

Since the DDP is a model-agnostic framework and is easily plugged into various
deep CTR models, we conduct experiments with six different interaction modules
to verify the compatibility of the proposed method. The experimental results
are shown in Figure 4.

FP consistently improves the performance on all interaction modules, achieving
an average 0.44% improvement on AUC and 0.66% on LogLoss. DDP further improves
the performance with the additional MP. The improvements demonstrate the
capability of the proposed method being suitable to any modern interaction
modules and the superiority of alleviating the overfitting problem in
incremental update for online recommenders.

###  4.5. Deep Analysis of Feature Prior (FP)

As we claim aforementioned, the FP is optimized with SGD optimizer while MP is
updated with Adam optimizer. We conduct experiments to compare the performance
of updating FP with SGD and Adam optimizers, as shown in Figure 5. The
performance under SGD is tuned over different learning rate. The orange dashed
line represents the best performance with Adam optimizer. The grey line
represents the performance of the competing method MECP, which adopts the
previously trained CTR of features.

The results demonstrate the superiority of integrating the FP in the end-to-
end framework and we have the following observation: 1) DDP, capturing the
dynamics of features along with instances, shows better performances than
MECP, indicating the importance of constantly capturing data distribution at
the feature level. 2) SGD outperforms Adam over a range of learning rates.
This suggests that the decreasing learning rate of Adam throughout training
hinders the continuous learning of the FP module. 3) Lower SGD learning rates
lead to better performance by preserving long-term knowledge and ensuring
parameter stability for feature CTR values.

Figure 5. FP Optimized with SGD

##  5\. Experiments on Industrial Dataset

The industrial dataset is a collection of logs from the homepage streaming
data, which is from an online advertising platform of a famous e-commerce
company. It consists of 7 billion instances over 121 consecutive days. We use
the 120-day data for incremental training directly. The performance is
evaluated on the data of the last day, as reported in Table 3. DDP outperforms
the base models, achieving a relative 0.58% improvement of AUC, even though
the model has been already optimized with 7 billion logs. This supports our
point that under the environment of rapid changes in data distribution, how
maintaining the relatively stable prior information of the model and feature
is of great significance to improve the overall performance of the model.

Furthermore, we conduct online A/B test to validate the effectiveness of the
proposed framework DDP, which is deployed on a mainstream advertising position
on the homepage. There are around ten million impressions each day and we
allocate 5% of all traffic to DDP for 7 days. The results is reported in Table
3. DDP increases CTR at an average 1.99% improvement and increases eCPM
(effective cost per mille) at 2.97%, indicating significant improvements for
online recommenders.

Table 3. Experiments on the Industrial Dataset Offline | AUC $\uparrow$ | LogLoss $\downarrow$
---|---|---
Base Model | 0.7382 | 0.1208
DDP | 0.7425 (+0.58%) | 0.1199 (-0.75%)
Online | CTR Gain | eCPM Gain
DDP | +1.99% | +2.97%

##  6\. Conclusion

In this paper, we propose an incremental update framework for online
recommenders with Data-Driven Prior (DDP) to ease the overfitting problem on
incremental data, which is model-agnostic and can be learned end-to-end. The
proposed FP is intended to estimate the average CTR value for each feature,
allowing for more stable training and more accurate CTR estimates for long-
tail items. Further, the proposed MP approximates the posterior on the
complete data by optimizing the likelihood function on the incremental data
and reducing the distance with the previous model. Extensive experiments on
two public datasets demonstrate the advantages of the proposed framework,
while offline and online studies on an industrial dataset show its
effectiveness.

## References

  * (1)
  * Benjamin et al. (2018) Ari Benjamin, David Rolnick, and Konrad Kording. 2018\.  Measuring and regularizing networks in function space. In _International Conference on Learning Representations_.
  * Cai et al. (2022) Guohao Cai, Jieming Zhu, Quanyu Dai, Zhenhua Dong, Xiuqiang He, Ruiming Tang, and Rui Zhang. 2022.  ReLoop: A Self-Correction Continual Learning Loop for Recommender Systems.  _Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval_ (2022).
  * Cao et al. (2020) Ermei Cao, Difeng Wang, Jiacheng Huang, and Wei Hu. 2020\.  Open knowledge enrichment for long-tail entities. In _Proceedings of The Web Conference 2020_. 384–394.
  * Cao et al. (2021) Yixin Cao, Jun Kuang, Ming Gao, Aoying Zhou, Yonggang Wen, and Tat-Seng Chua. 2021\.  Learning relation prototype from unlabeled texts for long-tail relation extraction.  _IEEE Transactions on Knowledge and Data Engineering_ (2021).
  * Chen et al. (2022a) Jin Chen, Defu Lian, Binbin Jin, Xu Huang, Kai Zheng, and Enhong Chen. 2022a.  Fast variational autoencoder with inverted multi-index for collaborative filtering. In _Proceedings of the ACM Web Conference 2022_. 1944–1954.
  * Chen et al. (2022b) Jin Chen, Defu Lian, Binbin Jin, Kai Zheng, and Enhong Chen. 2022b.  Learning Recommenders for Implicit Feedback with Importance Resampling. In _Proceedings of the ACM Web Conference 2022_. 1997–2005.
  * Chen et al. (2022c) Jin Chen, Defu Lian, Yucheng Li, Baoyun Wang, Kai Zheng, and Enhong Chen. 2022c.  Cache-Augmented Inbatch Importance Resampling for Training Recommender Retriever.  _Advances in Neural Information Processing Systems_ 35 (2022), 34817–34830.
  * Cheng et al. (2016) Heng-Tze Cheng, Levent Koc, Jeremiah Harmsen, Tal Shaked, Tushar Chandra, Hrishi Aradhye, Glen Anderson, Greg Corrado, Wei Chai, Mustafa Ispir, et al. 2016\.  Wide & deep learning for recommender systems. In _Proceedings of the 1st workshop on deep learning for recommender systems_. 7–10.
  * Guan et al. (2022) Renchu Guan, Haoyu Pang, Fausto Giunchiglia, Ximing Li, Xuefeng Yang, and Xiaoyue Feng. 2022\.  Deployable and Continuable Meta-learning-Based Recommender System with Fast User-Incremental Updates. In _Proceedings of the 45th International ACM SIGIR Conference on Research and Development in Information Retrieval_. 1423–1433.
  * Guo et al. (2017) Huifeng Guo, Ruiming Tang, Yunming Ye, Zhenguo Li, and Xiuqiang He. 2017.  DeepFM: A Factorization-Machine based Neural Network for CTR Prediction. In _IJCAI_.
  * Kingma and Ba (2015) Diederik P Kingma and Jimmy Ba. 2015.  Adam: A Method for Stochastic Optimization. In _ICLR (Poster)_.
  * Kirkpatrick et al. (2017) James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. 2017\.  Overcoming catastrophic forgetting in neural networks.  _Proceedings of the national academy of sciences_ 114, 13 (2017), 3521–3526.
  * Lian et al. (2020) Defu Lian, Qi Liu, and Enhong Chen. 2020.  Personalized ranking with importance sampling. In _Proceedings of The Web Conference 2020_. 1093–1103.
  * Ling et al. (2017) Xiaoliang Ling, Weiwei Deng, Chen Gu, Hucheng Zhou, Cui Li, and Feng Sun. 2017\.  Model ensemble for click prediction in bing search ads. In _Proceedings of the 26th international conference on world wide web companion_. 689–698.
  * Liu et al. (2020) Bin Liu, Chenxu Zhu, Guilin Li, Weinan Zhang, Jincai Lai, Ruiming Tang, Xiuqiang He, Zhenguo Li, and Yong Yu. 2020.  Autofis: Automatic feature interaction selection in factorization models for click-through rate prediction. In _Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining_. 2636–2645.
  * Mi et al. (2020) Fei Mi, Xiaoyu Lin, and Boi Faltings. 2020.  Ader: Adaptively distilled exemplar replay towards continual learning for session-based recommendation. In _Fourteenth ACM Conference on Recommender Systems_. 408–413.
  * Nadler and Coifman (2005) Boaz Nadler and Ronald R Coifman. 2005.  The prediction error in CLS and PLS: the importance of feature selection prior to multivariate calibration.  _Journal of Chemometrics: A Journal of the Chemometrics Society_ 19, 2 (2005), 107–118.
  * Pan et al. (2018) Junwei Pan, Jian Xu, Alfonso Lobos Ruiz, Wenliang Zhao, Shengjun Pan, Yu Sun, and Quan Lu. 2018.  Field-weighted factorization machines for click-through rate prediction in display advertising. In _Proceedings of the 2018 World Wide Web Conference_. 1349–1357.
  * Peng et al. (2021) Danni Peng, Sinno Jialin Pan, Jie Zhang, and Anxiang Zeng. 2021\.  Learning an Adaptive Meta Model-Generator for Incrementally Updating Recommender Systems. In _Fifteenth ACM Conference on Recommender Systems_. 411–421.
  * Song et al. (2019) Weiping Song, Chence Shi, Zhiping Xiao, Zhijian Duan, Yewen Xu, Ming Zhang, and Jian Tang. 2019.  Autoint: Automatic feature interaction learning via self-attentive neural networks. In _Proceedings of the 28th ACM International Conference on Information and Knowledge Management_. 1161–1170.
  * Wang et al. (2017) Ruoxi Wang, Bin Fu, Gang Fu, and Mingliang Wang. 2017\.  Deep & cross network for ad click predictions.  In _Proceedings of the ADKDD’17_. 1–7.
  * Wang et al. (2020b) Weitao Wang, Meng Wang, Sen Wang, Guodong Long, Lina Yao, Guilin Qi, and Yang Chen. 2020b.  One-shot learning for long-tail visual relation detection. In _Proceedings of the AAAI Conference on Artificial Intelligence_ , Vol. 34. 12225–12232.
  * Wang et al. (2020a) Yichao Wang, Huifeng Guo, Ruiming Tang, Zhirong Liu, and Xiuqiang He. 2020a.  A practical incremental method to train deep ctr models.  _arXiv preprint arXiv:2009.02147_ (2020).
  * Wang et al. (2022) Yejing Wang, Xiangyu Zhao, Tong Xu, and Xian Wu. 2022\.  Autofield: Automating feature selection in deep recommender systems. In _Proceedings of the ACM Web Conference 2022_. 1977–1986.
  * Wilson et al. (2017) Ashia C Wilson, Rebecca Roelofs, Mitchell Stern, Nati Srebro, and Benjamin Recht. 2017.  The marginal value of adaptive gradient methods in machine learning.  _Advances in neural information processing systems_ 30 (2017).
  * Wu et al. (2021) Kailun Wu, Weijie Bian, Zhangming Chan, Lejian Ren, Shiming Xiang, Shuguang Han, Hongbo Deng, and Bo Zheng. 2021\.  Adversarial Gradient Driven Exploration for Deep Click-Through Rate Prediction.  _arXiv preprint arXiv:2112.11136_ (2021).
  * Xia et al. (2022) Jiafeng Xia, Dongsheng Li, Hansu Gu, Jiahao Liu, Tun Lu, and Ning Gu. 2022\.  FIRE: Fast Incremental Recommendation with Graph Signal Processing. In _Proceedings of the ACM Web Conference 2022_. 2360–2369.
  * Zhou et al. (2019) Guorui Zhou, Na Mou, Ying Fan, Qi Pi, Weijie Bian, Chang Zhou, Xiaoqiang Zhu, and Kun Gai. 2019.  Deep interest evolution network for click-through rate prediction. In _Proceedings of the AAAI conference on artificial intelligence_ , Vol. 33. 5941–5948.
  * Zhou et al. (2018) Guorui Zhou, Xiaoqiang Zhu, Chenru Song, Ying Fan, Han Zhu, Xiao Ma, Yanghui Yan, Junqi Jin, Han Li, and Kun Gai. 2018\.  Deep interest network for click-through rate prediction. In _Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining_. 1059–1068.
  * Zhu et al. (2021) Yongchun Zhu, Ruobing Xie, Fuzhen Zhuang, Kaikai Ge, Ying Sun, Xu Zhang, Leyu Lin, and Juan Cao. 2021.  Learning to warm up cold item embeddings for cold-start recommendation with meta scaling and shifting networks. In _Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval_. 1167–1176.


# Looking for Lorentz invariance violation (LIV) in the latest long baseline
accelerator neutrino oscillation data

Ushak Rahaman  Centre for Astro-Particle Physics (CAPP) and Department of
Physics, University of Johannesburg, PO Box 524, Auckland Park 2006, South
Africa

(Received: date / Revised version: date)

###### Abstract

In this paper, we have analysed the latest data from NO$\nu$A and T2K with the
Lorentz invariance violation along with the standard oscillation hypothesis.
We have found that the NO$\nu$A data cannot distinguish between the two
hypotheses at $1\,\sigma$ confidence level. T2K data and the combined data
analysis excludes standard oscillation at $1\,\sigma$. All three cases do not
have any hierarchy sensitivity when analysed with LIV. There is a mild tension
between the two experiments, when analysed with LIV, as ${\theta_{23}}$ at
NO$\nu$A best-fit is at higher octant but the same for T2K is at lower octant.
The present data from accelerator neutrino long baseline experiments lose
octant determination sensitivity when analysed with LIV. The tension between
the two experiments is also reduced when the data are analysed with LIV.

##  I Introduction

The three flavour neutrino oscillation phenomenon is parameterised by the
unitary Pontecorvo-Maki-Nakagawa-Sakata (PMNS) mixing matrix:

$U=\left[\begin{array}[]{ccc}c_{13}c_{12}&s_{12}c_{13}&s_{13}e^{-i{\delta_{CP}}}\\\
-s_{12}c_{23}-c_{12}s_{23}s_{13}e^{i{\delta_{CP}}}&c_{12}c_{23}-s_{12}s_{23}s_{13}e^{i{\delta_{CP}}}&s_{23}c_{13}\\\
s_{12}s_{23}-s_{13}c_{12}c_{23}e^{i{\delta_{CP}}}&-c_{12}s_{23}-s_{13}c_{23}s_{12}e^{i{\delta_{CP}}}&c_{23}c_{13}\\\
\end{array}\right],$ (I.1)

where $c_{ij}=\cos\theta_{ij}$ and $s_{ij}=\sin\theta_{ij}$. The neutrino
oscillation probabilities depend on three mixing angles: ${\theta_{12}}$,
${\theta_{13}}$, and ${\theta_{23}}$; two independent mass squared
differences: ${\Delta_{21}}=m_{2}^{2}-m_{1}^{2}$ and
${\Delta_{31}}=m_{3}^{2}-m_{1}^{2}$, where $m_{1}$, $m_{2}$ and $m_{3}$ are
the masses of the neutrino mass eigenstates $\nu_{1}$, $\nu_{2}$ and $\nu_{3}$
respectively; and a CP violating phase: ${\delta_{CP}}$. Among these
parameters, ${\theta_{12}}$ and ${\Delta_{21}}$ have been measured in solar
neutrino experiments Bahcall et al. (2004); Ahmad et al. (2002).
$\sin^{2}2{\theta_{23}}$ and $|{\Delta_{31}}|$ has been obtained by measuring
the $\nu_{\mu}$ survival probabilities in accelerator neutrino long baseline
experiment MINOS Nichol (2012a).Recently reactor neutrino experiments have
measured non-zero value of ${\theta_{13}}$ An et al. (2012); Ahn et al.
(2012); Abe et al. (2012a). In table 1, we have noted down the current best-
fit values along with the $1\,\sigma$ uncertainties of the oscillation
parameters.

Parameters | NH foo | IH
---|---|---
${\theta_{12}}/^{o}$ | $33.44^{+0.77}_{-0.74}$ | $33.45^{+0.78}_{-0.75}$
${\theta_{23}}/^{o}$ | $49.2^{+0.9}_{-1.2}$ | $49.3^{+0.9}_{-1.1}$
${\theta_{13}}/^{o}$ | $8.57^{+0.12}_{-0.12}$ | $8.60^{+0.12}_{-0.12}$
${\delta_{CP}}/^{o}$ | $197^{+24}_{-24}$ | $282^{+26}_{-30}$
$\frac{{\Delta_{21}}}{10^{-5}\,{\rm eV}^{2}}$ | $7.42^{+0.21}_{-0.20}$ | $7.42^{+0.21}_{-0.20}$
$\frac{\Delta_{3l}}{10^{-3}\,{\rm eV}^{2}}$ | $2.517^{+0.026}_{-0.028}$ | $-2.498^{+0.028}_{-0.028}$
Table 1: Global best-fit values of neutrino oscillation parameters nuf ;
Esteban et al. (2020). $\Delta_{3l}={\Delta_{31}}>0$ ($\Delta_{32}<0$) for NH
(IH).

The current unknowns are the sign of ${\Delta_{31}}$, octant of
${\theta_{23}}$ and the CP violating phase ${\delta_{CP}}$.There can be two
possible mass orderings for the masses of the neutrino mass eigenstates,
depending on the sign of ${\Delta_{31}}$: normal hierarchy (NH), which implies
$m_{3}>>m_{2}>m_{1}$, and inverted hierarchy (IH), which implies
$m_{2}>m_{1}>>m_{3}$ De Salas et al. (2018). It is expected that the current
accelerator based long baseline neutrino oscillation experiments NO$\nu$A
Ayres et al. (2004) and T2K Itow et al. (2001) will measure these unknowns by
measuring the $\nu_{\mu}\to\nu_{e}$ oscillation probabilities in presence of
matter effect. Recently both the experiments have published their latest
analysis. The best-fit values for NO$\nu$A is
$\sin^{2}{\theta_{23}}=0.57^{+0.04}_{-0.03}$ and ${\delta_{CP}}=0.82\pi$ for
NH Himmel (2020). For T2K, the best-fit values are
$\sin^{2}{\theta_{23}}=0.53^{+0.03}_{-0.04}$ for both mass hierarchies, and
${\delta_{CP}}/\pi=-1.89^{+0.70}_{-0.58}$ ($-1.38^{+0.48}_{-0.54}$) for normal
(inverted) hierarchy Dunne (2020). Therefore, there is a moderate tension
between the outcomes of the two experiments. The measured best-fit
${\delta_{CP}}$ values of both the experiments are far apart. Moreover, there
is no overlap between the allowed regions on
$\sin^{2}{\theta_{23}}-{\delta_{CP}}$ plane at $1\,\sigma$ confidence level
(C.L.). Although the individual experiments prefer NH over IH, their combined
analysis has the best-fit point at IH Kelly et al. (2021).

Apart from the unknown standard oscillation parameters, these experiments will
also investigate about the possibility of beyond standard model (BSM)
physics.A large number of studies have been done about exploring BSM physics
with long baseline neutrino oscillation experiments Argüelles et al. (2020).
Recently non-unitary neutrino mixing Miranda et al. (2019) and non-standard
neutrino interaction during propagation through matter Chatterjee and Palazzo
(2021); Denton et al. (2021) have been used to resolve the tension between
NO$\nu$A and T2K. It is important to test other BSM physics models to resolve
the tension as well.

Neutrino oscillation requires neutrinos to be massive albeit extremely light.
This curious and interesting characteristic makes neutrino oscillation the
first experimental signature of BSM physics. Without loss of any generality,
SM can be considered as the low energy effective theory derived from a more
general theory governed by Planck mass ($M_{P}\simeq 10^{19}\,{\rm GeV}$).
This more fundamental theory unifies gravitational interactions along with
strong, weak and electro-magnetic interactions. There exists theoretical
models Kostelecky and Samuel (1989a, b); Kostelecky and Potting (1991, 1995,
1996) which include spontaneous Lorentz invariance violation (LIV) and CPT
violations in that more complete framework at Planck scale. At the observable
low energy, these violations can give rise to minimal extension of SM through
perturbative terms suppressed by $M_{P}$. Particles and anti-particles have
same mass and lifetime due to CPT invariance. Any observed difference between
masses or lifetimes of particles and anti-particles would be a signal for CPT
violation. The present upper limit on CPT violation from kaon system is
$|m_{K^{0}}-m_{\bar{K}_{0}}|/m_{K}<6\times 10^{-18}$ Tanabashi et al. (2018).
Since, kaons are Bosons and the natural mass term appearing in the Lagrangian
is mass squared term, the above constraints can be rewritten as
$|m_{K^{0}}^{2}-m_{\bar{K}_{0}}^{2}|<0.25\,{\rm eV}^{2}$. Current neutrino
oscillation data provide the bounds
$|{\Delta_{21}}-\bar{\Delta}_{21}|<5.9\times 10^{-5}\,{\rm eV}^{2}$ and
$|{\Delta_{31}}-\bar{\Delta}_{31}|<1.1\times 10^{-3}\,{\rm eV}^{2}$ Ohlsson
and Zhou (2015). If these differences are non-zero and they are manifestation
of some kind of CPT violating effects, they can induce changes in neutrino
oscillation probabilities Kostelecky and Mewes (2004a); Diaz and Kostelecky
(2012); Kostelecky and Mewes (2004b); Katori et al. (2006). Various studies
have been done about the LIV/CPT violation with neutrinos Dighe and Ray
(2008); Barenboim and Lykken (2009); Rebel and Mufson (2013); de Gouvêa and
Kelly (2017); Barenboim et al. (2018, 2019); Majhi et al. (2020); Giunti and
Laveder (2010); Datta et al. (2004); Chatterjee et al. (2014); Singh Koranga
and Khurana (2014); Diaz and Schwetz (2016); Hooper et al. (2005); Tomar et
al. (2015); Liao and Marfatia (2018); Kumar Agarwalla and Masud (2020).
Several neutrino oscillation experiments have looked for LIV/CPT violations
and put on constraints on the LIV/CPT violating parameters Auerbach et al.
(2005); Adamson et al. (2008, 2012); Aguilar-Arevalo et al. (2018); Abe et al.
(2012b, 2015); Aartsen et al. (2018); Abe et al. (2017). Ref. Kostelecky and
Russell (2008) includes the list of constraints on all the relevant LIV/CPT
violating parameters. But till date no study has been made to look for LIV/CPT
violation in the long baseline accelerator neutrino oscillation experiments
data. In this paper, we will consider the minimal extension of SM that
violates Lorentz invariance as well as CPT symmetry. We will test the model
with the latest data from NO$\nu$A and T2K and try to see if there is any hint
of CPT violating LIV in the individual as well as combined data set and
whether the tension between the two experiments can be resolved with the help
of this new physics hypothesis.

In section II, we will discuss the theoretical framework of LIV in neutrino
oscillation and present the comparison between oscillation probabilities with
and without LIV. In section III, we will discuss the details method of our
analysis and in section IV we will present our results after analysing data
from NO$\nu$A and T2K. The conclusion will be drawn in section V.

##  II Theoretical framework

The Lorentz invariance violating neutrinos and anti-neutrinos can be described
by the effective Lagrangian Kostelecky and Mewes (2004a); Kostelecky and Mewes
(2012)

$\mathcal{L}=\bar{\Psi}_{A}\left(i\gamma_{\mu}\partial_{\mu}\delta_{AB}-M_{AB}+\hat{\mathcal{Q}}_{AB}\right)\Psi_{B}+{\rm
h.c.}.$ (II.2)

$\Psi_{A(B)}$ is a $2N$ dimensional spinor containing $\psi_{\alpha(\beta)}$,
which is a spinor field with $\alpha(\beta)$ ranging over $N$ spinor flavours,
and their charge conjugates given by
$\psi_{\alpha(\beta)}^{C}=C\bar{\psi}_{\alpha(\beta)}^{T}$. Therefore,
$\Psi_{A(B)}$ can be expressed as

$\Psi_{A(B)}=\left(\psi_{\alpha(\beta)},\psi_{\alpha(\beta)}^{C}\right)^{T}.$
(II.3)

$\hat{\mathcal{Q}}$ in eq. II.2 is a generic Lorentz invariance violating
operator. The first term in the right side of eq. II.2 is the kinetic term,
the second term is the mass term involving the mass matrix $M$ and the third
term gives rise to the LIV effect. $\hat{\mathcal{Q}}$ is small and
perturbative in nature.

We will restrict ourselves only to the renormalizable Dirac couplings in the
theory, i.e. terms only with mass dimension $\leq 4$ will be incorporated.
Doing so, one can write the Lorentz invariance violating Lagrangian in the
flavour basis as Kostelecky and Mewes (2004a)

$\mathcal{L}_{\rm
LIV}=-\frac{1}{2}\left[a^{\mu}_{\alpha\beta}\bar{\psi}_{\alpha}\gamma_{\mu}\psi_{\beta}+b^{\mu}_{\alpha\beta}\bar{\psi}_{\alpha}\gamma_{5}\gamma_{\mu}\psi_{\beta}-ic_{\alpha\beta}^{\mu\nu}\bar{\psi}_{\alpha}\gamma_{\mu}\partial_{\nu}\psi_{\beta}-id_{\alpha\beta}^{\mu\nu}\bar{\psi}_{\alpha}\gamma_{\mu}\gamma_{5}\partial_{\nu}\psi_{\beta}\right],$
(II.4)

where $a^{\mu}_{\alpha\beta}$, $b^{\mu}_{\alpha\beta}$,
$c^{\mu\nu}_{\alpha\beta}$ and $d^{\mu\nu}_{\alpha\beta}$ Lorentz invariance
violating parameters. Since, only left handed neutrinos are present in SM, the
observable effects in the neutrino oscillation experiments can be
parameterized as

$\left(a_{L}\right)^{\mu}_{\alpha\beta}=\left(a+b\right)^{\mu}_{\alpha\beta},\left(c_{L}\right)^{\mu\nu}_{\alpha\beta}=\left(c+d\right)^{\mu\nu}_{\alpha\beta}.$
(II.5)

These are constant Hermitian matrices which can modify the standard
Hamiltonian in vacuum. The first combination involves CPT violation, where as
the second combination is the CPT conserving Lorentz invariance violating
neutrinos. In this paper, we will consider only direction independent
isotropic terms, and hence we will only consider the $\mu=\nu=0$. From now on,
for simplicity, we will call $a_{\alpha\beta}^{0}$ terms as $a_{\alpha\beta}$
and $c_{\alpha\beta}^{00}$ term as $c_{\alpha\beta}$. Taking into account only
these isotropic LIV terms, the neutrino Hamiltonian with LIV effect becomes:

$H=H_{\rm vac}+H_{\rm mat}+H_{\rm LIV},$ (II.6)

where

$H_{\rm vac}=\frac{1}{2E}U\left[\begin{array}[]{ccc}m_{1}^{2}&0&0\\\
0&m_{2}^{2}&0\\\ 0&0&m_{3}^{2}\\\ \end{array}\right]U^{\dagger};H_{\rm
mat}=\sqrt{2}G_{F}N_{e}\left[\begin{array}[]{ccc}1&0&0\\\ 0&0&0\\\ 0&0&0\\\
\end{array}\right];$ (II.7) $H_{\rm
LIV}=\left[\begin{array}[]{ccc}a_{ee}&a_{e\mu}&a_{e\tau}\\\
a_{e\mu}^{*}&a_{\mu\mu}&a_{\mu\tau}\\\
a_{e\tau}^{*}&a_{\mu\tau}^{*}&a_{\tau\tau}\\\
\end{array}\right]-\frac{4}{3}E\left[\begin{array}[]{ccc}c_{ee}&c_{e\mu}&c_{e\tau}\\\
c_{e\mu}^{*}&c_{\mu\mu}&c_{\mu\tau}\\\
c_{e\tau}^{*}&c_{\mu\tau}^{*}&c_{\tau\tau}\\\ \end{array}\right].$ (II.8)

$G_{F}$ is the Fermi coupling constant and $N_{e}$ is the electron density
along the neutrino path. The $-4/3$ in front of the second term arises due to
non observability of the Minkowski trace of the CPT conserving LIV term
$c_{L}$ which relates $xx$, $yy$, and $zz$ component to the $00$ component
Kostelecky and Mewes (2004a). The effects of $a_{\alpha\beta}$ are
proportional to the baseline $L$ and those of $c_{\alpha\beta}$ are
proportional to $LE$. In this paper, we will consider the effects of CPT
violating LIV parameters $a_{\alpha\beta}$ only.

It is noteworthy that the Hamiltonian due to LIV is analogous to that with
neutral current (NC) non standard interaction (NSI) during the propagation of
neutrinos through matter

$H^{\prime}=H_{\rm vac}+H_{\rm mat}+H_{\rm NSI},$ (II.9)

where

$H_{\rm
NSI}=\sqrt{2}G_{F}N_{e}\left[\begin{array}[]{ccc}\epsilon_{ee}&\epsilon_{e\mu}&\epsilon_{e\tau}\\\
\epsilon_{e\mu}^{*}&\epsilon_{\mu\mu}&\epsilon_{\mu\tau}\\\
\epsilon_{e\tau}^{*}&\epsilon_{\mu\tau}^{*}&\epsilon_{\tau\tau}\\\
\end{array}\right].$ (II.10)

$\epsilon_{\alpha\beta}$ are the strength of NSI. Thus, a relation between CPT
violating LIV and NSI can be found by following equation Diaz (2015):

$\epsilon_{\alpha\beta}=\frac{a_{\alpha\beta}}{\sqrt{2}G_{F}N_{e}}.$ (II.11)

In this work, we will consider the effects of parameters
$a_{e\mu}=|a_{e\mu}|e^{i\phi_{e\mu}}$ and
$a_{e\tau}=|a_{e\tau}|e^{i\phi_{e\tau}}$, because these two parameters have
the highest influences on $\nu_{\mu}\to\nu_{e}$ oscillation probability Kumar
Agarwalla and Masud (2020), which is responsible for octant, ${\delta_{CP}}$,
and hierarchy sensitivity of long baseline accelerator neutrino like NO$\nu$A
and T2K. Since, we are mostly concerned about determining these unknown
standard oscillation parameters in the long baseline accelerator neutrino
experiment, we fixed all other LIV parameters, except $a_{e\mu}$ and
$a_{e\tau}$, to zero. It implies that in this paper, the contribution from
Lorez invariance violation in the NO$\nu$A and T2K experiments is coming only
from the CPT violating Lorentz violation and mostly in the appearance
channels. The $\nu_{\mu}$ and $\bar{\nu}_{\mu}$ disappearance channels still
conserve Lorentz invariance. The current constraint on these parameters from
Super-kamiokande experiment at $95\%$ confidence level (C.L.) is Abe et al.
(2015)

$|a_{e\mu}|<2.5\times 10^{-23}\,{\rm GeV};\,|a_{e\tau}|<5\times 10^{-23}\,{\rm
GeV}$ (II.12)

The $\nu_{\mu}\to\nu_{e}$ oscillation probability in presence of LIV
parameters $a_{e\mu}$ and $a_{e\tau}$ can be written in the similar way as the
oscillation probability in presence of NSI parameters $\epsilon_{e\mu}$ and
$\epsilon_{e\tau}$ Kikuchi et al. (2009); Agarwalla et al. (2016); Masud et
al. (2019):

$P_{\mu e}^{\rm SM+LIV}\simeq P_{\mu e}(\rm SM)+P_{\mu e}(a_{e\mu})+P_{\mu
e}(a_{e\tau}).$ (II.13)

The first term in eq. II.13 is the oscillation probability in the presence of
standard matter effect. It can be written as Cervera et al. (2000)

$\displaystyle P_{\mu e}(\rm SM)$
$\displaystyle=\sin^{2}2{\theta_{13}}\sin^{2}{\theta_{23}}\frac{\sin^{2}{\hat{\Delta}}(1-{\hat{A}})}{(1-{\hat{A}})^{2}}$
(II.14) $\displaystyle+$ $\displaystyle\alpha\cos{\theta_{13}}\sin
2{\theta_{12}}\sin 2{\theta_{13}}\sin
2{\theta_{23}}\cos({\hat{\Delta}}+{\delta_{CP}})$
$\displaystyle\frac{\sin{\hat{\Delta}}{\hat{A}}}{{\hat{A}}}\frac{\sin{\hat{\Delta}}(1-{\hat{A}})}{1-{\hat{A}}}$
$\displaystyle+$
$\displaystyle\alpha^{2}\sin^{2}2\theta_{12}\cos^{2}\theta_{13}\cos^{2}\theta_{23}\frac{\sin^{2}{\hat{\Delta}}{\hat{A}}}{{\hat{A}}^{2}},$

where $\alpha=\frac{{\Delta_{21}}}{{\Delta_{31}}}$,
${\hat{\Delta}}=\frac{{\Delta_{31}}L}{4E}$ and
${\hat{A}}=\frac{A}{{\Delta_{31}}}$. $A$ is the Wolfenstein matter term
Wolfenstein (1978), given by $A=2\sqrt{2}G_{F}N_{e}E$, where $E$ is the
neutrino beam energy and $L$ is the length of the baseline.

For the second and third terms in eq. II.13, describing the effects of
$a_{e\mu}$ and $a_{e\tau}$ respectively, we follow the similar approach of
NSI, described in references Kikuchi et al. (2009); Agarwalla et al. (2016);
Masud et al. (2019), and replace $\epsilon_{\alpha\beta}$ terms by
$a_{\alpha\beta}$ terms according to eq. II.11. Doing so, we can write

$\displaystyle P_{\mu e}(a_{e\beta})$ $\displaystyle=$
$\displaystyle\frac{4|a_{e\beta}|{\hat{A}}{\hat{\Delta}}\sin\theta_{13}\sin
2\theta_{23}\sin{\hat{\Delta}}}{\sqrt{2}G_{F}N_{e}}\left[Z_{e\beta}\sin({\delta_{CP}}+\phi_{e\beta})+W_{e\beta}\cos({\delta_{CP}}+\phi_{e\beta})\right]$
$\displaystyle=$ $\displaystyle\frac{4|a_{e\beta}|L\sin\theta_{13}\sin
2\theta_{23}\sin{\hat{\Delta}}}{2}\left[Z_{e\beta}\sin({\delta_{CP}}+\phi_{e\beta})+W_{e\beta}\cos({\delta_{CP}}+\phi_{e\beta})\right]$

where $\beta=\mu,\,\tau$;

$\displaystyle Z_{e\beta}$ $\displaystyle=$
$\displaystyle-\cos\theta_{23}\sin{\hat{\Delta}},\,{\rm if}\,\beta=\mu$
(II.16) $\displaystyle=$
$\displaystyle\sin\theta_{23}\sin{\hat{\Delta}},\,{\rm if}\,\beta=\tau$

and

$\displaystyle W_{e\beta}$ $\displaystyle=$
$\displaystyle\cos\theta_{23}\left(\frac{\sin^{2}{\theta_{23}}\sin{\hat{\Delta}}}{\cos^{2}{\theta_{12}}{\hat{\Delta}}}+\cos{\hat{\Delta}}\right),\,{\rm
if}\,\beta=\mu$ (II.17) $\displaystyle=$
$\displaystyle\sin\theta_{23}\left(\frac{\sin{\hat{\Delta}}}{{\hat{\Delta}}}-\cos{\hat{\Delta}}\right),\,{\rm
if}\,\beta=\tau.$

From, eq. LABEL:pmue-a, it can be seen that the LIV effects considered in this
paper are matter independent.

The oscillation probability $P_{\bar{\mu}\bar{e}}$ for anti-neutrino can be
calculated from equations II.14 and LABEL:pmue-a by substituting $A\to-A$,
${\delta_{CP}}\to-{\delta_{CP}}$, $|a_{e\beta}|\to-|a_{e\beta}|$ and
$\phi_{e\beta}\to-\phi_{e\beta}$, where $\beta=\mu,\,\tau$.

In our analysis, however, we have used GLoBES Huber et al. (2005, 2007) to
calculate the oscillation probability. To do so, we modified the probability
code of the software to include LIV. After that, GLoBES is capable of
calculating the oscillation probability without the approximations required to
derive equations II.13-LABEL:pmue-a. The evolution equation for a neutrino
state $|\nu>=\left(|\nu_{e}>,|\nu_{\mu}>,|\nu_{\tau}>\right)^{T}$ travelling a
small distance $x$ can be written in presence of LIV as

$i\frac{d}{dx}|\nu>=H|\nu>.$ (II.18)

$H$ is the Hamiltonian from eq. II.6. The oscillation probability of
$\nu_{\mu}\to\nu_{e}$ after travelling through a distance $L$ can be written
as

$P_{\mu e}^{\rm SM+LIV}=|<\nu_{e}|e^{-iHL}|\nu_{\mu}>|^{2}.$ (II.19)

In fig. 1, we have shown the effects of $|a_{e\mu}|$ and $|a_{e\tau}|$ on the
oscillation probability for NH of NO$\nu$A for different values of
${\delta_{CP}}$ and $\phi_{e\mu}$, $\phi_{e\tau}$. To generate these plots, we
have fixed the standard oscillation parameter values to their best-fit values
taken from nuf ; Esteban et al. (2020). It is obvious that there is a large
difference between standard oscillation and oscillation with LIV at
probability level.This difference makes our motivation to test LIV with
oscillation data from long baseline experiments even stronger. It can be
observed that $\phi_{e\mu}$ $(\phi_{e\tau})$ have quite opposite effects on
$P_{\mu e}$ compared to that of ${\delta_{CP}}$.

Figure 1: Effect of $|a_{e\mu}|(|a_{e\tau}|)=2\times 10^{-23}$ GeV on the
oscillation probability for NH of NO$\nu$A in the left (right panel). The
black line shows the oscillation probability with standard matter effect. The
standard oscillation parameter values have been fixed to their best-fit values
nuf ; Esteban et al. (2020). The red line indicates the oscillation
probability with LIV effect. Values of ${\delta_{CP}}$, $\phi_{e\mu}$ and
$\phi_{e\tau}$ have been mentioned in the panels.

In fig. 2, we have shown the similar comparison for T2K. It is obvious that
the effect of LIV is less prominent in case of T2K than it was for NO$\nu$A.
It is expected because the CPT violating LIV effect is proportional to the
length of the baseline and T2K baseline (295 km) is very small compared to
that of NO$\nu$A (810 km).

Figure 2: Effect of $|a_{e\mu}|(|a_{e\tau}|)=2\times 10^{-23}$ GeV on the
oscillation probability for NH of T2K in the left (right panel). The black
line shows the oscillation probability with standard matter effect. The
standard oscillation parameter values have been fixed to their best-fit values
nuf ; Esteban et al. (2020). The red line indicates the oscillation
probability with LIV effect. Values of ${\delta_{CP}}$, $\phi_{e\mu}$ and
$\phi_{e\tau}$ have been mentioned in the panels.

It would be interesting to note down the the ability to discriminate between
the two models for different ${\delta_{CP}}$ values and baselines. To do so,
we first fixed the neutrino energy at the NO$\nu$A flux peak enegy $2.0$ GeV
and varied ${\delta_{CP}}$ in the range $[-180^{\circ}:180^{\circ}]$ and the
baseline from $100$ km to $1400$ km. In figures 3-5, the ratio between $P_{\mu
e}^{\rm SM+LIV}$ and $P_{\mu e}^{\rm SM}$ has been shown as a function of
$L/E$ and ${\delta_{CP}}$ for different $\phi_{e\mu}$ and $\phi_{e\tau}$.
values mentioned in the figure panels. $|a_{e\mu}|$ and $|a_{e\tau}|$ have
been fixed at $2\times 10^{-23}$ GeV each. The standard oscillation parameters
have been fixed at the current global best-fit values taken from nuf ; Esteban
et al. (2020). The farther away the ratio is from 1, the better is the
discrimination capability between the two models. It can be said that with
$810$ km baseline and flux peak energy at $2.0$ GeV, NO$\nu$A has a good
discrimination capability between the two models at the oscillation
probability level for all three cases and with both the hierarchies.

Figure 3: Ratio of the LIV to standard oscillation probabilities as a function
of $L/E$ and ${\delta_{CP}}$. The reference energy $E=2.0$ GeV has been fixed
to the NO$\nu$A flux peak energy. For this peak energy and NO$\nu$A baseline,
$L/E=405$ km/GeV. The upper (lower) panel shows the ratio for NH (IH), the
left (right) panel shows it for neutrino (anti-neutrino). $|a_{e\mu}|$ and
$|a_{e\tau}|$ have been fixed at $2\times 10^{-23}$ GeV each and we have set
$\phi_{e\mu}=\phi_{e\tau}=0$. Figure 4: Ratio of the LIV to standard
oscillation probabilities as a function of $L/E$ and ${\delta_{CP}}$. The
reference energy $E=2.0$ GeV has been fixed to the NO$\nu$A flux peak energy.
For this peak energy and NO$\nu$A baseline, $L/E=405$ km/GeV. The upper
(lower) panel shows the ratio for NH (IH), the left (right) panel shows it for
neutrino (anti-neutrino). $|a_{e\mu}|$ and $|a_{e\tau}|$ have been fixed at
$2\times 10^{-23}$ GeV each and we have set
$\phi_{e\mu}=\phi_{e\tau}=90^{\circ}$. Figure 5: Ratio of the LIV to standard
oscillation probabilities as a function of $L/E$ and ${\delta_{CP}}$. The
reference energy $E=2.0$ GeV has been fixed to the NO$\nu$A flux peak energy.
For this peak energy and NO$\nu$A baseline, $L/E=405$ km/GeV. The upper
(lower) panel shows the ratio for NH (IH), the left (right) panel shows it for
neutrino (anti-neutrino). $|a_{e\mu}|$ and $|a_{e\tau}|$ have been fixed at
$2\times 10^{-23}$ GeV each and we have set
$\phi_{e\mu}=\phi_{e\tau}=-90^{\circ}$.

In the next step, we repeated the same exercise by fixing the energy $E=0.7$
GeV at the T2K flux peak energy. The results have been shown in figures 6-8.
It can be observed that T2K with its baseline of $L=295$ km, has a good
discrimination capability at the probability level between the two models for
most of the CP violating ${\delta_{CP}}$ values except for $\nu$ ($\bar{\nu}$)
run when the hierarchy is IH (NH) and $\phi_{e\mu}=\phi_{e\tau}=0$.

Figure 6: Ratio of the LIV to standard oscillation probabilities as a function
of $L/E$ and ${\delta_{CP}}$. The reference energy $E=0.7$ GeV has been fixed
to the T2K flux peak energy. For this peak energy and T2K baseline, $L/E=421$
km/GeV. The upper (lower) panel shows the ratio for NH (IH), the left (right)
panel shows it for neutrino (anti-neutrino). $|a_{e\mu}|$ and $|a_{e\tau}|$
have been fixed at $2\times 10^{-23}$ GeV each and we have set
$\phi_{e\mu}=\phi_{e\tau}=0$. Figure 7: Ratio of the LIV to standard
oscillation probabilities as a function of $L/E$ and ${\delta_{CP}}$. The
reference energy $E=0.7$ GeV has been fixed to the T2K flux peak energy. For
this peak energy and T2K baseline, $L/E=421$ km/GeV. The upper (lower) panel
shows the ratio for NH (IH), the left (right) panel shows it for neutrino
(anti-neutrino). $|a_{e\mu}|$ and $|a_{e\tau}|$ have been fixed at $2\times
10^{-23}$ GeV each and we have set $\phi_{e\mu}=\phi_{e\tau}=90^{\circ}$.
Figure 8: Ratio of the LIV to standard oscillation probabilities as a function
of $L/E$ and ${\delta_{CP}}$. The reference energy $E=0.7$ GeV has been fixed
to the T2K flux peak energy. For this peak energy and T2K baseline, $L/E=421$
km/GeV. The upper (lower) panel shows the ratio for NH (IH), the left (right)
panel shows it for neutrino (anti-neutrino). $|a_{e\mu}|$ and $|a_{e\tau}|$
have been fixed at $2\times 10^{-23}$ GeV each and we have set
$\phi_{e\mu}=\phi_{e\tau}=-90^{\circ}$.

##  III Analysis details

The NO$\nu$A detector Ayres et al. (2004) is a 14 kt totally active
scintillator detector (TASD), placed 810 km away from the neutrino source at
the Fermilab and it is situated at $0.8^{\circ}$ off-axis of the NuMI beam.
The flux peaks at $2$ GeV, close to the oscillation maxima at 1.4 GeV for NH
and at 1.8 GeV for IH. NO$\nu$A started taking data in 2014 and took data that
until 2020 release Himmel (2020), correspond to $1.36\times 10^{21}$
($1.25\times 10^{21}$) POT in $\nu$ ($\bar{\nu}$) mode. The T2K experiment
Itow et al. (2001) uses the $\nu_{\mu}$ beam from the J-PARC accelerator at
Tokai and the water Cherenkov detector at Super-Kamiokande, which is 295 km
away from the source. The detector is situated $2.5^{\circ}$ off-axis. The
flux peaks at $0.7$ GeV, which is also close to the first oscillation maximum.
T2K started taking data in 2009 and until 2020 release of results these Dunne
(2020) correspond to $1.97\times 10^{21}$ ($1.63\times 10^{21}$) POT in $\nu$
($\bar{\nu}$) mode.

To analyse the data, we kept $\sin^{2}{\theta_{12}}$ and ${\Delta_{21}}$ to
their best-fit values $0.304$ and $7.42\times 10^{-5}\,{\rm eV}^{2}$. These
parameters have been determined by measuring electron neutrino survivavl
probabilities in solar neutrino experiments. The LIV parameters which mostly
affect electron neutrino survival probabilities are $a_{ee}$. We have fixed
this parameter to zero. Therefore, the new physics should not affect the
results from the solar neutrino experiments. We varied
$\sin^{2}2{\theta_{13}}$ in its $3\,\sigma$ range around its central value
$0.084$ with $3.5\%$ uncertainty. $\sin^{2}{\theta_{23}}$ has been varied in
its $3\,\sigma$ range $[0.41:0.62]$. These ranges have been taken from the
global best-fit ranges given in ref. Esteban et al. (2019). These global fit
was done without the 2020 data from NO$\nu$A and T2K. We varied
$|\Delta_{\mu\mu}|$ in its $3\,\sigma$ range around the MINOS best-fit value
$2.32\times 10^{-3}\,{\rm eV}^{2}$ with $3\%$ uncertainty Nichol (2012b).
$\Delta_{\mu\mu}$ is related with ${\Delta_{31}}$ by the following relation
Nunokawa et al. (2005)

$\Delta_{\mu\mu}=\sin^{2}{\theta_{23}}{\Delta_{31}}+\cos^{2}{\theta_{12}}\Delta_{32}+\cos{\delta_{CP}}\sin
2{\theta_{12}}\sin{\theta_{13}}\tan{\theta_{12}}{\Delta_{21}}.$ (III.20)

The CP violating phase ${\delta_{CP}}$ has been varied in its complete range
$[-180^{\circ}:180^{\circ}]$.

Among the new physics parameters, $|a_{e\mu}|$ and $|a_{e\tau}|$ have been
varied in the range $0-20\times 10^{-23}$ GeV range. The phases $\phi_{e\mu}$
and $\phi_{e\tau}$ have been varied in the range $[-180^{\circ}:180^{\circ}]$.

We calculated the theoretical event rates and the $\chi^{2}$ between data and
theoretical event rates using GLoBES Huber et al. (2005, 2007). The data has
been taken from Himmel (2020); Dunne (2020). To calculate the theoretical
event rates, we fixed the signal and background efficiencies by matching with
the Monte-Carlo simulations given by the collaborations Himmel (2020); Dunne
(2020). Automatic bin based energy smearing for generated theoretical events
has been implemented in the same way as described in the GLoBES manual Huber
et al. (2005, 2007). For this purpose, we used a Gaussian smearing function

$R^{c}(E,E^{\prime})=\frac{1}{\sqrt{2\pi}}e^{-\frac{(E-E^{\prime})^{2}}{2\sigma^{2}(E)}},$
(III.21)

where $E^{\prime}$ is the reconstructed energy. The energy resolution function
is given by

$\sigma(E)=\alpha E+\beta\sqrt{E}+\gamma.$ (III.22)

For NO$\nu$A, we used $\alpha=0.11\,(0.09)$, $\beta=\gamma=0$ for electron
(muon) like events. For T2K, we used $\alpha=0$, $\beta=0.075$, $\gamma=0.05$
for both electron and muon like events.

For NO$\nu$A, we used

  * •

$5\%$ normalisation and $5\%$ energy callibration systematics uncertainty for
$e$ like events, and

  * •

$5\%$ normalisation and $0.01\%$ energy callibration systematics uncertainty
for $\mu$ like events.

For T2K, we used

  * •

$5\%$ normalisation and $5\%$ energy callibration systematics uncertainty for
$e$ like events, and

  * •

$5\%$ normalisation and $0.01\%$ energy callibration systematics uncertainty
for $\mu$ like events.

Implementing systematics uncertainty has been discussed in details in GLoBES
manual Huber et al. (2005, 2007). During the calculations of $\chi^{2}$ we
added priors on $\sin^{2}2{\theta_{13}}$. After calculating $\chi^{2}$, we
found out minimum of these $\chi^{2}$s and subtracted it from all the
$\chi^{2}$s to calculate $\Delta\chi^{2}$.

##  IV Results and discussions

At first, we have analysed the data with the standard matter effect without
any LIV hypothesis. The minimum $\chi^{2}$ for NO$\nu$A (T2K) with 50 (88)
bins is 48.65 (95.85) and it is at NH. For the combined analysis, the minimum
$\chi^{2}$ with 138 bins is 147.14 and it occurs at IH. In fig. 9, we have
shown the analysis in the $\sin^{2}{\theta_{23}}-{\delta_{CP}}$ plane. This
plot is comparable to the ones presented by the collaborations in references
Himmel (2020); Dunne (2020). It is evident that there is a tension between the
two experiments in terms of the best-fit ${\delta_{CP}}$ values. Moreover,
there is no overlap between the $1\,\sigma$ region of each experiment.

Figure 9: Allowed region in the $\sin^{2}{\theta_{23}}-{\delta_{CP}}$ plane
after analysing NO$\nu$A and T2K complete data set with standard matter effect
without LIV hypothesis. The left (right) panel represents test hierarchy NH
(IH). The red (blue) lines indicate the results for NO$\nu$A (T2K) and the
black line indicates the combined analysis of both. The solid (dashed) lines
indicate the $1\,\sigma$ ($3\,\sigma$) allowed regions. The minimum $\chi^{2}$
for NO$\nu$A (T2K) with 50 (88) bins is 48.65 (95.85) and it occurs at NH. For
the combined analysis, the minimum $\chi^{2}$ with 138 bins is 147.14.

Once the standard analysis is done, we proceeded to analyse the data with LIV
hypothesis. We found out that the minimum $\chi^{2}$ for NO$\nu$A (T2K) with
50 (88) bins is 47.71 (93.14) and it is at NH. Both the experiments, however,
have a degenerate solution at IH with $\Delta\chi^{2}=0.1$ For the combined
analysis the minimum $\chi^{2}$ is 145.09 at IH for 138 bins. The combined
analysis has a degenerate solution at NH with $\Delta\chi^{2}=0.1$. Therefore,
the present accelerator neutrino oscillation data has no hierarchy sensitivity
when analysed with LIV.

In fig. 10, we have presented our result with LIV hypothesis on the
$\sin^{2}{\theta_{23}}-{\delta_{CP}}$ plane. T2K (NO$\nu$A) disfavours
(includes) NO$\nu$A (T2K) best-fit points at $1\,\sigma$ C.L. Now, there is a
large overlap between the $1\,\sigma$ allowed regions of the two experiments.
Hence, one can conclude that the tension between the two experiments gets
reduced when the data are analysed with LIV. However, there is a new mild
tension in terms of the best-fit $\sin^{2}{\theta_{23}}$ values.
${\theta_{23}}$ at T2K best-fit point is at lower octant, while the same is at
higher octant for NO$\nu$A. But NO$\nu$A has a nearly degenerate
($\Delta\chi^{2}=0.35$) best-fit point at lower octant. Similarly, T2K also
cannot rule out higher octant at $1\,\sigma$ C.L. Thus, both the experiments
lose their octant sensitivity when analysed with LIV.

Figure 10: Allowed region in the $\sin^{2}{\theta_{23}}-{\delta_{CP}}$ plane
after analysing NO$\nu$A and T2K complete data set with LIV hypothesis. The
left (right) panel represents test hierarchy NH (IH). The red (blue) lines
indicate the results for NO$\nu$A (T2K) and the black line indicates the
combined analysis of both. The solid (dashed) lines indicate the $1\,\sigma$
($3\,\sigma$) allowed regions. The minimum $\chi^{2}$ for NO$\nu$A (T2K) with
50 (88) bins is 47.71 (93.14) and it occurs at NH. For the combined analysis,
the minimum $\chi^{2}$ with 138 bins is 145.09.

In fig. 11, we have shown $\Delta\chi^{2}$ as a function of the individual LIV
parameters. To do so, we marginalised $\Delta\chi^{2}$ on all the parameters
except the one against which we want to plot it. It can be seen from fig. 11
that both $|a_{e\mu}|=0$ and $|a_{e\tau}|=0$ values have $\Delta\chi^{2}>1$
for T2K. For NO$\nu$A however, both these values have $\Delta\chi^{2}<1$. For
the combined analysis, $|a_{e\mu}|=0$ has a $\Delta\chi^{2}>1$, but
$|a_{e\tau}|=0$ has $\Delta\chi^{2}<1$. Therefore, it can be said that the
present NO$\nu$A data do not favour any of the two hypotheses over the other.
However T2K data and the combined analysis disfavour standard oscillation at
$1\,\sigma$ C.L. In tables 2, 3 and 4, we have presented the best-fit values
of the standard and non-standard unknown oscillation parameters.

Figure 11: $\Delta\chi^{2}$ as a function of individual LIV parameters.

To emphasize our result, in fig. 12, we have presented the expected electron
and positron events rates for each energy bins for both standard oscillation
and oscillation with LIV as a function of energy for both NO$\nu$A and T2K.
The experimental event rates have also been plotted. It is obvious that for
NO$\nu$A, there is not any significant difference between the expected event
rates at best-fit points of the two models and both models give a good fit to
the data. However, for T2K, there is a clear distinction at the expected event
rates at the best-fit points of the two models. Also, LIV gives a better fit
to the data especially at energies close to the flux peak energy.

Figure 12: Expected binned electron and positron event rates as a function of energy at the best-fit points for both the models and for both NO$\nu$A and T2K experiments. The experimental event rates have also been plotted. Parameters | standard | LIV | $90\%$
---|---|---|---
| NH | IH | NH | IH | NH | IH
$\frac{\Delta_{\mu\mu}}{10^{-3}\,{\rm eV}^{2}}$ | $2.44^{+0.02}_{-0.048}$ | $-(2.44^{+0.02}_{-0.048})$ | $2.40_{-0.026}^{+0.004}$ | $-(2.41_{-0.05}^{+0.01})$ |  |
$\sin^{2}2{\theta_{13}}$ | $0.084_{-0.002}^{+0.002}$ | $0.084_{-0.002}^{+0.003}$ | $0.084_{-0.003}^{+0.002}$ | $0.084_{-0.003}^{+0.002}$ |  |
$\sin^{2}{\theta_{23}}$ | $0.59^{+0.01}_{-0.01}$ | $0.59^{+0.01}_{-0.02}$ | $0.43^{+0.03}_{-0.02}\oplus 0.59^{+0.03}_{-0.03}$ | $0.43^{+0.02}_{-0.01}\oplus 0.59^{+0.03}_{-0.03}$ |  |
${\delta_{CP}}/^{\circ}$ | $130^{+40}_{-110}$ | $-(100^{+50}_{-60})$ | $-144.36$ | $-173.38$ |  |
$\frac{|a_{e\mu}|}{10^{-23}{\rm GeV}}$ |  |  | $4.81$ | $2.22$ | $<8.19$ | $<7.78$
$\frac{|a_{e\tau}|}{10^{-23}{\rm GeV}}$ |  |  | $2.52$ | $5.33^{+9.20}_{-5.33}$ | $<3.18$ | $<15.71$
$\phi_{e\mu}$ |  |  | $-114.52$ | $141.18$ |  |
$\phi_{e\tau}/^{\circ}$ |  |  | $-145.02$ | $25.04$ |  |
Table 2: Parameter values at the best-fit points for NO$\nu$A.The $1\,\sigma$ error bars have been mentioned where possible. The $90\%$ limits for 1 d.o.f have also been mentioned. Parameters | standard | LIV | $90\%$
---|---|---|---
| NH | IH | NH | IH | NH | IH
$\frac{\Delta_{\mu\mu}}{10^{-3}\,{\rm eV}^{2}}$ | $2.512^{+0.048}_{-0.048}$ | $-(2.512_{+0.048}^{-0.048})$ | $2.462_{-0.04}^{+0.04}$ | $-(2.47_{-0.03}^{+0.05})$ |  |
$\sin^{2}2{\theta_{13}}$ | $0.084_{-0.002}^{+0.002}$ | $0.084_{-0.002}^{+0.003}$ | $0.084_{-0.003}^{+0.002}$ | $0.084_{-0.003}^{+0.002}$ |  |
$\sin^{2}{\theta_{23}}$ | $0.55^{+0.03}_{-0.09}$ | $0.56^{+0.02}_{-0.03}$ | $0.45^{+0.03}_{-0.03}$ | $0.45^{+0.03}_{-0.03}$ |  |
${\delta_{CP}}/^{\circ}$ | $-(100^{+50}_{-60})$ | $-(90^{+30}_{-30})$ | $-(38.20^{+65.71}_{-66.04})$ | $-(134.72^{+45.28}_{-70.57})$ |  |
$\frac{|a_{e\mu}|}{10^{-23}{\rm GeV}}$ |  |  | $4.60$ | $6.17_{-6.02}^{+4.86}$ | $<15.25$ | $<14.92$
$\frac{|a_{e\tau}|}{10^{-23}{\rm GeV}}$ |  |  | $11.14$ | $8.06$ | Out of range | Out of range
$\phi_{e\mu}$ |  |  | $64.29$ | $-77.34$ |  |
$\phi_{e\tau}/^{\circ}$ |  |  | $-153.24$ | $158.07$ |  |
Table 3: Parameter values at the best-fit points for T2K.The $1\,\sigma$ error bars have been mentioned where possible. The $90\%$ limits for 1 d.o.f have also been mentioned. Parameters | standard | LIV | $90\%$
---|---|---|---
| NH | IH | NH | IH | NH | IH
$\frac{\Delta_{\mu\mu}}{10^{-3}\,{\rm eV}^{2}}$ | $2.464^{+0.024}_{-0.048}$ | $-(2.464^{+0.024}_{-0.048})$ | $2.462_{-0.04}^{+0.02}$ | $-(2.462_{-0.023}^{+0.003})$ |  |
$\sin^{2}2{\theta_{13}}$ | $0.084_{-0.002}^{+0.002}$ | $0.084_{-0.002}^{+0.003}$ | $0.084_{-0.003}^{+0.002}$ | $0.084_{-0.003}^{+0.002}$ |  |
$\sin^{2}{\theta_{23}}$ | $0.55^{+0.03}_{-0.09}$ | $0.56^{+0.02}_{-0.03}$ | $0.44^{+0.02}_{-0.01}\oplus 0.58^{+0.02}_{-0.02}$ | $0.58^{+0.02}_{-0.02}$ |  |
${\delta_{CP}}/^{\circ}$ | $-(100^{+50}_{-60})$ | $-(90^{+30}_{-30})$ | $-(80.19^{+69.80}_{-36.21})$ | $-(86.50^{+38.62}_{-54.67})$ |  |
$\frac{|a_{e\mu}|}{10^{-23}{\rm GeV}}$ |  |  | $1.86^{+2.57}_{-1.86}$ | $1.52$ | $<4.80$ | $<3.80$
$\frac{|a_{e\tau}|}{10^{-23}{\rm GeV}}$ |  |  | $0.57$ | $4.16$ | $<6.70$ | $<9.50$
$\phi_{e\mu}/^{\circ}$ |  |  | $-115.72$ | $110.08$ |  |
$\phi_{e\tau}/^{\circ}$ |  |  | $84.36$ | $-89.27$ |  |
Table 4: Parameter values at the best-fit points for the combined analysis of
NO$\nu$A and T2K.The $1\,\sigma$ error bars have been mentioned where
possible. The $90\%$ limits for 1 d.o.f have also been mentioned.

##  V Conclusion

At present, NO$\nu$A cannot disfavour any of the two hypotheses. However, both
T2K and the combined analysis disfavours standard oscillation at $1\,\sigma$
C.L. The latest accelerator neutrino oscillation data lose hierarchy
sensitivity when analysed with LIV. The $1\,\sigma$ allowed regions from the
two experiments have a large overlap with LIV, unlike the standard oscillation
case. Therefore, one can comment that the tension between the two experiments
is reduced when the data are analysed with LIV. However, there is a new mild
tension between the best-fit values of $\sin^{2}{\theta_{23}}$. While
${\theta_{23}}$ for NO$\nu$A best-fit is at HO, it is at LO for the T2K bst-
fit point. But NO$\nu$A (T2K) has a nearly degenerate best-fit point at lower
(higher) octant as well. Therefore, both NO$\nu$A and T2K data do not have
octant and hierarchy determination capability when analysed with LIV.

In light of these, we recommend that the long baseline accelerator neutrino
experiments to be analysed with LIV along with other BSM physics. If the
future data continue to favour LIV over standard oscillation, it can be
considered as a prominent signal for LIV.

##  VI Acknowledgement

U.R. thanks Prof. Soebur Razzaque for the valuable comments.

## References

  * Bahcall et al. (2004) J. N. Bahcall, M. C. Gonzalez-Garcia, and C. Pena-Garay, JHEP 08, 016 (2004), eprint hep-ph/0406294.
  * Ahmad et al. (2002) Q. R. Ahmad et al. (SNO), Phys. Rev. Lett. 89, 011301 (2002), eprint nucl-ex/0204008.
  * Nichol (2012a) R. Nichol (MINOS) (2012a), talk given at the Neutrino 2012 Conference, June 3-9, 2012, Kyoto, Japan, http://neu2012.kek.jp/.
  * An et al. (2012) F. An et al. (DAYA-BAY), Phys.Rev.Lett. 108, 171803 (2012), eprint 1203.1669.
  * Ahn et al. (2012) J. Ahn et al. (RENO), Phys.Rev.Lett. 108, 191802 (2012), eprint 1204.0626.
  * Abe et al. (2012a) Y. Abe et al. (Double Chooz), Phys.Rev.Lett. 108, 131801 (2012a), eprint 1112.6353.
  * (7) (????).
  * Esteban et al. (2020) I. Esteban, M. C. Gonzalez-Garcia, M. Maltoni, T. Schwetz, and A. Zhou, JHEP 09, 178 (2020), eprint 2007.14792.
  * De Salas et al. (2018) P. F. De Salas, S. Gariazzo, O. Mena, C. A. Ternes, and M. Tórtola, Front. Astron. Space Sci. 5, 36 (2018), eprint 1806.11051.
  * Ayres et al. (2004) D. Ayres et al. (NOvA) (2004), eprint hep-ex/0503053.
  * Itow et al. (2001) Y. Itow et al. (T2K), pp. 239–248 (2001), eprint hep-ex/0106019.
  * Himmel (2020) A. Himmel (2020), talk given at the Neutrino 2020 meeting on July, 2nd, 2020, https://indico.fnal.gov/event/43209/contributions/187840/attachments/130740/159597/NOvA-Oscilations-NEUTRINO2020.pdf.
  * Dunne (2020) P. Dunne (2020), talk given at the Neutrino 2020 meeting on July, 2nd, 2020, https://indico.fnal.gov/event/43209/contributions/187830/attachments/129636/159603/T2K_Neutrino2020.pdf.
  * Kelly et al. (2021) K. J. Kelly, P. A. N. Machado, S. J. Parke, Y. F. Perez-Gonzalez, and R. Z. Funchal, Phys. Rev. D 103, 013004 (2021), eprint 2007.08526.
  * Argüelles et al. (2020) C. A. Argüelles et al., Rept. Prog. Phys. 83, 124201 (2020), eprint 1907.08311.
  * Miranda et al. (2019) L. S. Miranda, P. Pasquini, U. Rahaman, and S. Razzaque (2019), eprint 1911.09398.
  * Chatterjee and Palazzo (2021) S. S. Chatterjee and A. Palazzo, Phys. Rev. Lett. 126, 051802 (2021), eprint 2008.04161.
  * Denton et al. (2021) P. B. Denton, J. Gehrlein, and R. Pestes, Phys. Rev. Lett. 126, 051801 (2021), eprint 2008.01110.
  * Kostelecky and Samuel (1989a) V. A. Kostelecky and S. Samuel, Phys. Rev. D 39, 683 (1989a).
  * Kostelecky and Samuel (1989b) V. A. Kostelecky and S. Samuel, Phys. Rev. Lett. 63, 224 (1989b).
  * Kostelecky and Potting (1991) V. A. Kostelecky and R. Potting, Nucl. Phys. B 359, 545 (1991).
  * Kostelecky and Potting (1995) V. A. Kostelecky and R. Potting, Phys. Rev. D 51, 3923 (1995), eprint hep-ph/9501341.
  * Kostelecky and Potting (1996) V. A. Kostelecky and R. Potting, Phys. Lett. B 381, 89 (1996), eprint hep-th/9605088.
  * Tanabashi et al. (2018) M. Tanabashi et al. (Particle Data Group), Phys. Rev. D 98, 030001 (2018).
  * Ohlsson and Zhou (2015) T. Ohlsson and S. Zhou, Nucl. Phys. B 893, 482 (2015), eprint 1408.4722.
  * Kostelecky and Mewes (2004a) V. A. Kostelecky and M. Mewes, Phys. Rev. D 69, 016005 (2004a), eprint hep-ph/0309025.
  * Diaz and Kostelecky (2012) J. S. Diaz and A. Kostelecky, Phys. Rev. D 85, 016013 (2012), eprint 1108.1799.
  * Kostelecky and Mewes (2004b) V. A. Kostelecky and M. Mewes, Phys. Rev. D 70, 076002 (2004b), eprint hep-ph/0406255.
  * Katori et al. (2006) T. Katori, V. A. Kostelecky, and R. Tayloe, Phys. Rev. D 74, 105009 (2006), eprint hep-ph/0606154.
  * Dighe and Ray (2008) A. Dighe and S. Ray, Phys. Rev. D 78, 036002 (2008), eprint 0802.0121.
  * Barenboim and Lykken (2009) G. Barenboim and J. D. Lykken, Phys. Rev. D 80, 113008 (2009), eprint 0908.2993.
  * Rebel and Mufson (2013) B. Rebel and S. Mufson, Astropart. Phys. 48, 78 (2013), eprint 1301.4684.
  * de Gouvêa and Kelly (2017) A. de Gouvêa and K. J. Kelly, Phys. Rev. D 96, 095018 (2017), eprint 1709.06090.
  * Barenboim et al. (2018) G. Barenboim, C. A. Ternes, and M. Tórtola, Phys. Lett. B 780, 631 (2018), eprint 1712.01714.
  * Barenboim et al. (2019) G. Barenboim, M. Masud, C. A. Ternes, and M. Tórtola, Phys. Lett. B 788, 308 (2019), eprint 1805.11094.
  * Majhi et al. (2020) R. Majhi, S. Chembra, and R. Mohanta, Eur. Phys. J. C 80, 364 (2020), eprint 1907.09145.
  * Giunti and Laveder (2010) C. Giunti and M. Laveder, Phys. Rev. D 82, 113009 (2010), eprint 1008.4750.
  * Datta et al. (2004) A. Datta, R. Gandhi, P. Mehta, and S. U. Sankar, Phys. Lett. B 597, 356 (2004), eprint hep-ph/0312027.
  * Chatterjee et al. (2014) A. Chatterjee, R. Gandhi, and J. Singh, JHEP 06, 045 (2014), eprint 1402.6265.
  * Singh Koranga and Khurana (2014) B. Singh Koranga and P. Khurana, Int. J. Theor. Phys. 53, 3737 (2014).
  * Diaz and Schwetz (2016) J. S. Diaz and T. Schwetz, Phys. Rev. D 93, 093004 (2016), eprint 1603.04468.
  * Hooper et al. (2005) D. Hooper, D. Morgan, and E. Winstanley, Phys. Rev. D 72, 065009 (2005), eprint hep-ph/0506091.
  * Tomar et al. (2015) G. Tomar, S. Mohanty, and S. Pakvasa, JHEP 11, 022 (2015), eprint 1507.03193.
  * Liao and Marfatia (2018) J. Liao and D. Marfatia, Phys. Rev. D 97, 041302 (2018), eprint 1711.09266.
  * Kumar Agarwalla and Masud (2020) S. Kumar Agarwalla and M. Masud, Eur. Phys. J. C 80, 716 (2020), eprint 1912.13306.
  * Auerbach et al. (2005) L. B. Auerbach et al. (LSND), Phys. Rev. D 72, 076004 (2005), eprint hep-ex/0506067.
  * Adamson et al. (2008) P. Adamson et al. (MINOS), Phys. Rev. Lett. 101, 151601 (2008), eprint 0806.4945.
  * Adamson et al. (2012) P. Adamson et al. (MINOS), Phys. Rev. D 85, 031101 (2012), eprint 1201.2631.
  * Aguilar-Arevalo et al. (2018) A. A. Aguilar-Arevalo et al. (MiniBooNE), Phys. Rev. Lett. 121, 221801 (2018), eprint 1805.12028.
  * Abe et al. (2012b) Y. Abe et al. (Double Chooz), Phys. Rev. D 86, 112009 (2012b), eprint 1209.5810.
  * Abe et al. (2015) K. Abe et al. (Super-Kamiokande), Phys. Rev. D 91, 052003 (2015), eprint 1410.4267.
  * Aartsen et al. (2018) M. G. Aartsen et al. (IceCube), Nature Phys. 14, 961 (2018), eprint 1709.03434.
  * Abe et al. (2017) K. Abe et al. (T2K), Phys. Rev. D 95, 111101 (2017), eprint 1703.01361.
  * Kostelecky and Russell (2008) V. A. Kostelecky and N. Russell (2008), eprint 0801.0287.
  * Kostelecky and Mewes (2012) A. Kostelecky and M. Mewes, Phys. Rev. D 85, 096005 (2012), eprint 1112.6395.
  * Diaz (2015) J. S. Diaz (2015), eprint 1506.01936.
  * Kikuchi et al. (2009) T. Kikuchi, H. Minakata, and S. Uchinami, JHEP 03, 114 (2009), eprint 0809.3312.
  * Agarwalla et al. (2016) S. K. Agarwalla, S. S. Chatterjee, and A. Palazzo, Phys. Lett. B 762, 64 (2016), eprint 1607.01745.
  * Masud et al. (2019) M. Masud, S. Roy, and P. Mehta, Phys. Rev. D 99, 115032 (2019), eprint 1812.10290.
  * Cervera et al. (2000) A. Cervera, A. Donini, M. B. Gavela, J. J. Gomez Cadenas, P. Hernandez, O. Mena, and S. Rigolin, Nucl. Phys. B579, 17 (2000), [Erratum: Nucl. Phys.B593,731(2001)], eprint hep-ph/0002108.
  * Wolfenstein (1978) L. Wolfenstein, Phys. Rev. D17, 2369 (1978).
  * Huber et al. (2005) P. Huber, M. Lindner, and W. Winter, Comput.Phys.Commun. 167, 195 (2005), eprint hep-ph/0407333.
  * Huber et al. (2007) P. Huber, J. Kopp, M. Lindner, M. Rolinec, and W. Winter, Comput.Phys.Commun. 177, 432 (2007), eprint hep-ph/0701187.
  * Esteban et al. (2019) I. Esteban, M. C. Gonzalez-Garcia, A. Hernandez-Cabezudo, M. Maltoni, and T. Schwetz, JHEP 01, 106 (2019), eprint 1811.05487.
  * Nichol (2012b) R. Nichol (MINOS) (2012b), talk given at the Neutrino 2012 Conference, June 3-9, 2012, Kyoto, Japan, http://neu2012.kek.jp/.
  * Nunokawa et al. (2005) H. Nunokawa, S. J. Parke, and R. Zukanovich Funchal, Phys.Rev. D72, 013009 (2005), eprint hep-ph/0503283.