# EFT, Decoupling, Higgs Mixing and All That Jazz

Upalaparna Banerjee, a Joydeep Chakrabortty b Christoph Englert b Wrishik
Naskar a Shakeel Ur Rahaman c Michael Spannowsky<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>Indian Institute of Technology Kanpur,
Kalyanpur, Kanpur 208016, Uttar Pradesh, IndiaSchool of Physics & Astronomy,
University of Glasgow, Glasgow G12 8QQ, United KingdomInstitute for Particle
Physics Phenomenology, Department of Physics, Durham University, Durham DH1
3LE, United Kingdom

###### Abstract

The effective field theory (EFT) framework is a precise approximation
procedure when the inherent assumptions of a large-scale separation between
the Standard Model (SM) and new interactions alongside perturbativity are
realised. Constraints from available data might not automatically guarantee
these circumstances when contrasted with UV scenarios that the EFT analysis
wishes to inform. From an EFT perspective, achieving sufficient precision in
navigating the alignment or decoupling limits beyond the SM scenarios can
necessitate moving beyond the SM’s leading, dimension six EFT deformation.
Using the example of Higgs boson mixing, we demonstrated the importance of
higher-dimensional terms in the EFT expansion. We analyse the relevance of
virtual EFT corrections and dimension eight contributions for well-determined
electroweak precision observables. We find that when moving away from the
decoupling limit, the relevance of additional terms in the EFT expansion
quickly becomes relevant. This demonstrates the necessity to move beyond
dimension six interactions for any scenario that contains Higgs boson mixing.

††preprint: IPPP/23/14

##  1 Introduction

Effective field theory (EFT) Weinberg:1978kz  is a formidable tool for
communicating sensitivity beyond the Standard Model (BSM) physics in times
when particle physics data seemingly points towards a large-scale separation
of new states relative to the Standard Model (SM) degrees of freedom. The
extension of the SM by effective interactions relevant to the high-energy
frontier of, e.g., the Large Hadron Collider (LHC), i.e. Standard Model
Effective Theory (SMEFT) at dimension six Grzadkowski:2010es  has received a
lot of theoretical attention and improvement alongside its application in a
series of experimental investigations. Matching calculations Jiang:2018pbd ;
Haisch:2020ahr ; Gherardi:2020det ; Du:2022vso ; Dawson:2017vgm ;
Zhang:2021jdf ; Dittmaier:2021fls ; Li:2022ipc ; Zhang:2022osj ;
Carmona:2021xtq ; Cohen:2020qvb ; Bakshi:2018ics  that coarse grain ultra-
violet (UV) BSM scenarios into EFT provide the technical framework to marry
together concrete scenarios of new interactions with the generic EFT analysis
of particle data. The latter is typically plagued with considerable
uncertainties, both experimentally and theoretically. Even optimistic
extrapolations of specific processes to the LHC’s high luminosity (HL) phase
can imply a significant tension between the intrinsic viability criteria that
underpin the EFT limit setting trying to inform the UV scenarios’ parameter
spaces: EFT cut-offs need to be lowered into domains that can be directly
resolved at the LHC. This can be at odds with the perturbativity of the
obtained constraints (and hence limits the reliability of the fixed-order
matching).

The obvious way out of this conundrum is to include higher-dimensional terms
in the EFT expansion. Dimension eight interactions have increasingly moved
into the focus of the theory community Murphy:2020rsh ; Li:2020gnx ;
Banerjee:2022thk . From a practical point of view, this prompts the question
of when we can be confident about reaching the point where phenomenologically-
minded practitioners can stop. Unfortunately, an answer to this question is as
process and model-dependent as matching a UV-ignorant EFT to a concrete UV
scenario.

Therefore, the phenomenological task is developing theory-guided intuition
using representative scenarios that transparently capture key issues. The
purpose of this note is to contribute to this evolving discussion using
(custodial iso-singlet) Higgs boson mixing as an example. This scenario has
seen much attention from the EFT perspective as the number of degrees of
freedom and free parameters is relatively small, thus enabling a transparent
connection of EFT and UV theory beyond the leading order of the EFT approach
(see, e.g., Refs. Jiang:2018pbd ; Dawson:2022cmu ; Dawson:2021jcl ). Higgs
mixing also arises in many BSM theories. We focus on electroweak precision
observables as these are well-constrained by collider data, thus enabling us
to navigate cut-offs and Wilson coefficients of the effective theory under
experimental circumstances where precise predictions and matching are very
relevant. This work is structured as follows: In Sec. 2, we first discuss the
oblique corrections and their relation to the polarisation functions to make
this work self-contained; Sec. 2.1 gives a quick discussion of the oblique
corrections in the singlet scenario (see also Pruna:2013bma ; Binoth:1996au ;
Bowen:2007ia ; Englert:2011yb ; Batell:2011pz ; Anisha:2021hgc ) with formulae
provided in the appendix. We then focus on the oblique parameters for this
case in dimensions six and eight SMEFT in Sec. 2.2. We detail the comparison
in Sec. 3 with a view towards perturbative unitarity. Finally, we provide
conclusions in Sec. 4.

##  2 Electroweak Precision Observables

Extensions of the SM with modified Higgs sectors can be constrained through
electroweak precision measurements. A famous subset of these that were
instrumental in discovering the Higgs boson is the so-called oblique
corrections parametrized by the Peskin-Takeuchi parameters PhysRevD.46.381
(see also ALTARELLI1991161 ). These $\widehat{S},\widehat{T},\widehat{U}$ are
chiefly extracted from Drell-Yan-like production during the LEP era using
global fits, e.g., Ref. Bardin:1999yd ; Flacher:2008zq . Defining the off-
shell two-point gauge boson functions for the SM gauge bosons as

$\parbox{130.88284pt}{\vspace{0.cm}\includegraphics[width=128.0374pt]{twopoint.pdf}}=\Pi_{VV^{\prime}}(p^{2})g^{\mu\nu}+\Sigma_{VV^{\prime}}(p^{2})p^{\mu}p^{\nu},$
(1)

with $V,V^{\prime}=W,Z,\gamma$. The Peskin-Takeuchi parameters can then be
written as

$\begin{split}\alpha\widehat{S}=&\left(\frac{4s_{W}^{2}c_{W}^{2}}{M_{Z}^{2}}\right)\left[\Pi_{ZZ}(M_{Z}^{2})-\Pi_{ZZ}(0)-\Pi_{\gamma\gamma}(M_{Z}^{2})-\frac{c_{W}^{2}-s_{W}^{2}}{c_{W}s_{W}}\left(\Pi_{\gamma
Z}(M_{Z}^{2})-\Pi_{\gamma Z}(0)\right)\right],\\\
\alpha\widehat{T}=&\frac{\Pi_{WW}(0)}{M_{W}^{2}}-\frac{\Pi_{ZZ}(0)}{M_{Z}^{2}}-\frac{2s_{W}}{c_{W}}\frac{\Pi_{\gamma
Z}(0)}{M_{Z}^{2}},\\\
\alpha\widehat{U}=&4s_{W}^{2}\left[\frac{\Pi_{WW}(M_{W}^{2})-\Pi_{WW}(0)}{M_{W}^{2}}-c_{W}^{2}\left(\frac{\Pi_{ZZ}(M_{Z}^{2})-\Pi_{ZZ}(0)}{M_{Z}^{2}}\right)\right.\\\
&\hskip 142.26378pt\left.-2s_{W}c_{W}\left(\frac{\Pi_{\gamma
Z}(M_{Z}^{2})-\Pi_{\gamma
Z}(0)}{M_{Z}^{2}}\right)-s_{W}^{2}\frac{\Pi_{\gamma\gamma}(M_{Z}^{2})}{M_{Z}^{2}}\right]\,.\end{split}$
(2)

$s_{W},c_{W}$ denote the sine and cosine of the Weinberg angle, $\alpha$ is
the fine structure constant, and $M_{i}$ stands for the gauge boson
masses.111Note that we employ the normalization of Peskin and Takeuchi,
although the hatted quantities are typically defined in the normalization of
Barbieri:2004qk . This is to avoid confusion between the oblique corrections
and the singlet field introduced below. Constraints on new physics can then be
formulated by examining the difference of these parameters from the best SM
fit point. To draw a comparison between the full theory and EFT we restrict
our analysis to one loop order. In the next subsection, we provide the
contributions to the oblique parameters from the full theory.

###  2.1 SM extended by a real singlet scalar

The most general scalar potential for the SM Higgs Sector extended by a real
singlet scalar field ($S$) is (ignoring tadpoles fixed through minimizing
conditions),

$V(H,S)=-\mu^{2}H^{\dagger}H+\frac{1}{2}m^{2}_{S}S^{2}+\eta_{S}(H^{\dagger}H)S+k_{S}(H^{\dagger}H)S^{2}+\lambda_{h}(H^{\dagger}H)^{2}+\frac{1}{4!}\lambda_{S}S^{4},$
(3)

with $H$ being the SM Higgs doublet, which gets a vacuum expectation value
(vev) $v\simeq 246~{}\text{GeV}$. $H$ can then be expanded around the vev:

$H=\frac{1}{\sqrt{2}}\begin{pmatrix}\sqrt{2}\,G^{+}\\\
v+h+i\,\eta\end{pmatrix}.$ (4)

The presence of the mixing terms in the potential given in Eq. (3), results in
different mass eigenstates that are a mixture of $h$ that is the neutral
component of $H$, and $S$, related by the mixing angle $\theta$:

$\begin{pmatrix}\tilde{h}\\\
\tilde{s}\end{pmatrix}=\begin{pmatrix}\cos{\theta}&&-\sin{\theta}\\\
\sin{\theta}&&\cos{\theta}\end{pmatrix}\begin{pmatrix}h\\\ S\end{pmatrix}.$
(5)

Here, $\cos{\theta}$ can be written in terms of the Lagrangian parameters as

$\cos^{2}{\theta}=\frac{1}{2}\left[1+\left(1+\frac{4\,v^{2}\eta_{S}^{2}}{(M_{\cal{S}}^{2}-M_{h}^{2})^{2}-4v^{2}\eta_{S}^{2}}\,\right)^{-1/2}\right]\\\
=1-\left(\frac{v\,\eta_{S}}{M_{\cal{S}}^{2}-M^{2}_{h}}\right)^{2}-\left(\frac{v\,\eta_{S}}{M_{\cal{S}}^{2}-M^{2}_{h}}\right)^{4}+...\,.$
(6)

The eigenvalues corresponding to these mass eigenstates shown in Eq. (5)
correspond to the masses of the scalars in the theory, i.e.,
$M_{h}=125~{}\text{GeV}$ and a free parameter $M_{\cal{S}}$, respectively.
These mass eigenvalues can be expressed in terms of the Lagrangian parameters
and the Higgs vev ($v$) as,

$\displaystyle
M_{h}^{2},M_{\mathcal{S}}^{2}=\frac{1}{2}\left(m_{S}^{2}+m_{h}^{2}\mp\sqrt{4v^{2}\eta_{S}+(m_{S}^{2}-m_{h}^{2})^{2}}\right)\,,$
(7)

where $m_{h}^{2}=2\lambda_{h}v^{2}$. For the computation of the oblique
parameters we only consider the radiative corrections from the scalar-involved
diagrams shown in Fig. 1, since the other diagrams will provide the same
contribution to BSM and SM theory, therefore dropping out from the deviation.
The explicit expressions are given in the appendix A.1.

Figure 1: Relevant Feynman diagrams with scalars in the loop that have been
considered to compute the oblique corrections. Here
$\phi\in(\tilde{h},\tilde{s})$ when one-loop correction in the full theory is
computed.

Equation (5) clearly shows that the light (heavy) scalar couplings to the SM
particles are suppressed by a factor of $\cos\theta$ ($\sin\theta$).
Therefore, the contributions to the gauge boson self-energies get modified by
a factor of $\cos^{2}{\theta}$ or $\sin^{2}\theta$ depending on the neutral
scalar coupled to. We then express the mixing angle regarding the BSM
parameters in the potential. Limits are then imposed on the independent BSM
parameters (in our case, it is just $\eta_{S}$) and the mass of the heavy
scalar ($M_{\cal{S}}$) using the constraints of GFitter data Haller:2018nnx
as shown in Fig. 2.

Figure 2: $95\%$ and $68\%$ confidence interval bounds on the BSM trilinear
coupling $\eta_{S}$ and the Mass of the heavy scalar ($M_{\cal{S}}$) obtained
from the full-theory calculation setting constraints from GFitter
Haller:2018nnx . An additional unitarity bound is imposed which sets the limit
of $\eta_{S}$, as $|\eta_{S}|\lesssim M_{\cal{S}}$ (see appendix B).

###  2.2 Real Singlet Model from SMEFT perspective

To investigate how well the effective theory replicates the minute signatures
of the singlet extension of the SM described in Sec. 2.1 or, in turn, adjudge
the significance of the higher-order effective corrections, we extend the
effective series with relevant operator structures till dimension eight:

${\cal{L}}={\cal{L}}_{\text{SM}}+\sum_{i}\frac{{\cal{C}}_{i}^{(6)}}{\Lambda^{2}}\,{\cal{O}}_{i}^{(6)}+\sum_{j}\frac{{\cal{C}}_{j}^{(8)}}{\Lambda^{4}}\,{\cal{O}}_{j}^{(8)}.$
(8)

Here, the Wilson coefficients $\mathcal{C}_{i}$ parametrize the strengths of
the operators $\mathcal{O}_{i}$ that are produced after integrating out the
heavy real singlet scalar (for a complete matching of such operators at
dimension six, see Refs. Banerjee:2022thk ). We have chosen the cut-off scale
$\Lambda$ to be $m_{s}=M_{S}$. To validate EFT, as we will be working with
small mixing, thus the parameter space of our interest will satisfy the
following equivalence $M_{S}\simeq m_{s}$.

Figure 3: Tree-level correction to the gauge boson propagators due to the
presence of effective operators.

In particular, this implies that in the case of effective theory, there may be
tree-level electroweak corrections, as shown in Fig. 3, to Eq. (1) from the
effective operators that may emerge in the process of integrating out heavy
fields from the UV diagram and(or) through the renormalization group running
of effective operators generated by integrating out tree-level heavy
propagator at the cut-off scale. These contributions depend on the
renormalization scale and play an essential role in our further computation,
see also Jiang:2018pbd . Depending on whether the operators that could
contribute to the dominant (when considered in a model-independent way) tree-
level correction, as shown in Fig. 3, are generated at one-loop itself, the
contributions from the operators generated at the tree level, which can modify
the interactions at the one-loop, can become significant.

We categorically list the effective corrections to $\widehat{S},\widehat{T}$
up to one loop:

  * •

Tree-level correction: Expanding the Lagrangian with dimension six and
dimension eight operators can induce corrections to the transverse tree-level
vector boson propagators ($\Pi_{VV^{\prime}}$) itself, which in turn modifies
$\widehat{S}$, $\widehat{T}$ parameters Murphy:2020rsh

$\begin{split}\widehat{S}_{\text{eft,tree}}&=\frac{4\,c_{W}\,s_{W}\,v^{2}}{\alpha}\,\mathcal{C}_{HWB}^{(6)}\,+\frac{2\,c_{W}\,s_{W}\,v^{4}}{\alpha}\,\mathcal{C}_{HWB}^{(8)},\\\
\widehat{T}_{\text{eft,tree}}&=-\frac{v^{2}}{2\,\alpha}\,\mathcal{C}_{H\mathcal{D}}^{(6)}\,-\,\frac{v^{4}}{2\,\alpha}\,\mathcal{C}_{H\mathcal{D},2}^{(8)}.\end{split}$
(9)

The expressions for modifying the individual $\Pi_{VV^{\prime}}$ functions are
given in the appendix A.2. The dimension six operators contributing to Eq. (9)
are generated at one-loop while integrating out the heavy field. The matching
expressions for these coefficients are given in Tab. 1. We have also computed
the one-loop matching for the dimension eight operators involved in Eq. (9)
and noticed that these do not receive any correction while integrating out
complete heavy loop diagrams. On the other hand, these coefficients receive
contributions from removing the redundant structures at dimension six, as
discussed in Ref. Banerjee:2022thk . Since the latter corresponds to a two-
loop suppressed sub-leading contribution, we neglect the associated effects in
our analysis.

Operator |  Op. Structure |  Wilson coeffs.
---|---|---
$\mathcal{O}_{H\square}^{(6)}$ | $(H^{\dagger}H)\square(H^{\dagger}H)$ | $-\frac{\eta_{\mathcal{S}}^{2}}{2\,M_{\mathcal{S}}^{4}}$
${\color[rgb]{0,0,1}\mathcal{O}_{HWB}^{(6)}}$ | ${\color[rgb]{0,0,1}(H^{\dagger}\tau^{I}H)W^{I}_{\mu\nu}B^{\mu\nu}}$ | ${\color[rgb]{0,0,1}\frac{g_{{}_{W}}g_{Y}\eta_{\mathcal{S}}^{2}}{128\,M_{\mathcal{S}}^{4}\,\pi^{2}}}$
${\color[rgb]{0,0,1}\mathcal{O}_{HD}^{(6)}}$ | ${\color[rgb]{0,0,1}(H^{\dagger}\mathcal{D}_{\mu}H)^{*}(H^{\dagger}\mathcal{D}^{\mu}H)}$ | ${\color[rgb]{0,0,1}-\frac{7g_{Y}^{2}\eta_{\mathcal{S}}^{2}}{288\,M_{\mathcal{S}}^{4}\,\pi^{2}}}$

Table 1: Relevant operators that produce tree and one-loop corrections to the
gauge boson self-energy. The structures in blue first appear at tree-level
correction, whereas the rest of the operators contribute at one-loop first.

  * •

One-loop insertion of operators: One-loop corrections to the oblique
parameters are essential for the tree-level generated operators, for they
provide a similar contribution as the operators that are produced at one-loop
contributing to the tree-level propagator corrections shown in Eq. (9) for a
model-dependent analysis. In our case, such a contribution arises from the
operators $\mathcal{O}_{H\square}^{(6)}$ and
$\mathcal{O}_{H\mathcal{D},1}^{(8)}$. The explicit forms of their structures
are given in Tabs. 1 and 2, respectively. These operators modify the canonical
form of the kinetic term for the Higgs field

$\displaystyle\mathcal{L}_{h,\text{kin}}$ $\displaystyle=$
$\displaystyle\frac{1}{2}\,\left(1-2\,v^{2}\mathcal{C}_{H\square}^{(6)}+\frac{v^{4}}{4}\mathcal{C}_{H\mathcal{D},1}^{(8)}\right)(\partial_{\mu}h)^{2},$
(10)

which can be removed by redefining the field $h^{\prime}\to Z_{h}\,h$ with

$\displaystyle
Z_{h}=\left(1-v^{2}\,\mathcal{C}_{H\square}^{(6)}+\frac{v^{4}}{8}\,\mathcal{C}_{H\mathcal{D},1}^{(8)}\right).$
(11)

This implies that while computing the higher order corrections for EFT, we
need to recall that $\phi=h^{\prime}\to Z_{h}\,h$ in Fig. 1. This also
accounts for suitable modifications in the vertices, involving Higgs and
Goldstone in Fig. 1. This correction, up to $\mathcal{O}(1/\Lambda^{4})$
capturing the effects from both $(\text{dimension six})^{2}$ and dimension
eight terms, is incorporated by replacing the $\cos^{2}{\theta}$ with
$Z_{h}^{2}$ and setting $\sin^{2}\theta$ to zero in the expressions shown in
appendix A.1.

  * •

RGE improved correction: It is important to include the running effects to the
Wilson coefficients which arise at tree-level. $\widehat{T}_{\text{eft,tree}}$
in Eq. (9) at dimension six receives such an additional contribution from the
operator $\mathcal{O}_{H\mathcal{D}}^{(6)}$. Contributions arising from the
running of the coefficient of the operator $\mathcal{O}_{H\square}^{(6)}$
Jenkins:2013zja ; Alonso:2013hga

$\displaystyle
16\pi^{2}\,\frac{d\,\mathcal{C}^{(6)}_{H\mathcal{D}}\,(\mu)}{d\ln{\mu}}$
$\displaystyle=$
$\displaystyle\frac{20}{3}\,g_{Y}^{2}\,\mathcal{C}_{H\square}^{(6)},$
$\displaystyle\Longrightarrow\,\mathcal{C}_{H\mathcal{D}}^{(6)}\,|_{\text{RGE}}$
$\displaystyle=$
$\displaystyle-\frac{5\,g_{Y}^{2}\eta_{\mathcal{S}}^{2}}{24\,\pi^{2}\,M_{\mathcal{S}}^{4}}\,\log{\left[\frac{M_{Z}}{M_{\mathcal{S}}}\right]}.$
(12)

So the total contribution to $\mathcal{C}_{H\mathcal{D}}$ at the EW scale is:

$\displaystyle\mathcal{C}_{H\mathcal{D}}^{(6)}(M_{Z})={-\frac{7g_{Y}^{2}\eta_{\mathcal{S}}^{2}}{288\,M_{\mathcal{S}}^{4}\,\pi^{2}}}-\frac{5\,g_{Y}^{2}\eta_{\mathcal{S}}^{2}}{24\,\pi^{2}\,M_{\mathcal{S}}^{4}}\,\log{\left[\frac{M_{Z}}{M_{\mathcal{S}}}\right]}.$
(13)

The part of the beta function (cf. Eq. (• ‣ 2.2)) for the dimension eight
Wilson coefficient $\mathcal{C}_{H\mathcal{D},2}^{(8)}$ and
$\mathcal{C}_{HWB}^{(8)}$ stems from

$\displaystyle
16\pi^{2}\,\frac{d\,\mathcal{C}_{H\mathcal{D},2}^{(8)}\,(\mu)}{d\ln{\mu}}$
$\displaystyle=$
$\displaystyle\frac{40}{3}\,g_{Y}^{2}\,(\mathcal{C}_{H\square}^{(6)})^{2}\,+\cfrac{10}{3}g_{Y}^{2}\mathcal{C}_{H\mathcal{D},1}^{(8)}\,+\mathcal{C}_{H^{4}D^{4},3}^{(8)}\left(-\cfrac{11}{24}g_{Y}^{4}-\cfrac{79}{48}g_{Y}^{2}g_{W}^{2}+3g_{Y}^{2}\lambda_{h}\right)$
$\displaystyle\Longrightarrow\,\mathcal{C}_{H\mathcal{D},2}^{(8)}\,|_{\text{RGE}}$
$\displaystyle=$
$\displaystyle\cfrac{1}{16\pi^{2}}\Bigg{[}\frac{10\,g_{Y}^{2}\eta_{\mathcal{S}}^{4}}{3\,M_{\mathcal{S}}^{8}}+\cfrac{10}{3}g_{Y}^{2}\left(\frac{4\eta_{\mathcal{S}}^{2}k_{\mathcal{S}}}{M_{\mathcal{S}}^{6}}-\frac{8\lambda_{\mathcal{S}}\eta_{\mathcal{S}}^{2}k_{\mathcal{S}}}{M_{\mathcal{S}}^{6}}\right)$
(14)
$\displaystyle+\cfrac{2\eta_{\mathcal{S}}^{2}}{M_{\mathcal{S}}^{6}}\left(-\cfrac{11}{24}g_{Y}^{4}-\cfrac{79}{48}g_{Y}^{2}g_{W}^{2}+3g_{Y}^{2}\lambda_{h}\right)\Bigg{]}\log{\left[\frac{M_{Z}}{M_{\mathcal{S}}}\right]},$
$\displaystyle 16\pi^{2}\,\frac{d\,\mathcal{C}_{HWB}^{(8)}\,(\mu)}{d\ln{\mu}}$
$\displaystyle=$
$\displaystyle\mathcal{C}_{H^{4}D^{4},3}^{(8)}\left(-\cfrac{11}{48}g_{Y}^{3}g_{W}-\cfrac{29}{24}g_{Y}g_{W}^{3}+g_{Y}g_{W}\lambda_{h}\right)$
$\displaystyle\Longrightarrow\,\mathcal{C}_{HWB}^{(8)}\,|_{\text{RGE}}$
$\displaystyle=$
$\displaystyle\cfrac{\eta_{\mathcal{S}}^{2}}{8\pi^{2}M_{\mathcal{S}}^{6}}\left(-\cfrac{11}{48}g_{Y}^{3}g_{W}-\cfrac{29}{24}g_{Y}g_{W}^{3}+g_{Y}g_{W}\lambda_{h}\right)\log{\left[\frac{M_{Z}}{M_{\mathcal{S}}}\right]},$
(15)

with $g_{Y}=e/c_{W},g_{W}=e/s_{W}$. In addition to contributions from
dimension six effective operators, we also compute the contribution to
dimension eight operators from the equations of motion of the dimension six
operators and the RGE-improved corrections due to dimension six operators, see
further Refs. Li:2020gnx ; Murphy:2020rsh ; Chala:2021pll ; DasBakshi:2022mwk
. We note that the correction to $\Delta\widehat{T}$ and $\Delta\widehat{S}$
due to the dimension eight inclusion is of the order of the deviations. Thus,
dimension eight interactions may be crucial to bringing EFT predictions close
to the full theory for given measured constraints.

##  3 Full theory vs EFT

In this section, we compare a full theory and its effective version captured
in SMEFT. We carefully investigate how the inclusion of higher mass
dimensional operators, suppressed by the mass of the heavy integrated-out
field, in the EFT expansion affects the computation of our chosen observable
$(\widehat{T},\widehat{S})$. For this, we categorize the EFT contribution into
three parts. To start with, we discuss the dimension six $(d_{6})$ part, which
contributes at ${\mathcal{O}}(M_{\mathcal{S}}^{-2})$ containing linear
dimension six Wilson coefficients (WCs). Here, we include the cumulative
effects of field redefinition and radiative corrections on the oblique
parameters. Then, we consider the corrections from the dimension eight
$(d_{8})$ operators that are linear functions of dimension eight WCs at the
${\mathcal{O}}(M_{\mathcal{S}}^{-4})$. We also include the dimension eight
equivalent contributions (referred to as $(d_{6})^{2}$), at
${\mathcal{O}}(M_{\mathcal{S}}^{-4})$, from dimension six operators which are
quadratic functions of dimension six WCs. This takes care of the radiative
generation of dimension eight operators from dimension six ones, see Eq. (• ‣
2.2) and the expansion of $Z_{h}^{2}$. We list all those operators that
contribute in different orders:

  * •

$d_{6}$:
$\mathcal{C}_{HD}^{(6)},\;\mathcal{C}_{HWB}^{(6)},\;\mathcal{C}_{H\square}^{(6)}$
;

  * •

$d_{8}$:
$\mathcal{C}_{H\mathcal{D},1}^{(8)},\;\mathcal{C}_{H\mathcal{D},2}^{(8)},\;\mathcal{C}_{H^{4}D^{4},3}^{(8)},\;\mathcal{C}_{HWB}^{(8)}$
;

  * •

$(d_{6})^{2}:$ $(\mathcal{C}_{H\square}^{(6)})^{2}$ .

Operator |  Op. Structure |  Wilson coeffs.
---|---|---
$\mathcal{O}_{HD,1}^{(8)}$ | $(H^{\dagger}H)^{2}\,(\mathcal{D}_{\mu}H^{\dagger}\mathcal{D}_{\mu}H)$ | $\frac{4\eta_{\mathcal{S}}^{2}k_{\mathcal{S}}}{M_{\mathcal{S}}^{6}}-\frac{8\lambda_{\mathcal{S}}\eta_{\mathcal{S}}^{2}k_{\mathcal{S}}}{M_{\mathcal{S}}^{6}}$
$\mathcal{O}_{H^{4}D^{4},3}^{(8)}$ | $(\mathcal{D}_{\mu}H^{\dagger}\mathcal{D}^{\mu}H)(\mathcal{D}_{\nu}H^{\dagger}\mathcal{D}^{\nu}H)$ | $\frac{2\eta_{\mathcal{S}}^{2}}{M_{\mathcal{S}}^{6}}$

Table 2: Relevant operators that produce tree and one-loop corrections to the
gauge boson self-energy. The structures in blue first appear at tree-level
correction, whereas the rest of the operators contribute at one-loop first.

(a) (b)

Figure 4: (a) shows the variation of the trilinear coupling with respect to
heavy scalar mass. The orange and yellow dots denote $\eta_{\mathcal{S}}$
values to reproduce values of the respective $\cos^{2}\theta$ choices. In (b)
we show how the mixing angle is a function of the heavy mass for fixed values
of the trilinear coupling $\eta_{\mathcal{S}}$. In both plots, the gray-shaded
region respects perturbative unitarity (see appendix B). We can give two plots
separately. The first one shows how large cos theta can be achieved for wide
ranges of m. The second one is the allowed range of theta that can be achieved
by minimizing $\eta_{S}$. Figure 5: Impact of individual contributions on
$\Delta\widehat{T}=|\widehat{T}_{\text{Full}}-\widehat{T}_{\text{eft}}|$ for
three benchmark choices of $\eta_{s}=M_{S}$, having maximal allowed mixing.
$\Delta\widehat{T}$, computed upto $\mathcal{O}(M_{S}^{-2})$, i.e., truncating
effective Lagrangian at mass dimension six, receives a positive contribution
from dimension eight operators. Though, the total contributions upto order
$\mathcal{O}(M_{S}^{-2})$ reduces $\Delta\widehat{T}$ signifying inclusion of
dimension eight operators brings the effective theory prediction relatively
closer to that from the full theory.  Figure 6: Impact of individual
contributions on
$\Delta\widehat{S}=|\widehat{S}_{\text{Full}}-\widehat{S}_{\text{eft}}|$ for
three benchmark choices of $\eta_{s}=M_{S}$. We note that adding the RG
evolution contribution from dimension eight operators improves the effective
theory to replicate the full theory.

(a) (b)

(c)

Figure 7: Difference between full theory and the EFT computation for
$\widehat{T}$ parameter at different heavy field mass scales. The mass scales
are chosen to be (a) 700 GeV, (b) 1 TeV and (c) 5 TeV. The values for
$\eta_{S}$ are chosen such that they satisfy the unitarity bounds.

(a) (b)

(c)

Figure 8: Difference between full theory and the EFT computation for
$\widehat{S}$ parameter at different heavy field mass scales. The mass scales
are chosen to be (a) 700 GeV, (b) 1 TeV and (c) 5 TeV. The values for
$\eta_{S}$ are chosen such that they satisfy the unitarity bounds.

We investigate the departure of the truncated-EFT computation at dimension six
from the full theory calculations and the role that the Higgs mixing plays in
matching these two. The mixing can be expressed as a function of the trilinear
coupling $\eta_{S}$ and the heavy cut-off scale $M_{\mathcal{S}}$, for allowed
$\eta_{S}$ values, the decoupling can be quantified through the difference of
the two theories. In Fig. 4(a), we show the lines for the constant mixing
angles that allow a single $\eta_{S}$ value for each choice of the cut-off. We
also impose the constraint from the perturbative unitarity that rules out a
specific region in the $\eta_{S}-M_{\mathcal{S}}$ plane, in turn putting a
lower-bound for the mixing for each value of the cut-off $M_{\mathcal{S}}$,
that can be seen in Fig. 4(b).

Intuitively, adding higher and higher order terms in EFT expansion would take
the EFT closer to the full theory. This concept is illustrated through the
$\widehat{T}$ parameter in Fig. 5. Here, we consider three different types of
contributions. Firstly, the leading order terms in the expansion, i.e.,
$d_{6}$ ones. Then, we add $d_{8}$ contributions and finally, the
$(d_{6})^{2}$ ones. In passing, we want to highlight that though the $d_{8}$
term adds positively to the difference between the full theory and EFT, the
further addition of $(d_{6})^{2}$, the equivalent of $d_{8}$ ones, allows us
to capture the complete contribution at $\mathcal{O}(M_{\mathcal{S}}^{-4})$.
Ultimately, it reflects that going to higher order in EFT expansion reduces
the gap between full theory and EFT, especially for a relatively large mixing.
We perform similar analyses for $\widehat{S}$ parameter in Fig. 6. We draw a
similar conclusion as the previous one, and that makes our conclusion more
generic.

In Fig. 7, we have calculated the difference between full theory and EFT in
calculating the $\widehat{T}$ parameter for three different heavy mass scales.
In each subfigure, we have shown that if we lower the value of $\eta_{S}$ for
a fixed mass, the value of $\cos\theta$ increases. As the $\cos\theta$ reaches
unity, the full theory and EFT are in excellent agreement, which is expected
as the new physics contribution vanishes. It is also evident that for a fixed
$\eta_{S}$, once we go for higher masses the difference also decreases. This
illustrates the interplay among the coupling $\eta_{S}$, the heavy mass scale
$M_{\mathcal{S}}$, and mixing parameter $\cos\theta$. One can tune the value
of these parameters so that EFT can be a good explanation for the full theory.
Doing the same kind of investigation for the $\widehat{S}$ parameter in Fig. 8
further emphasises the idea.

##  4 Summary and Conclusions

Effective Field Theory is a powerful tool to look for deviations from the SM
expectation in a theoretically well-motivated way. In a modern sense, it
enables us to extend good quantum field theoretic properties to generic
departures from the SM interactions, with potential relevance for UV complete
scenarios depending on the accuracy with which constraints can be formulated.
Along these lines, a set of particularly well-motivated observables are the
oblique corrections as a subset of relevant electroweak corrections. In this
work, we have analysed these observables from their dimension six and eight
points of view with a critical perspective on how accurately EFT methods
describe the full theory in the singlet extension scenarios where decoupling
and alignment limits are exceptionally transparent. As expected, EFT
approximates the full theory well in regions where it is valid. However,
moving away from the alignment/decoupling limit, the relevance of the higher-
dimensional terms in the EFT expansion quickly becomes relevant. Although
these ranges are currently not probed by the experiments, it demonstrates the
need to include higher-dimensional corrections (chiefly squared dimension six
terms) to well-approximate the full theory. Furthermore, as Higgs boson mixing
is a feature in almost all BSM theories with a non-minimal Higgs sector, this
shows the necessity to go beyond dimension six interactions when data is very
precise or when we want to inform a potential UV scenario accurately.

## Acknowledgements

C.E. is supported by the STFC under grant ST/T000945/1, by the Leverhulme
Trust under grant RPG-2021-031, and the IPPP Associateship Scheme. M.S. is
supported by the STFC under grant ST/P001246/1. W.N. is funded by a University
of Glasgow College of Science and Engineering Scholarship.

##  Appendix A Gauge boson two-point functions

###  A.1 Modification due to a singlet scalar extension

We note down the modifications to the gauge boson two-point functions due to
the presence of a new heavier scalar degree of freedom. Here, only the
contributions from the scalar-involved diagrams are presented. The BSM
contribution to the two-point functions (in Feynman gauge) are then
Hahn:2000kx

$\displaystyle\Pi_{ZZ}(p^{2})$ $\displaystyle=$
$\displaystyle\left[-\frac{M_{W}^{4}B_{0}\left(p^{2},M_{h}^{2},M_{Z}^{2}\right)}{4\pi^{2}c_{W}^{4}v^{2}}+\frac{M_{W}^{2}B_{00}\left(p^{2},M_{h}^{2},M_{Z}^{2}\right)}{4\pi^{2}c_{W}^{2}v^{2}}-\frac{M_{h}^{2}M_{W}^{2}\left(1-\log\left(\frac{M_{h}^{2}}{\mu^{2}}\right)\right)}{16\pi^{2}c_{W}^{2}v^{2}}\right]$
(16)
$\displaystyle\cos^{2}{\theta}+\left[-\frac{M_{W}^{4}B_{0}\left(p^{2},M_{\cal{S}}^{2},M_{Z}^{2}\right)}{4\pi^{2}c_{W}^{4}v^{2}}+\frac{M_{W}^{2}B_{00}\left(p^{2},M_{\cal{S}}^{2},M_{Z}^{2}\right)}{4\pi^{2}c_{W}^{2}v^{2}}\right.$
$\displaystyle\left.-\frac{M_{\cal{S}}^{2}M_{W}^{2}\left(1-\log\left(\frac{M_{\cal{S}}^{2}}{\mu^{2}}\right)\right)}{16\pi^{2}c_{W}^{2}v^{2}}\right]\sin^{2}{\theta},$
$\displaystyle\Pi_{WW}(p^{2})$ $\displaystyle=$
$\displaystyle\left[-\frac{M_{W}^{4}B_{0}\left(p^{2},M_{h}^{2},M_{W}^{2}\right)}{4\pi^{2}v^{2}}+\frac{M_{W}^{2}B_{00}\left(p^{2},M_{h}^{2},M_{W}^{2}\right)}{4\pi^{2}v^{2}}-\frac{M_{h}^{2}M_{W}^{2}\left(1-\log\left(\frac{M_{h}^{2}}{\mu^{2}}\right)\right)}{16\pi^{2}v^{2}}\right]$
(17)
$\displaystyle\cos^{2}{\theta}+\left[-\frac{M_{W}^{4}B_{0}\left(p^{2},M_{\cal{S}}^{2},M_{W}^{2}\right)}{4\pi^{2}v^{2}}+\frac{M_{W}^{2}B_{00}\left(p^{2},M_{\cal{S}}^{2},M_{W}^{2}\right)}{4\pi^{2}v^{2}}\right.$
$\displaystyle-\left.\frac{M_{\cal{S}}^{2}M_{W}^{2}\left(1-\log\left(\frac{M_{\cal{S}}^{2}}{\mu^{2}}\right)\right)}{16\pi^{2}v^{2}}\right]\sin^{2}{\theta},$
$\displaystyle\Pi_{\gamma\gamma}(p^{2})$ $\displaystyle=$
$\displaystyle\Pi_{\gamma Z}(p^{2})=0\,,$ (18)

where, the Passarino-Veltman functions Passarino:1978jh  (see also
Denner:1991kt ; Denner:2019vbn ) $A_{0}$, $B_{0}$ and $B_{00}$ capture the
scalar one-loop dynamics (the vev is fixed via $v=2M_{W}s_{W}/e$). We have
cross checked these results numerically against previous results Bowen:2007ia
; Englert:2011yb .

###  A.2 Modification due to the corresponding EFT at tree-level

We note down the tree-level correction to the gauge boson propagators as shown
in Fig. 3 due to the presence of effective operators.

$\displaystyle\Pi_{WW}^{(\text{EFT})}(p^{2})$ $\displaystyle=$
$\displaystyle\frac{g_{W}^{2}\,v^{6}}{16}\mathcal{C}_{H\mathcal{D},1}^{(8)}-\frac{g_{W}^{2}\,v^{6}}{16}\mathcal{C}_{H\mathcal{D},2}^{(8)}+p^{2}v^{4}\,\mathcal{C}_{HW}^{(8)}+2p^{2}v^{2}\mathcal{C}_{HW}^{(6)},$
(19) $\displaystyle\Pi_{ZZ}^{(\text{EFT})}(p^{2})$ $\displaystyle=$
$\displaystyle\frac{c_{W}^{2}g_{W}^{2}v^{6}}{16}\mathcal{C}_{H\mathcal{D},1}^{(8)}+\frac{c_{W}^{2}g_{W}^{2}v^{6}}{16}\mathcal{C}_{H\mathcal{D},2}^{(8)}+\frac{c_{W}^{2}g_{W}^{2}v^{4}}{8}\mathcal{C}_{H\mathcal{D}}^{(6)}+c_{W}^{2}p^{2}v^{4}\mathcal{C}_{HW}^{(8)}$
(20)
$\displaystyle+2c_{W}^{2}p^{2}v^{2}\mathcal{C}_{HW}^{(6)}+\frac{c_{W}s_{W}g_{W}g_{Y}v^{6}}{8}\mathcal{C}_{H\mathcal{D},1}^{(8)}+\frac{c_{W}s_{W}g_{W}g_{Y}v^{6}}{8}\mathcal{C}_{H\mathcal{D},2}^{(8)}$
$\displaystyle+\frac{c_{W}s_{W}g_{W}g_{Y}v^{4}}{4}\mathcal{C}_{H\mathcal{D}}^{(6)}+p^{2}c_{W}s_{W}v^{4}\,\mathcal{C}_{HWB}^{(8)}+2p^{2}c_{W}s_{W}v^{2}\mathcal{C}_{{HWB}}^{(6)}$
$\displaystyle+\frac{s_{W}^{2}g_{Y}^{2}v^{6}\mathcal{C}_{H\mathcal{D},1}^{(8)}}{16}+\frac{s_{W}^{2}g_{Y}^{2}v^{6}\mathcal{C}_{H\mathcal{D},2}^{(8)}}{16}+\frac{s_{W}^{2}g_{Y}^{2}v^{4}\mathcal{C}_{H\mathcal{D}}^{(6)}}{8}$
$\displaystyle+p^{2}s_{W}^{2}v^{4}\mathcal{C}_{HB}^{(8)}+2p^{2}s_{W}^{2}v^{2}\mathcal{C}_{{HB}}^{(6)},$
$\displaystyle\Pi_{\gamma\gamma}^{(\text{EFT})}(p^{2})$ $\displaystyle=$
$\displaystyle\frac{c_{W}^{2}g_{Y}^{2}v^{6}}{16}\mathcal{C}_{H\mathcal{D},1}^{(8)}+\frac{c_{W}^{2}g_{Y}^{2}v^{6}}{16}\mathcal{C}_{H\mathcal{D},2}^{(8)}+\frac{c_{W}^{2}g_{Y}^{2}v^{4}}{8}\mathcal{C}_{H\mathcal{D}}^{(6)}+p^{2}c_{W}^{2}v^{4}\mathcal{C}_{HB}^{(8)}$
(21)
$\displaystyle+2p^{2}c_{W}^{2}v^{2}\mathcal{C}_{{HB}}^{(6)}-\frac{c_{W}s_{W}g_{W}g_{Y}v^{6}}{8}\mathcal{C}_{H\mathcal{D},1}^{(8)}-\frac{c_{W}s_{W}g_{W}g_{Y}v^{6}}{8}\mathcal{C}_{H\mathcal{D},2}^{(8)}$
$\displaystyle-\frac{g_{W}g_{Y}c_{W}s_{W}v^{4}}{4}\mathcal{C}_{H\mathcal{D}}^{(6)}-p^{2}c_{W}s_{W}v^{4}\mathcal{C}_{HWB}^{(8)}-2p^{2}c_{W}s_{W}v^{2}\mathcal{C}_{HWB}^{(6)}$
$\displaystyle+\frac{s_{W}^{2}g_{W}^{2}v^{6}}{16}\mathcal{C}_{H\mathcal{D},1}^{(8)}+\frac{s_{W}^{2}g_{W}^{2}v^{6}}{16}\mathcal{C}_{H\mathcal{D},2}^{(8)}+\frac{s_{W}^{2}g_{W}^{2}v^{6}}{8}\mathcal{C}_{H\mathcal{D}}^{(6)}$
$\displaystyle+p^{2}s_{W}^{2}v^{4}\mathcal{C}_{HW}^{(8)}+2p^{2}s_{W}^{2}v^{2}\mathcal{C}_{HW}^{(6)},$
$\displaystyle\Pi_{\gamma Z}^{(\text{EFT})}(p^{2})$ $\displaystyle=$
$\displaystyle-\frac{c_{W}^{2}g_{W}g_{Y}v^{6}}{16}\mathcal{C}_{H\mathcal{D},1}^{(8)}-\frac{c_{W}^{2}g_{W}g_{Y}v^{6}}{16}\mathcal{C}_{H\mathcal{D},2}^{(8)}-\frac{c_{W}^{2}g_{W}g_{Y}v^{4}}{8}\mathcal{C}_{H\mathcal{D}}^{(6)}$
(22)
$\displaystyle-\frac{p^{2}c_{W}^{2}v^{4}}{2}\mathcal{C}_{HWB}^{(8)}-p^{2}c_{W}^{2}v^{2}\mathcal{C}_{{HWB}}^{(6)}+\frac{c_{W}s_{W}g_{W}^{2}v^{6}}{16}\mathcal{C}_{H\mathcal{D},1}^{(8)}+\frac{c_{W}s_{W}g_{W}^{2}v^{6}}{16}\mathcal{C}_{H\mathcal{D},2}^{(8)}$
$\displaystyle+\frac{c_{W}s_{W}g_{W}^{2}v^{4}}{8}\mathcal{C}_{H\mathcal{D}}^{(6)}-\frac{c_{W}s_{W}g_{Y}^{2}v^{6}}{16}\mathcal{C}_{H\mathcal{D},1}^{(8)}-\frac{c_{W}s_{W}g_{Y}^{2}v^{6}}{16}\mathcal{C}_{H\mathcal{D},2}^{(8)}$
$\displaystyle-\frac{c_{W}s_{W}g_{Y}^{2}v^{4}}{8}\mathcal{C}_{H\mathcal{D}}^{(6)}-p^{2}c_{W}s_{W}v^{4}\mathcal{C}_{HB}^{(8)}+p^{2}c_{W}s_{W}v^{4}\mathcal{C}_{HW}^{(8)}-2p^{2}c_{W}s_{W}v^{2}C_{HB}^{(6)}$
$\displaystyle+2p^{2}c_{W}s_{W}v^{2}\mathcal{C}_{HW}^{(6)}+\frac{s_{W}^{2}g_{W}g_{Y}v^{6}}{16}\mathcal{C}_{H\mathcal{D},1}^{(8)}+\frac{s_{W}^{2}g_{W}g_{Y}v^{6}}{16}\mathcal{C}_{H\mathcal{D},2}^{(8)}+\frac{s_{W}^{2}g_{W}g_{Y}v^{4}}{8}\mathcal{C}_{H\mathcal{D}}^{(6)}$
$\displaystyle+\frac{p^{2}s_{W}v^{4}}{2}\mathcal{C}_{HWB}^{(8)}+p^{2}s_{W}^{2}v^{2}\mathcal{C}_{HWB}^{(6)}.$

The couplings are given by $g_{W}=e/s_{W},g_{Y}=e/c_{W}$.

##  Appendix B Unitarity Constraints

Unitarity provides a suitable tool to gauge whether the matching is indeed for
perturbative choices of the UV model parameters. Perturbativity, in one way or
another, is implicitly assumed in analysing any collider data and this extends
to the electroweak precision constraints as well. To this end, we consider the
partial wave constraints that can be derived from longitudinal gauge boson
scattering to identify the regions of validity this way. The zeroth partial
wave relevant for this is given for scattering $i_{1}\,i_{2}\to f_{1}\,f_{2}$
(see Ref. Jacob:1959at )

$a_{fi}^{0}=\frac{\beta^{1/4}(s,m_{i,1}^{2},m_{i,1}^{2})\,\beta^{1/4}(s,m_{f,1}^{2},m_{f,1}^{2})}{32\pi
s}\int_{-1}^{1}\text{d}\cos\theta\,\,{\cal{M}}(\sqrt{s},\cos\theta)\,,$ (23)

suppressing factors of $1/\sqrt{2}$ for identical particles in the initial $i$
or final state $f$. $\sqrt{s}$ denotes the centre-of-mass energy, and $\theta$
is the scattering angle in this frame for the $2\to 2$ scattering process
described by the amplitude $\sim{\cal{M}}$. Furthermore,

$\beta(x,y,z)=x^{2}+y^{2}+z^{2}-2xy-2yz-2xz\,.$ (24)

such that $\lim_{s\to\infty}\beta^{2}/s=1$. Unitarity of the $S$ matrix then
translates for $f=i$ to the conditions

$|\text{Re}\,a_{ii}^{0}|\leq\frac{1}{2}\quad\hbox{and}\quad|\text{Im}\,a_{ii}^{0}|\leq
1\,.$ (25)

of which we use the first one to obtain the constraints in Sec. 2. The
presence of large $\eta_{S}\sim M_{\cal{S}}$ leads to unitarity violation
through ${\cal{S}}$ contributions to $hh\to hh$ via the $s,t,u$ channels as
well as large values of $M_{\cal{S}}$ for $\cos^{2}\theta\neq 1$ in
longitudinal gauge boson scattering Lee:1977eg . Numerical investigation shows
that for our choice close to the alignment limits longitudinal unitarity
constraints are not as relevant as $hh$ scattering constraints. Assuming
perturbative unitarity up to a cut-off scale $\sim M_{\cal{S}}$ requires
$\eta_{S}\lesssim M_{\cal{S}}$. This reflects the fact that when a
dimensionful coupling (i.e. a mass scale) such as $\eta_{S}$ becomes
comparable to a UV cut-off ($M_{\cal{S}}$ in the EFT description), we enter
strong coupling. This is also visible from the expansion of phenomenologically
relevant quantities such as Eq. (6), which scales $\sim
v\eta_{S}/M_{\cal{S}}^{2}$ in the EFT regime $M_{\cal{S}}\gg M_{h}$.

## References

  * (1) S. Weinberg, _Phenomenological Lagrangians_ , _Physica A_ 96 (1979) 327–340.
  * (2) B. Grzadkowski, M. Iskrzynski, M. Misiak and J. Rosiek, _Dimension-Six Terms in the Standard Model Lagrangian_ , _JHEP_ 10 (2010) 085, [1008.4884].
  * (3) M. Jiang, N. Craig, Y.-Y. Li and D. Sutherland, _Complete one-loop matching for a singlet scalar in the Standard Model EFT_ , _JHEP_ 02 (2019) 031, [1811.08878].
  * (4) U. Haisch, M. Ruhdorfer, E. Salvioni, E. Venturini and A. Weiler, _Singlet night in Feynman-ville: one-loop matching of a real scalar_ , _JHEP_ 04 (2020) 164, [2003.05936].
  * (5) V. Gherardi, D. Marzocca and E. Venturini, _Matching scalar leptoquarks to the SMEFT at one loop_ , _JHEP_ 07 (2020) 225, [2003.12525].
  * (6) Y. Du, X.-X. Li and J.-H. Yu, _Neutrino seesaw models at one-loop matching: discrimination by effective operators_ , _JHEP_ 09 (2022) 207, [2201.04646].
  * (7) S. Dawson and C. W. Murphy, _Standard Model EFT and Extended Scalar Sectors_ , _Phys. Rev. D_ 96 (2017) 015041, [1704.07851].
  * (8) D. Zhang and S. Zhou, _Complete one-loop matching of the type-I seesaw model onto the Standard Model effective field theory_ , _JHEP_ 09 (2021) 163, [2107.12133].
  * (9) S. Dittmaier, S. Schuhmacher and M. Stahlhofen, _Integrating out heavy fields in the path integral using the background-field method: general formalism_ , _Eur. Phys. J. C_ 81 (2021) 826, [2102.12020].
  * (10) X. Li, D. Zhang and S. Zhou, _One-loop matching of the type-II seesaw model onto the Standard Model effective field theory_ , _JHEP_ 04 (2022) 038, [2201.05082].
  * (11) D. Zhang, _Complete One-loop Structure of the Type-(I+II) Seesaw Effective Field Theory_ , 2208.07869.
  * (12) A. Carmona, A. Lazopoulos, P. Olgoso and J. Santiago, _Matchmakereft: automated tree-level and one-loop matching_ , _SciPost Phys._ 12 (2022) 198, [2112.10787].
  * (13) T. Cohen, X. Lu and Z. Zhang, _STrEAMlining EFT Matching_ , _SciPost Phys._ 10 (2021) 098, [2012.07851].
  * (14) S. Das Bakshi, J. Chakrabortty and S. K. Patra, _CoDEx: Wilson coefficient calculator connecting SMEFT to UV theory_ , _Eur. Phys. J. C_ 79 (2019) 21, [1808.04403].
  * (15) C. W. Murphy, _Dimension-8 Operators in the Standard Model Effective Field Theory_ , 2005.00059.
  * (16) H.-L. Li, Z. Ren, J. Shu, M.-L. Xiao, J.-H. Yu and Y.-H. Zheng, _Complete set of dimension-eight operators in the standard model effective field theory_ , _Phys. Rev. D_ 104 (2021) 015026, [2005.00008].
  * (17) U. Banerjee, J. Chakrabortty, C. Englert, S. U. Rahaman and M. Spannowsky, _Integrating out heavy scalars with modified EOMs: matching computation of dimension-eight SMEFT coefficients_ , 2210.14761.
  * (18) S. Dawson, D. Fontes, S. Homiller and M. Sullivan, _Role of dimension-eight operators in an EFT for the 2HDM_ , _Phys. Rev. D_ 106 (2022) 055012, [2205.01561].
  * (19) S. Dawson, P. P. Giardino and S. Homiller, _Uncovering the High Scale Higgs Singlet Model_ , _Phys. Rev. D_ 103 (2021) 075016, [2102.02823].
  * (20) G. M. Pruna and T. Robens, _Higgs singlet extension parameter space in the light of the LHC discovery_ , _Phys. Rev. D_ 88 (2013) 115012, [1303.1150].
  * (21) T. Binoth and J. J. van der Bij, _Influence of strongly coupled, hidden scalars on Higgs signals_ , _Z. Phys. C_ 75 (1997) 17–25, [hep-ph/9608245].
  * (22) M. Bowen, Y. Cui and J. D. Wells, _Narrow trans-TeV Higgs bosons and H — $>$ hh decays: Two LHC search paths for a hidden sector Higgs boson_, _JHEP_ 03 (2007) 036, [hep-ph/0701035].
  * (23) C. Englert, T. Plehn, D. Zerwas and P. M. Zerwas, _Exploring the Higgs portal_ , _Phys. Lett. B_ 703 (2011) 298–305, [1106.3097].
  * (24) B. Batell, S. Gori and L.-T. Wang, _Exploring the Higgs Portal with 10/fb at the LHC_ , _JHEP_ 06 (2012) 172, [1112.5180].
  * (25) Anisha, S. Das Bakshi, S. Banerjee, A. Biekötter, J. Chakrabortty, S. Kumar Patra et al., _Effective limits on single scalar extensions in the light of recent LHC data_ , 2111.05876.
  * (26) M. E. Peskin and T. Takeuchi, _Estimation of oblique electroweak corrections_ , _Phys. Rev. D_ 46 (Jul, 1992) 381–409.
  * (27) G. Altarelli and R. Barbieri, _Vacuum polarization effects of new physics on electroweak processes_ , _Physics Letters B_ 253 (1991) 161–167.
  * (28) D. Y. Bardin, P. Christova, M. Jack, L. Kalinovskaya, A. Olchevski, S. Riemann et al., _ZFITTER v.6.21: A Semianalytical program for fermion pair production in $e^{+}e^{-}$ annihilation_, _Comput. Phys. Commun._ 133 (2001) 229–395, [hep-ph/9908433].
  * (29) H. Flacher, M. Goebel, J. Haller, A. Hocker, K. Monig and J. Stelzer, _Revisiting the Global Electroweak Fit of the Standard Model and Beyond with Gfitter_ , _Eur. Phys. J. C_ 60 (2009) 543–583, [0811.0009].
  * (30) R. Barbieri, A. Pomarol, R. Rattazzi and A. Strumia, _Electroweak symmetry breaking after LEP-1 and LEP-2_ , _Nucl. Phys. B_ 703 (2004) 127–146, [hep-ph/0405040].
  * (31) J. Haller, A. Hoecker, R. Kogler, K. Mönig, T. Peiffer and J. Stelzer, _Update of the global electroweak fit and constraints on two-Higgs-doublet models_ , _Eur. Phys. J. C_ 78 (2018) 675, [1803.01853].
  * (32) E. E. Jenkins, A. V. Manohar and M. Trott, _Renormalization Group Evolution of the Standard Model Dimension Six Operators I: Formalism and lambda Dependence_ , _JHEP_ 10 (2013) 087, [1308.2627].
  * (33) R. Alonso, E. E. Jenkins, A. V. Manohar and M. Trott, _Renormalization Group Evolution of the Standard Model Dimension Six Operators III: Gauge Coupling Dependence and Phenomenology_ , _JHEP_ 04 (2014) 159, [1312.2014].
  * (34) M. Chala, G. Guedes, M. Ramos and J. Santiago, _Towards the renormalisation of the Standard Model effective field theory to dimension eight: Bosonic interactions I_ , _SciPost Phys._ 11 (2021) 065, [2106.05291].
  * (35) S. Das Bakshi, M. Chala, A. Díaz-Carmona and G. Guedes, _Towards the renormalisation of the Standard Model effective field theory to dimension eight: bosonic interactions II_ , _Eur. Phys. J. Plus_ 137 (2022) 973, [2205.03301].
  * (36) T. Hahn, _Generating Feynman diagrams and amplitudes with FeynArts 3_ , _Comput. Phys. Commun._ 140 (2001) 418–431, [hep-ph/0012260].
  * (37) G. Passarino and M. J. G. Veltman, _One Loop Corrections for e+ e- Annihilation Into mu+ mu- in the Weinberg Model_ , _Nucl. Phys. B_ 160 (1979) 151–207.
  * (38) A. Denner, _Techniques for calculation of electroweak radiative corrections at the one loop level and results for W physics at LEP-200_ , _Fortsch. Phys._ 41 (1993) 307–420, [0709.1075].
  * (39) A. Denner and S. Dittmaier, _Electroweak Radiative Corrections for Collider Physics_ , _Phys. Rept._ 864 (2020) 1–163, [1912.06823].
  * (40) M. Jacob and G. C. Wick, _On the General Theory of Collisions for Particles with Spin_ , _Annals Phys._ 7 (1959) 404–428.
  * (41) B. W. Lee, C. Quigg and H. B. Thacker, _Weak Interactions at Very High-Energies: The Role of the Higgs Boson Mass_ , _Phys. Rev. D_ 16 (1977) 1519.


# Towards Label-Efficient Incremental Learning: A Survey

Mert Kilickaya1, Joost van de Weijer2&Yuki M. Asano3 1Eindhoven University of
Technology
2Autonomous University of Barcelona
3University of Amsterdam<EMAIL_ADDRESS><EMAIL_ADDRESS><EMAIL_ADDRESS>
###### Abstract

The current dominant paradigm when building a machine learning model is to
iterate over a dataset over and over until convergence. Such an approach is
non-incremental, as it assumes access to all images of all categories at once.
However, for many applications, non-incremental learning is unrealistic. To
that end, researchers study incremental learning, where a learner is required
to adapt to an incoming stream of data with a varying distribution while
preventing forgetting of past knowledge. Significant progress has been made,
however, the vast majority of works focus on the fully supervised setting,
making these algorithms label-hungry thus limiting their real-life deployment.
To that end, in this paper, we make the first attempt to survey recently
growing interest in label-efficient incremental learning. We identify three
subdivisions, namely semi-, few-shot- and self-supervised learning to reduce
labeling efforts. Finally, we identify novel directions that can further
enhance label-efficiency and improve incremental learning scalability. Project
website: https://github.com/kilickaya/label-efficient-il.

##  1 Introduction

Deep learning is the dominant approach to build highly performant machine
learning systems that are deployed in a wide range of scenarios – from self-
driving cars to mobile applications. To this end, the classic approach is to
choose a current state-of-the-art neural network architecture, such as a
ResNet He et al. (2016) or a Vision-Transformer Dosovitskiy et al. (2020) and
combine this with large-scale datasets containing millions of annotations,
such as ImageNet. The learning of the model is done supervisedly by iterating
over the dataset multiple times until some convergence or stopping criterion
is met. We name such approaches non-incremental, as the learner assumes access
to the whole data at all times.

Non-incremental learning, which implicitly assumes a static world, has severe
limitations with regard to its performance and applicability: First, the list
of categories the model can distinguish from is fixed. Second, once the model
is deployed, the model no longer makes use of the ever-growing data with or
without labels to self-improve. Third, the ability to iterate over all the
data may be impossible due to privacy regulations or data storage legislation.

Motivated by these challenges, researchers put increasing attention on
incremental (or continual) learning Masana et al. (2020). In incremental
learning, the learner receives the learning tasks sequentially (e.g. first
dog, then cat, and then finally cow, see Figure 1), whose data then disappears
after some iterations. This way, a deep classifier can be updated with novel
data, while preserving the performance on the previous tasks. Yet, despite its
clear setting and large potential for scalable applications, why is the
incremental learning paradigm not used as often as the non-incremental
counterpart?

One reason is often cited is the phenomenon of catastrophic forgetting French
(1999). The performance of the model on the previous tasks deteriorates while
observing more incremental tasks. However, thanks to efforts in regularizing
neural network weights from abruptly drifting Li and Hoiem (2017); Zenke et
al. (2017), or replaying previous data from the memory Shin et al. (2017),
forgetfulness has reduced dramatically in years. In this paper, we raise the
attention of the incremental learning researchers to an equally important, yet
up to now unsolved, issue, Scalability. In particular, incremental learners
are extremely annotation-hungry, as they demand large amounts of labeled data
for achieving comparable performances.

The importance of label-efficient learning generally is well recognized in
non-incremental learning, as researchers organize dedicated workshops on the
topic, see L2ID (2022). Success has been made since researchers can build upon
large-scale pre-trained models to transfer learning on their individual tasks
with limited data Radford et al. (2021).

However, the importance of label-efficiency specifically for incremental
learning is yet to be recognized and addressed. While there have been few
studies that surveys conventional incremental learners Masana et al. (2020);
De Lange et al. (2021); Wang et al. (2023); Zhou et al. (2023), their focus is
still on the old label-hungry paradigm in incremental learning. This motivates
our effort in surveying recent techniques to build large-scale, label-
efficient incremental learners.

Figure 1: In incremental learning, a disjoint set of classes (a task) arrive
sequentially. Then, the learner is optimized to distinguish across these
categories. Here, we provide three different forms of label-efficient
incremental learning. a). Semi-Supervision pseudo labels unlabeled data to
reduce the need for labeled data, b). Few-shot-Supervision expands a pre-
trained classifier with only few labeled instances (i.e., 1-shot), c). Self-
Supervision designates pre-text tasks to completely remove the need for human
labels to train incremental learners. Here, the goal is to accumulate
representations by matching different views (augmentations such as cropping,
rotation and color jitter) of the same input image through incremental
learning. Best viewed in color.

We identify three main directions, see Figure 1. Semi-supervised learners
combine limited labeled data with abundant unlabeled data to reduce label-
supervision via pseudo supervision Wang et al. (2021). Few-shot-supervised
learners reduce label-supervision by expanding a pre-trained classifier with
only few examples (e.g. 1-shot) Tao et al. (2020). Finally, self-supervised
learners omit the need for label-supervision by designing pretext tasks from
the incremental learning tasks Fini et al. (2022).

Our survey first introduces the necessary notation in Section section 2, and
then focuses on Semi-Supervision (Section section 3), Few-shot-Supervision
(Section section 4) and Self-Supervision (Section section 5). We then conclude
with future research directions and novel problems in Section section 6.

| Setting | Supervision | Reference
---|---|---|---
Incremental Learning (IL) |  | Label-only | LwF Li and Hoiem (2017)
1-4  Semi-Supervised IL | Within-data | Pseudo & Label | CNLL Baucum et al. (2017)
Semi-Supervised IL | Auxiliary-data | Pseudo & Label | DMC Zhang et al. (2020)
Semi-Supervised IL | Test-data | Pseudo-only | CoTTA Wang et al. (2022)
Few-shot-Supervised IL | Graph-based | Label-only (Few) | TOPIC Tao et al. (2020)
Few-shot-Supervised IL | Clustering-based | Label-only (Few) | IDL-VQ Chen and Lee (2020)
Few-shot-Supervised IL | Architectural-based | Label-only (Few) | FSLL Mazumder et al. (2021)
Self-Supervised IL | Pre-training | Label-only | SSL-OCL Gallardo et al. (2021)
Self-Supervised IL | Auxiliary-training | Self & Label | PASS Zhu et al. (2021)
Self-Supervised IL | Main-training | Self-only | CaSSLe Fini et al. (2022)
Table 1: Towards reducing manual human supervision of incremental learners via
semi-, few-shot- and self-supervision. We list three subgroups we identify for
each method, in terms of data, method or training setting. For each group, we
provide the type of supervision(s) necessary to train the particular
incremental learner.

##  2 Label-Efficient Incremental Learning

There are three main settings, namely task-, domain- and class-incremental
learning, see Van de Ven and Tolias (2019). This short survey focuses on
class-incremental learning as it received more advances in the literature,
however the proposed subdivision can also be applied to other settings.

###  2.1 Incremental Learning

The main goal of a class-incremental learner is two-fold: i) To learn to
recognize novel classes in the current task, ii) To preserve the performance
on previously learned classes (i.e., reduce forgetfulness). A class-
incremental learning task is defined by the length of the learning sequence
and the number of classes within each task (i.e., Figure 1 depicts a 3-step
1-class incremental learning setting).

Formally, following the notation in Madaan et al. (2022), the learner receives
a sequence of learning tasks
$\mathcal{T}_{1:t}=(\mathcal{T}_{1},\mathcal{T}_{2},...,\mathcal{T}_{t})$,
with a corresponding dataset
$\mathcal{D}_{\mathcal{T}}=\\{(x_{i,t},y_{i,t})_{i=1}^{n_{t}}\\}$ with $n_{t}$
instances. Here, an input pair
$\\{x_{i,t},y_{i,t}\\}\in\mathcal{X}_{t}\times\mathcal{Y}_{t}$ is sampled from
an unknown distribution.

When the learning task arrives, then a deep convolutional feature extractor is
optimized $f_{\Theta}:\mathcal{X}_{t}\rightarrow\mathcal{R}^{D}$ and a linear
classifier $h_{\phi}:\mathcal{R}^{D}\rightarrow\mathcal{Y}_{t}$ jointly
parameterized by $W=\\{\Theta,\phi\\}$. The feature extractor maps the input
to a $D$-dimensional embedding space, and the linear classifier further
projects the input to the class space, by typically optimizing the following
objective: $\mathbf{CE}(h_{\phi}(f_{\Theta}(x_{i,t})),y_{i,t})$ where
$\mathbf{CE}(\cdot)$ is the standard Cross-Entropy loss to classify the input.
This way, the model is fine-tuned sequentially with the incoming stream of
task data.

Mitigating Forgetting. Fine-tuning with exemplars may lead to catastrophic
forgetting of past tasks, since the only objective is classification. To that
end, a simple technique is to only update a few layers closer to classifier
head. Two fundamental ways to prevent forgetfulness are regularization and
replay. Regularization often constraints neural network weights from abrupt
drifts across learning tasks (i.e., via penalizing the norm of change, simply
by $||W_{t-1}-W_{t}||$, see Zenke et al. (2017)). Replay often stores a subset
of the learning task data within the memory to replay during incremental
learning tasks, such as experience replay Rolnick et al. (2019).

Evaluation. Incremental learners are often evaluated by accuracy (higher is
better) and forgetfulness (lower is better). i) Accuracy measures the test
accuracy for all the learned tasks until the task $t$ as
$A_{t}=\dfrac{1}{t}\sum_{i=1}^{t}a_{t,i}$. ii) Forgetfulness: is the average
performance decrease while learning incremental tasks, after the completion of
the learning, as:
$F=\dfrac{1}{T-1}\sum_{i=1}^{T-1}\max_{t\in\\{1,...,t\\}}(a_{t,i}-a_{T,i})$
for $T$ total learning tasks.

###  2.2 Label-Efficient Learning

Semi-Supervision. Semi-supervision reduces the label need by leveraging
unlabeled data Yang et al. (2022). Researchers train deep learners on a small
subset of labeled data, and then produce pseudo labels on unlabeled data Sohn
et al. (2020). The pseudo labels are used for further self-training Sahito et
al. (2022); Zoph et al. (2020).

In incremental learning, this corresponds to partitioning the incremental
learning dataset into labeled and unlabeled splits as:
$\mathcal{D}_{t}=\mathcal{L}_{t}\cap\mathcal{U}_{t}$, where $\mathcal{L}_{t}$
is the standard labeled data whereas
$\mathcal{U}_{t}=\\{(x_{i,t})_{i=1}^{m_{t}}\\}$ is the unlabeled subset with
$m_{t}$ instances.

In this survey, we categorize semi-supervision based incremental learners by
the type of unlabeled data they use, whether from within the same dataset, an
auxiliary dataset or simply test data, see Table 1.

Few-shot-Supervision. Few-shot learning reduces the label need to only a few-
exemplars per-category, such as 1-shot or 5-shots. Prominent works either
resort to meta-learning Vinyals et al. (2016); Snell et al. (2017) or build
upon a deep pre-trained feature extractor Tian et al. (2020).

In Few-Shot Class Incremental Learning (FSCIL), the objective is to update a
pre-trained classifier with incrementally arriving classes with only few-
exemplars, while maintaining performance on the pre-trained classes Mazumder
et al. (2021); Tao et al. (2020). Formally, researchers introduce an initial
(non-incremental) pre-training task
$\mathcal{D}_{0}=\\{(x_{i,0},y_{i,0})_{i=1}^{n_{0}}\\}$ with abundant data-
label pairs ($n_{0}>>5$) whereas the subsequent task only has few instances
(i.e., 5-shots, such as $n_{t>0}=5$).

In this survey, we categorize few-shot-supervised incremental learners by
their core method, as in graph-based, clustering-based or architecture-based,
see Table 1.

Self-Supervision. Self-supervision omits the need for label-supervision by
designing pre-text tasks Chen et al. (2020). A promising direction is
contrastive learning, where the deep learner has to pull the features of the
original input and its augmented version (view) closer while pushing all the
other features away. Prominent examples include SWAV Caron et al. (2020), MOCO
Chen et al. (2020), and BarlowTwins Zbontar et al. (2021).

In incremental learning, self-supervision corresponds to training only the
feature extractor $f_{\Theta}(\cdot)$ solely on unlabeled data
$\mathcal{U}_{t}=\\{(x_{i,t})_{i=1}^{m_{t}}\\}$.

In this survey, we categorize self-supervised learners by how it is being
leveraged in incremental learning, whether for pre-training, auxiliary
training or as the main (sole) training objective, see Table 1.

##  3 Semi-Supervision for Incremental Learning

Incremental learners leverage semi-supervision with different forms of pseudo-
supervision, see Table 2. Pseudo-supervision serves the purpose of memory
replay to reduce forgetfulness of the previous categories.

Researchers replay labels-only Baucum et al. (2017); Smith et al. (2021),
data-only Gong et al. (2022), label and data Wang et al. (2021); Brahma et al.
(2021) and finally gradients Luo et al. (2022).

Here, we group semi-supervision-based incremental learners by their definition
of unlabeled data: A subset of the target training set (Within Data), an
auxiliary dataset such as from the Web, or simply test data. Within-data
learners start from scratch, whereas others build upon a pre-trained neural
network.

Algorithm Data Pre-training Replayed Entity CNNL Within ✗ Pseudo-labels
DistillMatch Within ✗ Pseudo-labels ORDisCo Within ✗ Pseudo-labels & Data
MetaCon Within ✗ Pseudo-labels & Data PGL Within ✗ Pseudo-gradients DMC
Auxiliary ✓ Pseudo-labels CIL-QUD Auxiliary ✓ Pseudo-labels CoTTA Test ✓
Pseudo-labels NOTE Test ✓ Data

Table 2: Incremental Learning with Semi-Supervision.

###  3.1 Learning from Within Data

CNNL. Continuous neural network learning Baucum et al. (2017) is one of the
early works at the intersection of semi-supervision and incremental learning.
The authors train a vanilla CNN on the labeled set, which is then used to
generate pseudo-labels on the unlabeled dataset. Finally, they fine-tune their
incremental learner on the pseudo-labels for self-training.

DistillMatch. DistillMatch Smith et al. (2021) follows a knowledge-
distillation procedure, where the predictions over the unlabeled data are
distilled between the current and the previous model. The authors further
optimize an out-of-distribution detector to identify data points sufficiently
different from the current incremental learning task (i.e., past examples).
This way, they mitigate forgetfulness of the previous categories.

ORDisCo. Online replay with discriminator consistency Wang et al. (2021)
follows a generative replay strategy to replay both the data and the labels.
The authors leverage labeled data to train a conditional GAN generator, and
leverage unlabeled data as additional examples for real-fake discrimination.
To improve consistency across incremental learning tasks, the authors penalize
abrupt changes within discriminator weights.

MetaCon. Meta-Consolidation Brahma et al. (2021) extends the generative replay
scheme of ORDisCO to meta-learning setting. Instead of directly training a
conditional GAN for generative replay, the authors instead optimize a
conditional hyper-network Ha et al. (2016) that generates GAN weights. The
authors use the semantic word embedding of the current task as the condition,
and parameterize the hyper-network as a Variational Auto-Encoder Kingma and
Welling (2013). To stabilize the hyper-network, they store the first-order
statistics of the incremental learning classes to replay.

PGL. Pseudo Gradient Learners Luo et al. (2022) moves away from the pseudo-
labels, and instead (meta-)learns to predict gradients per-input. The authors
claim that the use of pseudo-labels puts too much pressure on the classifier,
leading to error accumulation and performance degradation over time. By
predicting gradients instead of labels, the model is not tied to a pre-defined
set of classes as in pseudo-labelling, and is able to leverage out-of-
distribution data to improve performance.

###  3.2 Learning from Auxiliary Data

DMC. Deep Model Consolidation Zhang et al. (2020) is one of the earliest to
leverage unlabeled auxiliary data to mitigate forgetfulness. The authors first
train a deep classifier on the labeled set, which is then used to generate
pseudo-labels over auxiliary data. Pseudo-labels act as a regularizer between
the current and the previous model to reduce forgetfulness of previously seen
classes.

CIL-QUD. Class-Incremental Learning with Queried Unlabeled Data Chen et al.
(2022) builds upon DMC, however, instead takes on a retrieval-based approach.
The authors store few-instances per-incremental class within the memory as
query anchors. The query anchors are used to retrieve visually similar
exemplars within the auxiliary dataset, which are then used for memory replay.

###  3.3 Learning from Test Data

CoTTA. Continual Test-Time Adaptation Wang et al. (2022) proposes to adapt a
pre-trained deep classifier at inference time. Such adaptation may be needed
especially when the test data diverges from the original training source, and
the source data is no longer available (imagine a self-driving car driving
through changing weather conditions within a city). To tackle this novel
problem, the authors propose a regularization-based approach, where they
enforce consistency regularization across multiple augmentations of the same
input, as well as selective fine-tuning of a few network parameters.

NOTE. NOn-i.i.d. TEst-time adaptation Gong et al. (2022) extends COTTA to
realistic scenes, where the subsequent test examples carry high temporal
correlation, such as in the case of self-driving cars. The authors claim in
such cases, relying on batch normalization statistics in the form of pseudo-
labels may severely bias the learner towards the current batch. To that end,
they utilize instance normalization instead of batch normalization, leading to
far greater performance in comparison to CoTTA-like baselines.

##  4 Few-shot-Supervision for Incremental Learning

Algorithm Method Regularization Replay Semantic TOPIC Graph Anchor Loss ✗ ✗
CEC Graph ✗ ✗ ✗ IDL-VQ Clustering Center Loss ✓ ✗ SA-KD Clustering ✗ ✓ ✓ Sub-
Reg Clustering $\ell_{1}$ Loss ✓ ✓ FACT Clustering Augmentation ✗ ✗ FSLL
Architectural $\ell_{1}$ Loss ✗ ✗ C-FSCIL Architectural Orthogonal Loss ✓ ✗

Table 3: Incremental Learning with Few-shot-Supervision.

We summarize incremental learners with few-shot-supervision in Table 3. Few-
shot-supervised learners update a pre-trained base classifier with few-
examples from novel categories during incremental training. In this regard,
few-shot-supervised methods tackle two fundamental challenges simultaneously:

i) Overfitting to Novel Categories: Optimizing for the novel classifier
weights from scratch may over-fit on the few training exemplars. To that end,
the authors propose to associate already learned base classifier weights with
relevant novel classes during incremental training. We group the learners
according to their machinery to learn base-novel class association, as in
graph Tao et al. (2020); Zhang et al. (2021), clustering Chen and Lee (2020);
Cheraghian et al. (2021); Akyürek et al. (2022); Zhou et al. (2022) or the
architecture Mazumder et al. (2021); Hersche et al. (2022). Some techniques
also leverage semantic word embeddings to identify semantically relevant
categories for transfer Akyürek et al. (2022); Cheraghian et al. (2021).

ii) Forgetfulness of Base Categories: Learning novel categories may drift base
classifier weights, eventually degrading the accuracy of these classes. Two
effective techniques to mitigate performance degradation includes
regularization via metric learning objectives such as anchor loss Tao et al.
(2020), or simply memory replay of base class data.

###  4.1 Graph-Based Methods

TOPIC. TOpology-Preserving knowledge InCrementer Tao et al. (2020) is a graph-
based incremental few-shot learner. The authors treat each incremental class
as a novel node to be inserted to an already existing fully connected graph of
base classifier embeddings. They propagate information from base classifier
nodes towards novel classifier nodes with respect to their pairwise relation,
as measured by the similarity of classifier embeddings. To prevent the graph
from drifting and to preserve the initial graph topology, they leverage metric
learning in the form of anchor loss for regularization.

CEC. Continually Evolving Classifier Zhang et al. (2021) builds upon TOPIC,
and instead leverages Graph Attention Networks Veličković et al. (2017). In
this regard, the initial graph continually evolves with the incoming stream of
few-shot learning tasks. A novel incremental learning task attends on already
existing, semantically relevant categories to build the classifier weights.
The authors further generate pseudo-incremental learning tasks from the base
category data, which enhances novel class learning ability. The emphasis of
CEC is more on the novel class learning rather than maintaining base class
performance, which eventually exacerbates forgetfulness.

###  4.2 Clustering-Based Methods

IDL-VQ. Incremental Deep Learning Vector Quantization Chen and Lee (2020)
utilizes Gaussian-Mixtures to quantize visual features of learned categories
to reference vector centroids. Then, any incoming novel class is represented
by their soft similarity with existing reference vectors. The authors further
store 1-shot per-class for replay to reduce forgetfulness.

SA-KD. Semantic-Aware Knowledge Distillation Cheraghian et al. (2021) uses
K-means clustering instead to build reference class centroids, which are used
to represent a novel input. The representation is then projected into semantic
word embedding space to further promote base-novel class association.

SUB-REG. Subspace-Regularization Akyürek et al. (2022) utilizes QR-
decomposition to project base classifier embeddings to an orthogonal sub-
space. They then measure soft similarities between a novel class input and
sub-space vectors to represent the novel class weights. The authors
additionally apply $\ell_{1}$ loss to penalize abrupt weight changes within
the classifier.

FACT. Forward-Compatible Training Zhou et al. (2022) is the current state-of-
the-art in few-shot incremental learning. Authors first showcase that the
feature space of few-shot learners is fully occupied by pre-training (base)
classes, leaving no room for future (few-shot) classes, hence limiting
forward-compatibility. To that end, they propose to simultaneously assign an
input image to a separate cluster orthogonal to the base classes, effectively
reserving room for novel categories. Combined with mixup augmentations, FACT
improves performance on both base and novel classes.

###  4.3 Architectural Methods

FSLL. Few-shot Lifelong Learning Mazumder et al. (2021) selects a few weights
from the architecture to fine-tune per-incremental learning task. In doing so,
the authors simultaneously prevent overfitting by limiting the model capacity,
while preventing forgetting via minimizing interference across learning tasks.
Combined with $\ell_{1}$ loss, FSLL significantly improves over TOPIC across
several benchmarks.

C-FSCIL. Constrained-FSCIL Hersche et al. (2022) is a recent technique that
operates on classifier embeddings (prototypes) instead. The authors expand the
architecture with novel classifier embeddings with incoming stream of tasks.
In doing so, they impose quasi-orthogonality across learned embeddings,
effectively minimizing task interference, hence less forgetful.

##  5 Self-Supervision for Incremental Learning

We summarize incremental learners that leverage self-supervision in Table 4.
We identify three different trends in self-supervision for incremental
learning: i) Pre-training: To pre-train the backbone prior to incremental
learning, to warm-start with discriminative weights Gallardo et al. (2021).
This line of research builds upon the idea that self-supervised pre-training
reduces the need for label-supervision in subsequent (transfer) learning
tasks. ii) Auxiliary-training: To supplement standard label-supervision with
self-supervised objectives during training to obtain a more discriminative
feature space Zhu et al. (2021). This line of research builds upon the idea
that self-supervised learning tasks can provide additional supervisory signals
to the learner to prevent overfitting. iii) Main-training: To train solely
based on self-supervised learning objectives Purushwalkam et al. (2022);
Madaan et al. (2022); Fini et al. (2022); Gomez-Villa et al. (2022), which are
then evaluated by linear probing after training. These learners build upon the
idea that self-supervision can replace label-supervision to induce a
discriminative deep feature extractor.

Algorithm Setting Self-Supervision SSL-OCL Pre-training MOCO/SwAV PASS
Auxiliary-training SLA Buffer-SSL Main-training SimSiam LUMP Main-training
SimSiam/Barlow-Twins CaSSLe Main-training SimCLR/Barlow-Twins/etc. PFR Main-
training Barlow-Twins

Table 4: Incremental Learning with Self-Supervision.

Pre-training. Self-Supervised Learning for Online Continual Learning (SSL-OCL)
Gallardo et al. (2021) proposes to pre-train the backbone weights prior to
incremental training via self-supervision. This way, the authors aim to
leverage transfer-learning abilities brought by pre-training dataset.
Specifically, they evaluate MoCo-v2 Chen et al. (2020), Barlow-Twins Zbontar
et al. (2021) and SwAV Caron et al. (2020) for self-supervised pre-training.
The authors conclude that in contrast to standard label-supervised pre-
training, self-supervised pre-training is always superior, and SwAV
consistently outperforms the compared alternatives.

Auxiliary-training. Prototype-Augmented Self-Supervision (PASS) Zhu et al.
(2021) is a regularization-based incremental learning technique. The model
optimizes a single-prototype per-incremental class, where the prototype is
learned by standard label-supervision. To improve generalization and avoid
overfitting, the authors resort to augmentation. Specifically, they use Self-
Supervised Label Augmentation (SLA) Lee et al. (2020) to generate four-fold
rotations of the original input ($[0,90,180,270]$), which are then used as
additional pre-text tasks to differentiate for the model. The authors show
such method mitigates overfitting to the previously learned classes, leading
to superior results.

###  5.1 Main-training

Using self-supervision as the sole supervision signal for incremental training
is probably the most promising direction, as it requires no labels at training
times. For this, we identify two memory-based and two regularization-based
approaches.

Buffer-SSL. Buffer Self-Supervised Learning Purushwalkam et al. (2022) is a
memory-based approach that extends an off-the-shelf self-supervision
algorithm, SimSiam Chen and He (2021) for the case of incremental learning. In
doing so, the authors identify three main challenges. First, SimSiam has no
mechanism to retain information for previously seen data, leading to severe
forgetting. To mitigate this, the authors supplement SimSiam with a memory
buffer for replay. A trivial approach would be to store all examples within
the memory, however, results in a large memory size, and there is a high
redundancy across subsequent learning frames (such as in a video). To that
end, the authors only store cluster centroids, which reduces memory size and
increase memory variability. Using such buffer is shown to significantly
reduce forgetfulness of self-supervised representations.

LUMP. Lifelong Unsupervised Mixup Madaan et al. (2022) is a memory-based
approach utilizes mixup augmentation to mitigate forgetfulness. Specifically,
the authors learn to mixup the input instances with those from the past
learning tasks stored within the memory. Replaying such examples effectively
reduces forgetting and improves performance. The authors demonstrate their
algorithm with both SimSiam Chen and He (2021) and Barlow-Twins Zbontar et al.
(2021), where both algorithms lead to similar performance.

CaSSLe. CaSSLe is a regularization-based approach to self-supervised
incremental learning Fini et al. (2022). Since storing data from past learning
tasks is memory inefficient and may violate privacy, the authors instead learn
to distill self-supervised representations between the current and the past
model. The distillation is performed in a predictive manner, where the current
model’s features are projected onto the previous model’s feature space. The
authors apply their method on several different self-supervised learning
algorithms, observing similar performance. The authors propose to go beyond
class-incremental setting, and also evaluate data-incremental (the data is
partitioned randomly regardless of the classes) and domain-incremental (data
is partitioned by domain label, such as i.e., real $\rightarrow$ sketch
$\rightarrow$ clipart). Regardless of the setting, self-supervised
representations are found to be always more accurate and less forgetful than
label-supervised counterparts, which is promising for reducing the need for
label-supervision in incremental learning.

PFR. Projected Functional Regularization Gomez-Villa et al. (2022) is a
regularization-based technique, very similar to CaSSLe. The authors extend
Barlow-Twins with a distillation-based objective. Specifically, they learn to
project the current visual representation to the previous model
representation. The authors showcase that Barlow-Twins with PFR objective
exhibits lower forgetting and higher accuracy.

##  6 Conclusion and Future Directions

In this survey, we establish the lack of label-efficiency as a major
bottleneck in deploying realistic incremental learners. We unify three
different ways to improve label-efficiency, namely semi, few-shot and self-
supervised learning.

Although promising, these set of learners are not without limitations. To that
end, to inspire future research, in this section we first identify limiting
factors for label-efficiency and methodology. We then conclude with novel
directions to explore for label-efficient incremental learning.

###  6.1 Limitations and Alternative Methods

Semi-Supervision. Incremental learning via semi-supervision leverages
partially labeled data in the form of pseudo-supervision. The quality of
pseudo-supervision is partly determined by the amount of labeled data. To that
end, the models still require a significant amount of labels to be provided to
work well, limiting their label-efficiency.

Also, the level of pseudo-labelling noise may accumulate over time, especially
for long incremental learning sequences, limiting the model performance Wang
et al. (2022). A potential remedy is to move from pseudo-labels to pseudo-
gradients Luo et al. (2022), however the gradient estimation may also be
suboptimal by time. To that end, we see promise in updating normalization
parameters via input statistics without any form of pseudo-supervision Gong et
al. (2022).

Few-shot-Supervision. Few-shot-supervised learners make use of labels both
pre- and during incremental training. The label complexity of pre-training is
especially huge, since it requires many shots and many categories relevant to
novel few-shot classes. In data hungry fields like medical imaging or visual
anomaly detection, limiting their applicability.

Additionally, few-shot-supervised learners freeze the backbone during
incremental learning to prevent over-fitting. While working well, such
practice is unnatural, since humans can leverage few-shots for learning. One
potential remedy would be to rely on Sharpness-Aware Minimization, as few-shot
learning is shown to exhibit loss landscape with sharp and poor local minimum
Abbas et al. (2022).

Self-Supervision. The label complexity of self-supervised incremental learners
are on par with vanilla incremental learners, when self-supervision is used
for pre-training or for auxiliary-training. Using self-supervision as the sole
supervisory signal holds the key to completely omit the labels during
training. However, purely self-supervised incremental learning requires a
separate labeled linear-probing stage for evaluation purposes, limiting their
use cases.

Also, self-supervision for incremental learning solely relied on contrastive
learning-based approaches to extract supervisory signals Chen et al. (2020);
Zbontar et al. (2021). However, recent studies like MAE show promise of
reconstruction-based objectives over contrastive-based counterparts, which we
believe is worthy of exploration He et al. (2022).

###  6.2 Novel Problems

Inspired by our survey, here we recommend novel problems to investigate in
future research to build more realistic incremental learners.

Mixed-Supervised Learning. In this survey, we show that researchers follow
three main directions to reduce the need for supervision in a disjoint manner.
However, a combination of different forms of supervision is common in non-
incremental learning, such as the combination of self- and few-shot-
supervision Su et al. (2020), or semi and few-shot-supervision Ren et al.
(2018); Li et al. (2019).

To that end, our first recommendation is to explore such mixed supervision
settings for incremental learning.

Incremental Dense Learning. We limit our survey to the fundamental task of
image classification. However, for incremental dense prediction tasks such as
image segmentation Maracani et al. (2021), the demand for labels is explosive.
For image segmentation, annotators label each and every pixel in a high-
resolution scene over thousands of imagery.

We believe future research should incorporate different ways to reduce the
need for incremental supervision for dense prediction tasks, as the demand for
building commercial incremental learners grows. We note that the initial
effort(s) have been made for weakly-supervised incremental object
segmentation, however with limited performance over non-incremental
counterparts Cermelli et al. (2022).

Incremental Active Learning. In active learning, the learning agent selects
the most influential examples to be annotated by a human expert Munjal et al.
(2022). This significantly reduces the cost of labeling, as only a small
portion of the examples suffice to train a good model.

Surprisingly, active learning has found little to no application in
incremental learning agents. However, such method can help to select a small
set of exemplars to annotate for semi- or few-shot-supervised incremental
learning, either for pre-training or incremental-training.

Incremental Object Discovery. Humans exhibit an astounding ability to discover
never-before-seen objects, with little to no supervision. We are able to group
instances of novel objects with little effort. However, existing incremental
learners currently has no novel object discovery capacity Han et al. (2020),
as all the objects are at least partially labeled. This is unrealistic, since
the visual world always presents novel objects in daily life, partially thanks
to the advances in technology.

To that end, we believe that label-efficient learners should be capable of not
only learning from limited supervision, but also discover novel objects.

To conclude, we believe label-efficiency is a key factor to build autonomous,
human-like life-long learning agents. The potential in leveraging the ever-
growing, massive-scale unlabeled data is yet to come, and we hope our survey
provides a good introduction to this important field of newly emerging
research.

##  7 Acknowledgements

Mert Kilickaya’s research is fully funded by ASM Pacific Technology (ASMPT).

## References

  * Abbas et al. [2022] Momin Abbas, Quan Xiao, Lisha Chen, Pin-Yu Chen, and Tianyi Chen.  Sharp-maml: Sharpness-aware model-agnostic meta learning.  arXiv preprint, 2022.
  * Akyürek et al. [2022] Afra Feyza Akyürek, Ekin Akyürek, Derry Wijaya, and Jacob Andreas.  Subspace regularizers for few-shot class incremental learning.  arXiv preprint arXiv:2110.07059, 2022.
  * Baucum et al. [2017] Michael Baucum, Daniel Belotto, Sayre Jeannet, Eric Savage, Prannoy Mupparaju, and Carlos W Morato.  Semi-supervised deep continuous learning.  In ICDLT, 2017.
  * Brahma et al. [2021] Dhanajit Brahma, Vinay Kumar Verma, and Piyush Rai.  Hypernetworks for continual semi-supervised learning.  arXiv preprint, 2021.
  * Caron et al. [2020] Mathilde Caron, Ishan Misra, Julien Mairal, Priya Goyal, Piotr Bojanowski, and Armand Joulin.  Unsupervised learning of visual features by contrasting cluster assignments.  NeurIPS, 2020.
  * Cermelli et al. [2022] Fabio Cermelli, Dario Fontanel, Antonio Tavera, Marco Ciccone, and Barbara Caputo.  Incremental learning in semantic segmentation from image labels.  In CVPR, 2022.
  * Chen and He [2021] Xinlei Chen and Kaiming He.  Exploring simple siamese representation learning.  In CVPR, 2021.
  * Chen and Lee [2020] Kuilin Chen and Chi-Guhn Lee.  Incremental few-shot learning via vector quantization in deep embedded space.  In ICLR, 2020.
  * Chen et al. [2020] Xinlei Chen, Haoqi Fan, Ross Girshick, and Kaiming He.  Improved baselines with momentum contrastive learning.  arXiv preprint, 2020.
  * Chen et al. [2022] Tianlong Chen, Sijia Liu, Shiyu Chang, Lisa Amini, and Zhangyang Wang.  Queried unlabeled data improves and robustifies class-incremental learning.  TMLR, 2022.
  * Cheraghian et al. [2021] Ali Cheraghian, Shafin Rahman, Pengfei Fang, Soumava Kumar Roy, Lars Petersson, and Mehrtash Harandi.  Semantic-aware knowledge distillation for few-shot class-incremental learning.  In CVPR, 2021.
  * De Lange et al. [2021] Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Aleš Leonardis, Gregory Slabaugh, and Tinne Tuytelaars.  A continual learning survey: Defying forgetting in classification tasks.  TPAMI, 2021.
  * Dosovitskiy et al. [2020] Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, et al.  An image is worth 16x16 words: Transformers for image recognition at scale.  ICLR, 2020.
  * Fini et al. [2022] Enrico Fini, Victor G Turrisi da Costa, Xavier Alameda-Pineda, Elisa Ricci, Karteek Alahari, and Julien Mairal.  Self-supervised models are continual learners.  In CVPR, 2022.
  * French [1999] Robert M French.  Catastrophic forgetting in connectionist networks.  CS, 1999.
  * Gallardo et al. [2021] Jhair Gallardo, Tyler L Hayes, and Christopher Kanan.  Self-supervised training enhances online continual learning.  arXiv preprint, 2021.
  * Gomez-Villa et al. [2022] Alex Gomez-Villa, Bartlomiej Twardowski, Lu Yu, Andrew D Bagdanov, and Joost van de Weijer.  Continually learning self-supervised representations with projected functional regularization.  In CVPR, 2022.
  * Gong et al. [2022] Taesik Gong, Jongheon Jeong, Taewon Kim, Yewon Kim, Jinwoo Shin, and Sung-Ju Lee.  Note: Robust continual test-time adaptation against temporal correlation.  In NeurIPS, 2022.
  * Ha et al. [2016] David Ha, Andrew Dai, and Quoc V Le.  Hypernetworks.  arXiv preprint, 2016.
  * Han et al. [2020] Kai Han, Sylvestre-Alvise Rebuffi, Sebastien Ehrhardt, Andrea Vedaldi, and Andrew Zisserman.  Automatically discovering and learning new visual categories with ranking statistics.  arXiv preprint, 2020.
  * He et al. [2016] Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.  Deep residual learning for image recognition.  In CVPR, 2016.
  * He et al. [2022] Kaiming He, Xinlei Chen, Saining Xie, Yanghao Li, Piotr Dollár, and Ross Girshick.  Masked autoencoders are scalable vision learners.  In CVPR, 2022.
  * Hersche et al. [2022] Michael Hersche, Geethan Karunaratne, Giovanni Cherubini, Luca Benini, Abu Sebastian, and Abbas Rahimi.  Constrained few-shot class-incremental learning.  In CVPR, 2022.
  * Kingma and Welling [2013] Diederik P Kingma and Max Welling.  Auto-encoding variational bayes.  arXiv preprint, 2013.
  * L2ID [2022] L2ID.  Learning from limited and imperfect data (l2id).  https://l2id.github.io/l2id2022/index.html, 2022.
  * Lee et al. [2020] Hankook Lee, Sung Ju Hwang, and Jinwoo Shin.  Self-supervised label augmentation via input transformations.  In ICML, 2020.
  * Li and Hoiem [2017] Zhizhong Li and Derek Hoiem.  Learning without forgetting.  TPAMI, 2017.
  * Li et al. [2019] Xinzhe Li, Qianru Sun, Yaoyao Liu, Qin Zhou, Shibao Zheng, Tat-Seng Chua, and Bernt Schiele.  Learning to self-train for semi-supervised few-shot classification.  NeurIPS, 2019.
  * Luo et al. [2022] Yan Luo, Yongkang Wong, Mohan Kankanhalli, and Qi Zhao.  Learning to predict gradients for semi-supervised continual learning.  arXiv preprint, 2022.
  * Madaan et al. [2022] Divyam Madaan, Jaehong Yoon, Yuanchun Li, Yunxin Liu, and Sung Ju Hwang.  Representational continuity for unsupervised continual learning.  In ICLR, 2022.
  * Maracani et al. [2021] Andrea Maracani, Umberto Michieli, Marco Toldo, and Pietro Zanuttigh.  Recall: Replay-based continual learning in semantic segmentation.  In ICCV, 2021.
  * Masana et al. [2020] Marc Masana, Xialei Liu, Bartlomiej Twardowski, Mikel Menta, Andrew D Bagdanov, and Joost van de Weijer.  Class-incremental learning: survey and performance evaluation on image classification.  arXiv preprint, 2020.
  * Mazumder et al. [2021] Pratik Mazumder, Pravendra Singh, and Piyush Rai.  Few-shot lifelong learning.  In AAAI, 2021.
  * Munjal et al. [2022] Prateek Munjal, Nasir Hayat, Munawar Hayat, Jamshid Sourati, and Shadab Khan.  Towards robust and reproducible active learning using neural networks.  In CVPR, 2022.
  * Purushwalkam et al. [2022] Senthil Purushwalkam, Pedro Morgado, and Abhinav Gupta.  The challenges of continuous self-supervised learning.  arXiv preprint, 2022.
  * Radford et al. [2021] Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, et al.  Learning transferable visual models from natural language supervision.  In ICML, 2021.
  * Ren et al. [2018] Mengye Ren, Eleni Triantafillou, Sachin Ravi, Jake Snell, Kevin Swersky, Joshua B Tenenbaum, Hugo Larochelle, and Richard S Zemel.  Meta-learning for semi-supervised few-shot classification.  arXiv preprint, 2018.
  * Rolnick et al. [2019] David Rolnick, Arun Ahuja, Jonathan Schwarz, Timothy Lillicrap, and Gregory Wayne.  Experience replay for continual learning.  NeurIPS, 2019.
  * Sahito et al. [2022] Attaullah Sahito, Eibe Frank, and Bernhard Pfahringer.  Better self-training for image classification through self-supervision.  In AI, 2022.
  * Shin et al. [2017] Hanul Shin, Jung Kwon Lee, Jaehong Kim, and Jiwon Kim.  Continual learning with deep generative replay.  NeurIPS, 2017.
  * Smith et al. [2021] James Smith, Jonathan Balloch, Yen-Chang Hsu, and Zsolt Kira.  Memory-efficient semi-supervised continual learning: The world is its own replay buffer.  In IJCNN, 2021.
  * Snell et al. [2017] Jake Snell, Kevin Swersky, and Richard Zemel.  Prototypical networks for few-shot learning.  NeurIPS, 2017.
  * Sohn et al. [2020] Kihyuk Sohn, David Berthelot, Nicholas Carlini, Zizhao Zhang, Han Zhang, Colin A Raffel, Ekin Dogus Cubuk, Alexey Kurakin, and Chun-Liang Li.  Fixmatch: Simplifying semi-supervised learning with consistency and confidence.  NeurIPS, 2020.
  * Su et al. [2020] Jong-Chyi Su, Subhransu Maji, and Bharath Hariharan.  When does self-supervision improve few-shot learning?  In ECCV, 2020.
  * Tao et al. [2020] Xiaoyu Tao, Xiaopeng Hong, Xinyuan Chang, Songlin Dong, Xing Wei, and Yihong Gong.  Few-shot class-incremental learning.  In CVPR, 2020.
  * Tian et al. [2020] Yonglong Tian, Yue Wang, Dilip Krishnan, Joshua B Tenenbaum, and Phillip Isola.  Rethinking few-shot image classification: a good embedding is all you need?  In ECCV, 2020.
  * Van de Ven and Tolias [2019] Gido M Van de Ven and Andreas S Tolias.  Three scenarios for continual learning.  arXiv preprint, 2019.
  * Veličković et al. [2017] Petar Veličković, Guillem Cucurull, Arantxa Casanova, Adriana Romero, Pietro Lio, and Yoshua Bengio.  Graph attention networks.  arXiv preprint, 2017.
  * Vinyals et al. [2016] Oriol Vinyals, Charles Blundell, Timothy Lillicrap, Daan Wierstra, et al.  Matching networks for one shot learning.  NeurIPS, 2016.
  * Wang et al. [2021] Liyuan Wang, Kuo Yang, Chongxuan Li, Lanqing Hong, Zhenguo Li, and Jun Zhu.  Ordisco: Effective and efficient usage of incremental unlabeled data for semi-supervised continual learning.  In CVPR, 2021.
  * Wang et al. [2022] Qin Wang, Olga Fink, Luc Van Gool, and Dengxin Dai.  Continual test-time domain adaptation.  In CVPR, 2022.
  * Wang et al. [2023] Liyuan Wang, Zhang Xingxing, Su Hang, and Zhu Jun.  A comprehensive survey of continual learning: Theory, method and application.  arXiv preprint, 2023.
  * Yang et al. [2022] Xiangli Yang, Zixing Song, Irwin King, and Zenglin Xu.  A survey on deep semi-supervised learning.  KDE, 2022.
  * Zbontar et al. [2021] Jure Zbontar, Li Jing, Ishan Misra, Yann LeCun, and Stéphane Deny.  Barlow twins: Self-supervised learning via redundancy reduction.  In ICML, 2021.
  * Zenke et al. [2017] Friedemann Zenke, Ben Poole, and Surya Ganguli.  Continual learning through synaptic intelligence.  In ICML, 2017.
  * Zhang et al. [2020] Junting Zhang, Jie Zhang, Shalini Ghosh, Dawei Li, Serafettin Tasci, Larry Heck, Heming Zhang, and C-C Jay Kuo.  Class-incremental learning via deep model consolidation.  In WACV, 2020.
  * Zhang et al. [2021] Chi Zhang, Nan Song, Guosheng Lin, Yun Zheng, Pan Pan, and Yinghui Xu.  Few-shot incremental learning with continually evolved classifiers.  In CVPR, 2021.
  * Zhou et al. [2022] Da-Wei Zhou, Fu-Yun Wang, Han-Jia Ye, Liang Ma, Shiliang Pu, and De-Chuan Zhan.  Forward compatible few-shot class-incremental learning.  In CVPR, 2022.
  * Zhou et al. [2023] Da-Wei Zhou, Qi-Wei Wang, Zhi-Hong Qi, Han-Jia Ye, De-Chuan Zhan, and Ziwei Liu.  Deep class-incremental learning: A survey.  arXiv preprint, 2023.
  * Zhu et al. [2021] Fei Zhu, Xu-Yao Zhang, Chuang Wang, Fei Yin, and Cheng-Lin Liu.  Prototype augmentation and self-supervision for incremental learning.  In CVPR, 2021.
  * Zoph et al. [2020] Barret Zoph, Golnaz Ghiasi, Tsung-Yi Lin, Yin Cui, Hanxiao Liu, Ekin Dogus Cubuk, and Quoc Le.  Rethinking pre-training and self-training.  NeurIPS, 2020.