11institutetext:  Institut d’Astrophysique de Paris, CNRS-Sorbonne Université,
98 bis boulevard Arago, F-75014 Paris, France. 22institutetext: Department of
Physics and Astronomy & Pittsburgh Particle Physics, Astrophysics, and
Cosmology Center (PITT PACC),
University of Pittsburgh, 3941 O’Hara Street, Pittsburgh, PA 15260, USA.

# Modeling the signatures of interaction in Type II supernovae:
UV emission, high-velocity features, broad-boxy profiles

Luc Dessart Modeling the signatures of interaction in Type II supernovae:  UV
emission, high-velocity features, broad-boxy profilesModeling the signatures
of interaction in Type II supernovae:  UV emission, high-velocity features,
broad-boxy profiles D. John Hillier Modeling the signatures of interaction in
Type II supernovae:  UV emission, high-velocity features, broad-boxy
profilesModeling the signatures of interaction in Type II supernovae:  UV
emission, high-velocity features, broad-boxy profiles

Because mass loss is a fundamental phenomenon in massive stars, interaction
with circumstellar material (CSM) should be universal in core-collapse
supernovae (SNe). Leaving aside the extreme CSM density, extent, or mass
typically encountered in Type IIn SNe, we investigate the diverse long-term
radiative signatures of interaction between a Type II SN ejecta and CSM
corresponding to mass loss rates up to 10-3 $M_{\odot}$ yr-1. Because these
CSM are relatively tenuous and optically-thin to electron-scattering beyond a
few stellar radii, radiation hydrodynamics is not essential and one may treat
the interaction directly as an additional power source in the non-local
thermodynamic equilibrium radiative transfer problem. The CSM accumulated
since shock breakout forms a dense shell in the outer ejecta and leads to
high-velocity absorption features in spectral lines, even for negligible shock
power. Besides Balmer lines, such features may appear in Na i D, He i lines
etc. A stronger interaction strengthens the continuum flux (preferentially in
the UV), quenches the absorption of P-Cygni profiles, boosts the Mg ii
$\lambda\lambda$ $2795,\,2802$ doublet, and fosters the production of a broad
boxy H$\alpha$ emission component. The rise in ionization in the outer ejecta
may quench some lines (e.g., the Ca ii near-infrared triplet). The interaction
power emerges preferentially in the UV, in particular at later times, shifting
the optical color to the blue, but increasing modestly the optical luminosity.
Strong thermalization and clumping seem to be required to make an interaction
superluminous in the optical. The UV range contains essential signatures that
provide critical constraints to infer the mass loss history and inner workings
of core-collapse SN progenitors at death.

###### Key Words.:

line: formation – radiative transfer – supernovae: general

##  1 Introduction

Mass loss is a fundamental phenomenon affecting the evolution of massive
stars. Steady-state mass loss in the form of a wind can significantly reduce
the mass of a star and affect many of its properties throughout its life
(Maeder & Meynet, 1987; Langer et al., 1994), as well as the mass of the
metal-rich core, the final yields, and the compactness at collapse (Woosley et
al., 2002; Renzo et al., 2017). Steady or dynamical mass transfer through
Roche-lobe overflow in interacting binaries are typically short lived but
manyfold stronger than wind mass loss and can thus lead to a dramatic change
in the evolution of a massive star (Podsiadlowski et al., 1992; Wellstein &
Langer, 1999; Eldridge et al., 2008; Langer, 2012). Interacting binaries are
likely central for the production of stripped-envelope supernovae (SNe; Yoon
et al., 2010). Hence, stellar wind mass loss and mass transfer in interacting
binaries play a central role in determining the gross properties of the SN
progenitor and the SN type.

However, the radiative properties of SNe depend also on the complex internal
structure of the star at death and the structure of its close environment. The
final structure of the star affects the shock propagation through the envelope
prior to shock breakout. When the close environment of the star is not a
vacuum, as generally assumed for simplicity, an interaction takes place with
the circumstellar material (CSM). While the existence of CSM is not surprising
for massive stars, a number of observed SNe suggest that this CSM may be
exceptionally dense, in the sense that it would correspond to wind mass loss
rates manyfold greater than standard steady-state mass loss rates typically
inferred in massive stars (see, for example, de Jager et al. 1988). This CSM
may also extend out to large distances above the star surface, from several
1014 cm (SN 2013fs; Dessart et al., 2017; Moriya et al., 2017; Morozova et
al., 2017; Yaron et al., 2017), to 1015 cm (SN 2020tlf; Jacobson-Galán et al.,
2022), to several 1015 cm (SN 1998S; Leonard et al., 2000; Fassia et al.,
2001; Chugai, 2001; Dessart et al., 2016), and up to 1016 cm (SN 2010jl; Zhang
et al., 2012; Fransson et al., 2014; Dessart et al., 2015). While large
relative to the progenitor radius, these distances correspond to mass loss
episodes occurring on year time scales before core collapse and thus may point
to dynamical phenomena tied to the last stages of massive star evolution
(Quataert & Shiode, 2012; Fuller, 2017).

A critical spectroscopic signature of interaction in a SN is the presence of
narrow and symmetric emission line profiles instead of the expected Doppler-
broadened P-Cygni profiles (Schlegel, 1990; Stathakis & Sadler, 1991; Chugai,
2001; Dessart et al., 2009). This is most obvious in H i Balmer lines of Type
II SNe (e.g., SN 1998S), but also seen in lines of He i, He ii, or more highly
ionized ions of C, N, or O (e.g., SN 2013fs; Yaron et al. 2017). Such lines
imply a reprocessing of the radiation from the underlying SN by the unshocked,
quasistatic, optically-thick CSM. This CSM releases the SN radiation on a
diffusion timescale, broadening significantly the shock breakout signal. In
the interaction between ejecta and CSM, the ejecta deceleration implies a
transformation of kinetic energy into radiative energy, which boosts the SN
luminosity. The CSM also induces a shift of the spectral energy distribution
to longer wavelength because the radiation is trapped within a larger
optically-thick volume. Together, these effects tend to boost the optical
luminosity of the SN.

While many studies have been devoted to superluminous interacting SNe, the
impact of mass loss is likely much more universal in core-collapse SNe. For
wind mass loss rates stronger than standard but too low to produce an
optically-thick CSM, the SN spectra cannot exhibit the narrow spectral
signatures that would flag the event as a Type IIn or Type Ibn. However, the
shock power may nonetheless be substantial and even supersede the ‘default’ SN
luminosity (i.e., in the absence of interaction). Overluminous SNe II
exhibiting non standard broad line profiles (e.g., H$\alpha$) are indeed
observed (Pessi et al., in prep). For a SN shock ramming at velocity $V_{\rm
sh}$ into a steady-state wind with mass loss rate $\dot{M}$ and velocity
$V_{\infty}$, the instantaneous power released by the interaction is $L_{\rm
sh}=\dot{M}V_{\rm sh}^{3}/2V_{\infty}=3.15\times
10^{40}\,\,\,\dot{M}_{-5}V_{\rm sh,4}^{3}/V_{\infty,2}\,\,\,{\rm erg}\,{\rm
s}^{-1}\,,$ where $\dot{M}_{-5}\equiv\dot{M}/10^{-5}M_{\odot}$ yr-1, $V_{\rm
sh,4}\equiv V_{\rm sh}$/10000 km s-1 and $V_{\infty,2}\equiv V_{\infty}/100$
km s-1. If in addition $V_{\rm sh}$ is fixed, the shock power is constant in
time. The representative luminosity of a Type II SN during the photospheric
phase, which is a few times 1042 erg s-1 (Bersten & Hamuy, 2009), can be
rivaled by the power from the ejecta interaction with a wind mass loss rate of
10-3 $M_{\odot}$ yr-1. This mass loss rate is still a factor of a 100 weaker
than inferred for a superluminous Type IIn SN like 2010jl (Fransson et al.,
2014; Dessart et al., 2015). Furthermore, if the wind density remains
sufficiently high at large distances, the shock power should eventually ‘win’
over the inner ejecta luminosity (powered by radioactive decay). For example,
for $\dot{M}_{-5}=$ 3.2, $V_{\rm sh,4}=$ 1.0, and $V_{\infty,2}=$ 1.0, the
interaction power is 1041 erg s-1 and comparable to the power at 300 d from
the decay of 0.1 $M_{\odot}$ of ${}^{56}\rm{N}$i. The contribution from
interaction power to the SN radiation at some stage in the evolution of a
core-collapse SN is thus likely.

For CSM configurations corresponding to wind mass loss rates of 0.1
$M_{\odot}$ yr-1 and extending over large scales, a multi-group radiation
hydrodynamics treatment is essential. This approach captures the dynamics of
the interaction, the deceleration of the ejecta, and the continuous pile-up of
CSM into a dense shell. Furthermore, as the CSM becomes optically thick
following the radiative precursor, the radiative transfer problem is time
dependent, and so are the gas properties. As long as the shock is embedded in
this optically-thick CSM, photons emerge from the unshocked CSM. This
situation changes as the shock progresses outwards and escaping photons
increasingly originate from the shocked CSM and ejecta. Such configurations
correspond to superluminous SNe (Moriya et al., 2011; Dessart et al., 2015).

In this letter, we leave aside these extreme CSM configurations. Instead, we
consider the interaction of a standard Type II SN ejecta with a CSM
corresponding to a wind mass loss rate that is always too weak to make the CSM
optically thick to electron scattering (except perhaps in the immediate
vicinity of the progenitor surface, which is quickly swept up by the ejecta on
a day timescale). In this case, there is no need for radiation hydrodynamics
since 1) the shock power is known analytically and given above by $L_{\rm
sh}$; 2) the CSM is transparent so its contribution in emission and absorption
may be neglected; 3) and the CSM is too tenuous to appreciably decelerate the
ejecta. Hence, in contrast to the approach used in our previous studies
(Dessart et al., 2015, 2016, 2017), we simulate the interaction between ejecta
and CSM directly within CMFGEN (Hillier & Dessart, 2012). While this ignores
the dynamics of the problem, this allows for a much better treatment of the
radiative transfer and the effects associated with departures from local
thermodynamic equilibrium (LTE). It also allows for the long-term modeling of
an interaction over weeks, months, and even years (see Sect. 4). In the next
section, we present our numerical approach, including the treatment of the
interaction power in CMFGEN and the set of simulations we cover in this work.
We then present our results in Sect. 3. We conclude in Sect. 4.

Figure 1: Top: Ejecta density structure versus velocity for the original Type
II SN model at 10 d after explosion (black). For this work, we introduce a
dense shell at 11700 km s-1 (blue), which we clump (maximum compression factor
of 100; red). Bottom: Corresponding mass fraction versus velocity for H, He,
O, and Si, as well as for the adopted (normalized) shock-power profile
associated with the interaction between CSM and ejecta.

##  2 Numerical setup

All models presented in this work start with the same default Type II SN
ejecta at an age of 10 d. It corresponds to a star of 15 $M_{\odot}$
initially, evolved at solar metallicity, and that exploded to produce an
ejecta of 10.81 $M_{\odot}$ with a kinetic energy of $1.3\times 10^{51}$ erg
and 0.03 $M_{\odot}$ of ${}^{56}\rm{N}$i (for details on the method, see
Hillier & Dessart 2019). To mimic the presence of a cocoon of material in the
direct vicinity of the progenitor red-supergiant (RSG) star, something that is
likely typical in SNe II-P with super-wind phases prior to explosion as
inferred for SN 2013fs (Yaron et al., 2017), we introduce in the initial
density structure a dense shell of 0.1 $M_{\odot}$ at a velocity of 11700 km
s-1 (compatible with model predictions; Dessart et al. 2017). Because of
multidimensional instabilities, this dense shell should break-up in 3D and
exhibit strong clumping. We thus spread the dense shell over a characteristic
scale set to 10 % of the local radius, and introduce clumping to reach a
maximum compression factor of 100 (volume filling factor of 1 %). Numerically,
this allows us to resolve this dense shell in 1D while preserving a high
density for the clumped material in the dense shell (Fig. 1).

All models are evolved assuming a fixed shock velocity ($V_{\rm sh,4}=$ 1.17)
and a steady progenitor wind ($\dot{M}_{-5}$ from zero to 100 and
$V_{\infty,2}=$ 0.5), corresponding to a fixed interaction power from zero to
1043 erg s-1. The accumulated swept-up mass by the ejecta after one year is
$0.001\dot{M}_{-5}V_{\rm sh,4}/V_{\infty,2}M_{\odot}$ (i.e., about 0.1
$M_{\odot}$ for $\dot{M}_{-5}=$ 100). Being relatively small, we decide to
neglect it and go further and neglect any dynamical influence of the CSM on
the initial ejecta density structure. We also neglect the absorption and
emission from such a low-density CSM. We reset its properties so that its
density is negligible and its velocity is homologous. With these adjustments
we can model the SN ejecta and the interaction using the same approach as we
use to model non-interacting SN ejecta in CMFGEN.

Figure 2: Photometric properties for our set of models. From top to bottom, we
show the evolution of the bolometric luminosity, $M_{V}$, $M_{U}$, $M_{UVW2}$,
and the $U-V$ color.

How one should deposit the interaction power is a delicate matter. In
radiation-hydrodynamics simulations of interactions, power is injected in the
forward and in the reverse shocks that bound the dense shell. Because of the
break-up of the dense shell in 3D, which leads to clumping and turbulence etc,
the deposition of shock power is highly complex. Here, we focus on that part
of the shock power that is absorbed in the ejecta and ignore any high-energy
radiation that escapes. Hence the power that we deposit underestimates the
true power of the shock (particularly for the forward shock, but much less so
for the reverse shock). As a first step, we deposit the shock power in the
dense shell and we model how it thermalizes in that region. In practice, we
adopt a Gaussian profile for the deposition of shock power, with a centroid at
the center of the dense shell at 11700 km s-1 with a characteristic width of
75 km s-1 (dashed-curve in the bottom panel of Fig. 1; the clumping profile
takes the same form). We treat equivalently shock power and decay power in
CMFGEN, that is, the shock power is injected as high-energy radiation and
degraded in the same way as $\gamma$-rays from radioactive decay. Clumping is
treated as in Dessart et al. (2018) and we therefore neglect porosity.
Nonthermal effects are thus computed consistently by the nonLTE solver.
Because of the rapid change in density, ionization, and temperature within the
dense shell, we enforce a high resolution in this region. Regridding is also
performed in individual models to resolve accurately the recombination fronts.

In this work, we present time sequences for a SN II ejecta under the influence
of eight different interaction powers (but constant in time for any given
model sequence). These seven distinct sequences take shock powers of 1040,
1041, 5 $\times$ 1041, 1042, 2.5 $\times$ 1042, 5 $\times$ 1042, and 1043 erg
s-1 (the corresponding model names are Pwr1e40 etc). One sequence without
interaction power serves as a reference (model NoPwr). All models have the
same density structure at any given SN age and were evolved until 150–300 d –
the models change little once nebular. The simulations are time dependent and
contain all the assets of the standard simulations performed in the past with
CMFGEN (for similar but noninteracting SNe II models, see, for example,
Hillier & Dessart 2019). Different ejecta models, levels of clumping (which is
known to impact the temperature and the gas ionization), shock velocities,
progenitor mass loss history, or composition will be considered in future
studies.

##  3 Results

The top panel of Fig. 2 shows the bolometric light curves for our set of
models. For increasing interaction power, the luminosity from the underlying
ejecta (stemming from energy deposited by the SN shock prior to shock breakout
as well as from the continuous power supply from radioactive decay) is
progressively dominated by the interaction power and eventually swamped for
$L_{\rm sh}\mathrel{\hbox{\hbox to0.0pt{\hbox{\lower
4.0pt\hbox{$\sim$}}\hss}\hbox{$>$}}}5\times 10^{42}$ erg s-1 during the
photospheric phase. Although the models span three orders of magnitude in
interaction power, there is only a factor of 5.6 difference in bolometric
luminosity at 40 d after explosion. This arises from the large intrinsic
luminosity of noninteracting SN II ejecta during the photospheric phase, which
stems from the release of stored radiation energy. Because the SN ejecta is
typically ten times fainter at nebular times, an interaction power of
$\mathrel{\hbox{\hbox to0.0pt{\hbox{\lower
4.0pt\hbox{$\sim$}}\hss}\hbox{$>$}}}$ 1041 erg s-1 is more easily discerned
after about 120 d.

Figure 3: Comparison of the observer’s frame luminosity in the UV and optical
ranges for our set of models with an interaction power covering from zero to
1043 erg s-1 and at a time of 41.76 d. The inset zooms on the H$\alpha$
region. The thick vertical line represents the rest wavelength of H$\alpha$,
and the thin vertical lines indicate the wavelength at $\pm$ 11700 km s-1 away
from that rest wavelength.

Figure 4: Left: Time evolution of the spectral properties in the UV and in the
optical for the SN II model with an interaction power of 1041 erg s-1 (black)
and the model without interaction power (blue). Both models have, however, the
same ejecta density structure (short-dashed curve in Fig. 1). For each epoch,
the spectra are divided by the maximum flux of the brighter model and then
stacked for visibility. Right: Same as left but now showing the spectra versus
Doppler velocity and centered on the rest wavelength of H$\alpha$. The red
curve shows the flux difference between the two models.

The lower panels of Fig. 2 give some photometric properties. It is essentially
impossible from the $V$-band brightness (0.5 mag offset between models at 42
d) to guess the presence of an interaction in this parameter space, but the
interaction is strikingly apparent in the $U$ band, and more so as we progress
to the UV (with offsets of several magnitudes). The sensitivity in $U$ makes
the $U-V$ color a good probe of these interaction models.

Figure 3 compares the synthetic spectra from 1000 to 9500 Å at 41.76 d after
explosion (see Fig. 6 for the counterpart at 108.3 d). At that time, the
maximum offset is only 50 % in optical luminosity but the disparity between
models is huge in the UV. Models without interaction power are UV faint as a
result of a cold photosphere essentially at the H recombination temperature
and affected by strong metal line blanketing. With increasing interaction
power, the UV flux progressively rises and the Mg ii $\lambda\lambda$
$2795,\,2802$ doublet emerges and strengthens. For the models with the highest
powers, other emission lines appear in the far-UV, namely multiplets of Si iii
and Fe iii around 1110 Å, Ly$\alpha$, or C ii at 1335 Å. Hence, interaction
power is an efficient process to produce superluminous SNe but such transients
are optically superluminous only if the interaction power is strongly degraded
into low energy photons. In this context, observations in both UV and optical
are essential to establish the bolometric luminosity of a SN and determine the
presence of an interaction.

In contrast to the diverse UV landscape sketched by the models, the
interaction yields subtle, though compelling, spectral features in the
optical. Numerous line profiles including H$\alpha$, H$\beta$, Na i D, or the
Ca ii near-infrared triplet, exhibit a strong and narrow absorption at the
Doppler velocity corresponding to the dense shell (i.e., a ‘high velocity’ or
HV feature in SN jargon). This HV notch is present even without interaction
power. However, as interaction power is increased, this absorption becomes
more obvious (see inset for H$\alpha$ in Fig. 3). For a given model sequence,
these HV features emerge at the onset of the recombination epoch (around 30–50
d) and strengthen thereafter (Fig. 4). Our results confirm the findings of
Chugai et al. (2007) based on observations of SNe 1999em and 2004dj – larger
samples of Type II SN spectra reveal that such HV features are not rare
although they tend to be more frequently observed in H$\alpha$ and H$\beta$
(Gutiérrez et al., 2017).

The HV features arise because the thin shell at large velocity is much larger
than the continuum emitting region. These notches are narrow because the shell
is narrow and because projection effects are small. Such a shell may have a
complex formation history. It first forms at the time of shock breakout, but
this alone produces a negligible shell mass. More mass can be accumulated if
some dense CSM is present close to the star, as in SN 2013fs (Yaron et al.,
2017), and in addition, or alternatively, if some lower density CSM is present
over a large volume, as proposed by Chugai et al. (2007). We do find in our
simulations that even without interaction power, a HV feature may be present
(model NoPwr shows a HV feature in H$\alpha$, H$\beta$, Na i D, and in the Ca
ii near infrared triplet at $\mathrel{\hbox{\hbox to0.0pt{\hbox{\lower
4.0pt\hbox{$\sim$}}\hss}\hbox{$>$}}}$ 50 d).

As interaction power is increased (mass loss rates greater than 10-5
$M_{\odot}$ yr-1, thus barely above the typical RSG mass loss rates), a broad
H$\alpha$ emission component appears (right panel in Fig. 4). Because of
optical depth effects, it is strongly blueshifted early in the photospheric
phase but eventually turns into a boxy emission bounded within $\pm$ 11700 km
s-1 (i.e., corresponding to the dense shell velocity). A boxy profile is
simply a consequence of having a narrow emitting shell and a Hubble flow. In
model Pwr1e41 at 300 d, this extended emission overlaps in the blue with [O i]
$\lambda\lambda$ $6300,\,6364$, making the red component of the O i doublet
appear stronger than the blue component (right panel of Fig. 4). This boxy
emission extends to the red of the H$\alpha$ rest wavelength and makes its
origin clear : It only occurs in the presence of interaction (model NoPwr does
not have it). Such a profile has been seen at nebular times in SN 2017ivv
(Gutiérrez et al., 2020).

In the models with the highest power, the model spectra in the UV and in the
optical strongly depart from the NoPwr model (Fig. 3). Besides the huge UV
flux, a blue continuum flux weakens the line absorptions in the optical. Some
lines are also weakened because the interaction power enhances the ionization
and the temperature of the ejecta layers (an ionization wave penetrates the
ejecta inwards from the CDS), overionizing certain species (e.g., at 41.76 d,
Ca is everywhere Ca2+ in model Pwr1e43 but partially Ca+ in model Pwr1e40).
The broad boxy emission component is very strong in model Pwr1e43 during the
photospheric phase, with a slight excess near line center reminiscent of the
H$\alpha$ emission seen in model NoPwr. An H$\alpha$ profile with a weak or no
absorption component testifies for the presence of an interaction (Hillier &
Dessart, 2019) and the connexion to brighter and faster declining Type II SN
light curves (Gutiérrez et al., 2014). At nebular times, the outer dense shell
in the models with the highest power is hot and highly ionized, which quenches
the broad boxy H$\alpha$ emission component (Figs. 10–12).

In nearly all of our simulations, the interaction power eventually dominates
over decay power as we progress in the nebular phase. The trends seen in a
given model with increasing SN age are qualitatively similar to those seen at
a given time with increasing interaction power (Fig. 3). In model Pwr1e41 at
300 d, the offset in flux is primarily confined to the UV (67 % of the total
flux emerges in the UV) where strong lines have formed (Ly$\alpha$ and Mg ii
$\lambda\lambda$ $2795,\,2802$). This strong effect results from our ad-hoc
prescription of a fixed interaction power at all times. However, it is clear
that the UV range is critical to evaluate the presence of an interaction since
the only obvious optical signature is limited to the broad boxy emission
component of H$\alpha$, which typically contains of order 1 % of the total
interaction power injected in the model.

##  4 Conclusions

In this letter, we have presented a new method for modeling interacting SNe
within the time-dependent nonLTE radiative transfer code CMFGEN. The method is
designed to cover conditions in which the CSM is optically thin in electron
scattering, to complement the alternate configurations with optically-thick
CSM handled with HERACLES and the non-monotonic solver in CMFGEN (Dessart et
al., 2015). The preliminary simulations presented in this letter were focused
on a Type II SN ejecta, modified to exhibit a dense shell at high velocity to
mimic the swept-up mass from the interaction with CSM, and influenced by
various shock powers. Although it depends on the level of clumping in the
dense shell and the efficiency of thermalization therein, the bulk of the
interaction power tends to emerge in the UV, making this spectral range
essential in this context and emphasizing the need for future observations of
transients in the UV (Kulkarni et al., 2021). In the optical, the presence of
a dense shell can give rise to HV features in numerous strong lines
(H$\alpha$, Na i D etc), even in the absence of interaction power. However, as
the interaction is increased, the HV absorptions tend to strengthen while a
broad boxy H$\alpha$ emission component develops.

The method can be applied to any type of SN ejecta and may help quantify the
level at which an interaction with a pre-SN wind can contaminate the SN
radiation. Because it is well suited at nebular times, it will allow
investigations on the long-term evolution of SN ejecta such as SN 1993J (an
exploratory model is shown in Fig. 5), and help constrain the mass loss
history of SN progenitors in the years, decades or centuries prior to
explosion.

Figure 5: Example calculation for an interaction at a very late time in the
nebular phase (black). The ejecta is limited to a 1 $M_{\odot}$ dense shell
moving at 10000 km s-1 and powered at a rate of $3\times 10^{41}$ erg s-1. A
uniform and strong clumping corresponding to a volume filling factor of 1 % is
used to compensate for the spatial spread of the dense shell (set to $\sim$
10% of the local radius). This spectrum is reminiscent of the observations of
SN 1993J at a similar epoch (red; Matheson et al., 2000). The observed
spectrum has been corrected for redshift and reddening, and then normalized at
4800 Å to match the model spectrum.

###### Acknowledgements.

We acknowledge fruitful discussion with J. Anderson and P. Pessi. This work
was supported by the “Programme National de Physique Stellaire” of CNRS/INSU
co-funded by CEA and CNES. DJH thanks NASA for partial support through the
astrophysical theory grant 80NSSC20K0524. This work was granted access to the
HPC resources of CINES under the 2020 allocation A0090410554 made by GENCI,
France. This research has made use of NASA’s Astrophysics Data System
Bibliographic Services.

## References

  * Bersten & Hamuy (2009) Bersten, M. C. & Hamuy, M. 2009, ApJ, 701, 200
  * Chugai (2001) Chugai, N. N. 2001, MNRAS, 326, 1448
  * Chugai et al. (2007) Chugai, N. N., Chevalier, R. A., & Utrobin, V. P. 2007, ApJ, 662, 1136
  * de Jager et al. (1988) de Jager, C., Nieuwenhuijzen, H., & van der Hucht, K. A. 1988, A&AS, 72, 259
  * Dessart et al. (2015) Dessart, L., Audit, E., & Hillier, D. J. 2015, MNRAS, 449, 4304
  * Dessart et al. (2017) Dessart, L., Hillier, D. J., & Audit, E. 2017, A&A, 605, A83
  * Dessart et al. (2016) Dessart, L., Hillier, D. J., Audit, E., Livne, E., & Waldman, R. 2016, MNRAS, 458, 2094
  * Dessart et al. (2009) Dessart, L., Hillier, D. J., Gezari, S., Basa, S., & Matheson, T. 2009, MNRAS, 394, 21
  * Dessart et al. (2018) Dessart, L., Hillier, D. J., & Wilk, K. D. 2018, A&A, 619, A30
  * Eldridge et al. (2008) Eldridge, J. J., Izzard, R. G., & Tout, C. A. 2008, MNRAS, 384, 1109
  * Fassia et al. (2001) Fassia, A., Meikle, W. P. S., Chugai, N., et al. 2001, MNRAS, 325, 907
  * Fransson et al. (2014) Fransson, C., Ergon, M., Challis, P. J., et al. 2014, ApJ, 797, 118
  * Fuller (2017) Fuller, J. 2017, MNRAS, 470, 1642
  * Gutiérrez et al. (2014) Gutiérrez, C. P., Anderson, J. P., Hamuy, M., et al. 2014, ApJ, 786, L15
  * Gutiérrez et al. (2017) Gutiérrez, C. P., Anderson, J. P., Hamuy, M., et al. 2017, ApJ, 850, 89
  * Gutiérrez et al. (2020) Gutiérrez, C. P., Pastorello, A., Jerkstrand, A., et al. 2020, MNRAS, 499, 974
  * Hillier & Dessart (2012) Hillier, D. J. & Dessart, L. 2012, MNRAS, 424, 252
  * Hillier & Dessart (2019) Hillier, D. J. & Dessart, L. 2019, A&A, 631, A8
  * Jacobson-Galán et al. (2022) Jacobson-Galán, W. V., Dessart, L., Jones, D. O., et al. 2022, ApJ, 924, 15
  * Kulkarni et al. (2021) Kulkarni, S. R., Harrison, F. A., Grefenstette, B. W., et al. 2021, arXiv:2111.15608
  * Langer (2012) Langer, N. 2012, ARA&A, 50, 107
  * Langer et al. (1994) Langer, N., Hamann, W. R., Lennon, M., et al. 1994, A&A, 290, 819
  * Leonard et al. (2000) Leonard, D. C., Filippenko, A. V., Barth, A. J., & Matheson, T. 2000, ApJ, 536, 239
  * Maeder & Meynet (1987) Maeder, A. & Meynet, G. 1987, A&A, 182, 243
  * Matheson et al. (2000) Matheson, T., Filippenko, A. V., Barth, A. J., et al. 2000, AJ, 120, 1487
  * Moriya et al. (2011) Moriya, T., Tominaga, N., Blinnikov, S. I., Baklanov, P. V., & Sorokina, E. I. 2011, MNRAS, 415, 199
  * Moriya et al. (2017) Moriya, T. J., Yoon, S.-C., Gräfener, G., & Blinnikov, S. I. 2017, MNRAS, 469, L108
  * Morozova et al. (2017) Morozova, V., Piro, A. L., & Valenti, S. 2017, ApJ, 838, 28
  * Podsiadlowski et al. (1992) Podsiadlowski, P., Joss, P. C., & Hsu, J. J. L. 1992, ApJ, 391, 246
  * Quataert & Shiode (2012) Quataert, E. & Shiode, J. 2012, MNRAS, 423, L92
  * Renzo et al. (2017) Renzo, M., Ott, C. D., Shore, S. N., & de Mink, S. E. 2017, A&A, 603, A118
  * Schlegel (1990) Schlegel, E. M. 1990, MNRAS, 244, 269
  * Stathakis & Sadler (1991) Stathakis, R. A. & Sadler, E. M. 1991, MNRAS, 250, 786
  * Wellstein & Langer (1999) Wellstein, S. & Langer, N. 1999, A&A, 350, 148
  * Woosley et al. (2002) Woosley, S. E., Heger, A., & Weaver, T. A. 2002, Reviews of Modern Physics, 74, 1015
  * Yaron et al. (2017) Yaron, O., Perley, D. A., Gal-Yam, A., et al. 2017, Nature Physics, 13, 510
  * Yoon et al. (2010) Yoon, S.-C., Woosley, S. E., & Langer, N. 2010, ApJ, 725, 940
  * Zhang et al. (2012) Zhang, T., Wang, X., Wu, C., et al. 2012, AJ, 144, 131

##  Appendix A Additional figures

In this appendix, we provide additional illustrations to document the results
from our radiative transfer simulations. Figure 6 shows the UV and optical
properties at 108.3 d. This epoch corresponds to the onset of the nebular
phase for all models (see the bolometric light curves in the top panel of Fig.
2). Compared to the properties at 41.76 d (Fig. 3), the impact of the
interaction is greater because the radiation from the noninteracting part of
the ejecta is smaller. The lower densities also lead to a greater ionization
and temperature, which all conspire to producing a greater boost to the UV
luminosity. The broad and boxy emission component in the H$\alpha$ profile is
also more developed (except for the highest powers), while the H$\alpha$
emission from the underlying ejecta is weaker and narrower.

Figure 6: Same as Fig. 3, but now at 108.3 d after explosion.

Figures 7 to 12 are similar to Fig. 4, but show the results for the whole set
of models that cover from zero to the highest interaction power of 1043 erg
s-1. At high interaction power, the models were stopped earlier in the
sequence, around 50 d or at the onset of the nebular phase.

Figure 7: Same as Fig. 4, but now for the model Pwr1e40.

Figure 8: Same as Fig. 4, but now for the model Pwr5e41. The time sequence was
stopped at 144 d.

Figure 9: Same as Fig. 4, but now for the model Pwr1e42. The time sequence was
stopped at 191 d.

Figure 10: Same as Fig. 4, but now for the model Pwr2p5e42. The time sequence
was stopped at 174 d.

Figure 11: Same as Fig. 4, but now for the model Pwr5e42. The time sequence
was stopped at 144 d.

Figure 12: Same as Fig. 4, but now for the model Pwr1e43. The time sequence
was stopped at 119 d. Note, however, that the interaction already dominates
the radiative contribution from the underlying ejecta such that, with the
adopted constant interaction power, the subsequent spectral evolution should
be slow and exhibit similar properties.


# Orientation Convolutional Networks
for Image Recognition

Yalan Qin, Guorui Feng, Hanzhou Wu, Yanli Ren, and Xinpeng Zhang Y. Qin, G.
Feng, Y. Ren and X. Zhang are with the School of Communication and Information
Engineering, Shanghai University, Shanghai 200444, China. _(Corresponding
author: Guorui Feng.)_

###### Abstract

Deep Convolutional Neural Networks (DCNNs) are capable of obtaining powerful
image representations, which have attracted great attentions in image
recognition. However, they are limited in modeling orientation transformation
by the internal mechanism. In this paper, we develop Orientation Convolution
Networks (OCNs) for image recognition based on the proposed Landmark Gabor
Filters (LGFs) that the robustness of the learned representation against
changed of orientation can be enhanced. By modulating the convolutional filter
with LGFs, OCNs can be compatible with any existing deep learning networks.
LGFs act as a Gabor filter bank achieved by selecting $p$ $\left(\ll n\right)$
representative Gabor filters as andmarks and express the original Gabor
filters as sparse linear combinations of these landmarks. Specifically, based
on a matrix factorization framework, a flexible integration for the local and
the global structure of original Gabor filters by sparsity and low-rank
constraints is utilized. With the propogation of the low-rank structure, the
corresponding sparsity for representation of original Gabor filter bank can be
significantly promoted. Experimental results over several benchmarks
demonstrate that our method is less sensitive to the orientation and produce
higher performance both in accuracy and cost, compared with the existing
state-of-art methods. Besides, our OCNs have few parameters to learn and can
significantly reduce the complexity of training network.

###### Index Terms:

Convolutional neural networks, orientation, Landmark-based Gabor Filters,
sparsity, low-rank.

##  I INTRODUCTION

Oriented filters play a key role to extract robust image representation. Hand-
crafted features with orientational information encoded like Gabor features
[1] have been investigated. The sample rate of orientation interval for Gabor
function is expected to be dense enough at each fixed central frequency [2].
Such as in [3], the sample rate of 16 is adopted in the application of script
identification. Recently, Deep Convolutional Neural Networks (DCNNs) have
alleviated the effect of local transition by properties of the convolution and
max-pooling. Meanwhile, the inherent capability of dealing with large object
transformation is limited, resulting in poor performance in image
classification, oriented object detection, and many other visual tasks [4, 5].

Due to the lack of capacity to transformation, the direct way is to design the
filter in a proper way. In early years, a general architecture was proposed to
synthesize filters of arbitrary orientations based on linear combinations of
base filters. It enables one to steer a filter to any orientations adaptively,
and determine the output of the filter as a function of the orientation [6].
However, these filters are typically based on directional derivatives and
their symmetry produces orientational responses with period $\pi$ and
independent of image structure [7]. Currently, steerable CNNs [8] present a
theoretical framework for understanding the steerable representation in the
convolutional network, and show that steerability is a useful inductive bias
for improving the model accuracy. Actively rotating filters (ARFs) enforce
DCNNs to have the generalization capacity of the rotation [9]. However, such
rotation for the filter is not proper for large and complex filters. Gabor
convolutional networks (GCNs) [10] are proposed to give deep features the
capacity of dealing with orientation and scale based on Gabor filters.
However, such manipulation is limited to the discriminating capability for
Gabor filters with fixed orientations and scales.

In last decades, matrix factorization [11] has become very hot for the data
representation. It compresses data by seeking for the set of basis vectors and
the corresponding representation for the data Sparse coding [12, 13] method is
proposed as the matrix factorization and has been shown useful for real
applications in different areas such as image restoration [14, 15], salient
object detection [16, 17] and visual neuroscience [18]. Since the data can be
encoded by sparse representations with just a few coefficients, the
computational cost can be reduced and the achieved coding result is relatively
easy to explain. Different sparse based algorithms have been proposed, _i.e.,_
sparse nonnegative matrix factorization [19], sparse principle component
analysis [20]. Moreover, a variety of researches have been focused on
optimization algorithms for the sparse coding [21, 22].

Several studies have considered exploring both the local and global structures
simultaneously [23, 24]. The structured representation such as the sparse and
low-rank representation is evaluated over transformed data points. Although
these methods have achieved the state-of-art performances in various subspace
clustering applications [2021Sparse], a major problem of these existing
methods is the representation ability for one latent learned subspace to data
points which may lie in a union of subspaces. Furthermore, no explicit
theoretical proofs support that the final structured representation
corresponds to the correct clustering structure. To this end, we need to seek
for an efficient integration strategy for the local and global structure.

As is known, the convolution is a linear operation. Since the steerable filter
can synthesize filters of arbitrary orientations by the combinations of
different basis filters in a linear way. If a filter is steerable regarding
the rotation, we can obtain its output corresponding to the rotated version by
linearly combining outputs of filters. If the number of basis filters is
smaller than the total number of orientation samples to be filtered, both the
computation and the storage will be saved. For simplicity, we can assume that
original oriented filters are sampled from a union of different subspaces
which are independent to achieve the above effect.

Based on the above observation, we propose Landmark Gabor Filters (LGFs) and
then apply LGFs to extend the DCNNs to develop a new deep learning
architecture termed Orientation Convolutional Networks (OCNs) for image
recognition. Specifically, LGFs generates $p$ $\left(\ll n\right)$
representative Gabor filters as landmarks and the remaining Gabor filters are
represented as sparse combinations of the landmarks in a linear way. The basic
idea of OCNs is simple but effective. It can effectively capture the
orientation changes by using LGFs to manipulate the convolutional filter in
DCNNs. Unlike most existing DCNNs methods, our method simultaneously considers
learning transformations which are parameterized with a neural network and
achieving an effectively oriented filter bank. Compared with traditional
approaches, OCNs is a deep model, which can map samples with orientation
changes into the latent space explicitly, with parameters learned by a data-
driven manner.

The contribution of our work is twofold. From the perspective of pure oriented
filter bank, we focus on how to let it benefit from the merits of DCNNs where
the nonlinear feature representation can be obtained. From the perspective of
deep neural networks, we analyse that integrating the advantages of
effectively oriented filter bank and deep learning for reinforcing the
robustness of the representation against orientation changes is feasible .

##  II Related Work

###  II-A Steerable Filters and Gabor Filters

Steerable filter is adoted to express a kind of filters where a filter for
arbitrary orientation can be obtained using a linear combination of the set of
basis filters [6]. And the function of rotation
transform$f(x,y):R^{2}\rightarrow C$ is steerable regarding the rotation:

$f^{\theta}(x,y)=\sum\limits_{j=1}^{M}k_{j}(\theta)\varphi^{j}(x,y),$ (1)

where $f^{\theta}(x,y)$ denotes the rotated version of $f(x,y)$ by the angle
$\theta$. $\varphi^{j}(x,y)$ is the the base function independent of the
rotation angle $\theta$, and $k_{j}(\theta)$ is the steering function of
$f(x,y)$. Streerable CNNs [8] present a theoretical framework for
understanding the steerable representation in convolutional network, and show
that steerability is a useful inductive bias for improving model accuracy.

Gabor wavelets are proposed by utilizing complex functions to act as the basis
for related Fourier transforms [25]. Gabor filters are widely adopted for
modeling receptive fields of simple cells of the visual cortex, and are as
follows [26]:

$G(u,v)=\frac{||k_{u,v}||^{2}}{\sigma^{2}}e^{-(||k_{u,v}||^{2}||z||^{2}/2\sigma^{2})}\left[e^{ik_{u,v}z}-e^{-\sigma^{2}/2}\right],$
(2)

where $u$ denotes the orientation, $v$ is the frequency,
$k_{u}=u\frac{\pi}{U}$, $k_{v}=(\pi/2)/\sqrt{2}(v-1)$, with $u=0,1,...,U$ and
$v=0,1,...,V$, $\sigma=2\pi$ and $z=(x,y)$.

###  II-B Matrix Factorization and Sparse Coding

Matrix factorization is quite hot for data representation. For each data
point, it aims for compressing the data by seeking for a set of basis vectors
and the representation corresponding to the related basis. Given the data
matrix $X=\left[x_{1},x_{2},...,x_{n}\right]\in R^{m\times n}$, matrix
factorization is defined by seeking for matrices $U\in R^{m\times p}$ and
$V\in R^{p\times n}$ whose product can well approximate $X$ [27]:

$X\approx UV,$ (3)

where each column of $U$ can be seen the basis vector and each column of $V$
is the representation corresponding to the basis vector. Frobenius norm of a
matrix $||\;.\;||_{F}^{2}$ can be used to measure the approximation. Then, it
can be formulated as:

$\min_{U,V}||X-UV^{T}||_{F}^{2}.$ (4)

Obviously, the solution $V$ of the optimization problem is inevitablely non-
sparse. To address this issue, sparse coding [19] is proposed as a matrix
factorization technique for encoding different data points with only a few
coefficients. Sparse coding adds the sparse constraint on $V$ in the
optimization (3) and is defined as follows

$\min_{U,V}||X-UV||_{F}^{2}+\alpha||V||_{0}$ (5)

where $||\;.\;||_{0}$ is used for measuring the sparsity and $\alpha$ denotes
a coefficient controlling the sparsity penalty. By imposing the low-rank
structure on $V$ in (3), the low-rank representation is defined as follows

$\min_{U,V}||X-UV||_{F}^{2}+\beta rank(V)$ (6)

where $rank(V)$ denotes the rank value of $V$ and $\beta$ is a coefficient
controlling the low-rank penalty. Under the low-rank constraint, the
correlation between different data points is enhanced within clusters and
weakened for different clusters. However, the low-rank representation is not
guaranteed to be of the block-diagonal structure [28] in practice, which
reflects the expected relationship of data points. Besides, the low-rank
representation is not able to fully utilize the local information.

###  II-C Robust features learning

Given rich, and redundant, convolutional filters, data augmentation can be
used to obtain local or global transform invariance [29]. Despite the
performance of the data augmentation, the main drawback is that learning all
possible transformations often needs more parameters, resulting in the
increasing training cost and the risk of over-fitting significantly. Recently,
TI-Pooling [30] is proposed to use the parallel network architectures for the
considered transform set. Nevertheless, with the data augmentation, TI-Pooling
requires dramatically more cost for training and testing than the standard
DCNNs.

Deformable convolutional networks: Deformable convolution is proposed in [31]
to greatly increase the transformation modeling capability of CNNs, enabling
the network robust to dense spatial transformations in a simple, deep,
efficient and end-to-end manner. However, small size filters are preferred for
the deformable filters, _i.e.,_ $3\times 3$.

Spatial Transform Network: Spatial Transformer Network (STN) [32] uses an
additional network module to manipulate the feature maps. STN indeed proposes
a general framework for the spatial transform, but fails to estimate the
complex transform parameters precisely by giving a solution.

Wide Residual Networks: Wide Residual Networks (WRN) [33] performs a detailed
experimental investigation on the ResNet [34] networks as well as proposes a
novel architecture to decrease the depth and increase the width of the ResNet
networks. However, the size of parameters for WRN is relatively large.

Oriented Response Networks: Oriented Response Network (ORN) [9] is introduced
to actively rotate filters (ARFs) during the convolution and produce feature
maps with location and orientation encoded. The ARF can be viewed as a virtual
filter bank including the filter itself and multiple un-materialised rotated
versions corresponding to it. However, ORN is just applicable for filters with
small size.

Gabor Convolutional Networks: Gabor Convolutional Networks (GCNs) [10]
incorporates Gabor filters into deep convolutional neural networks (DCNNs). By
only manipulating the convolution operator, GCNs can be implemented and are
compatible with the current deep learning architecture. Such a modulation is
limited to the discriminating capability for orientation and scale using Gabor
filters, which is absent of considering some cases Gabor filters cannot
capture.

To deal with different rotation angels, we require a considerable amount of
oriented filters, _i.e.,_ Gabor filters. In practice, due to the orientation
interpolation caused by the operation of multi-layer pooling, we can utilize a
limited amount of orientations to ensure the accuracy. Furthermore, motivated
by the fact that the dimension of basis vector (column vectors of $U$) in (3)
is the same as original Gabor filters, we can regard basis vectors as LGFs of
original Gabor filters to reduce the complexity as well as guarantee the
performance.

Different from existing work, we propose a new model to bridge LGFs and deep
neural networks for nonlinear feature representation for samples with
orientation changes. To be specific, our framework, _i.e.,_ OCNs,
simultaneously considers finding effective oriented filter bank and learning
high-level features from inputs, whereas these existing methods do not embrace
the effectiveness of feature learning with orientation changes. We propose a
novel method for obtaining effective LGFs and the corresponding coefficients
in a matrix factorization framework. Since the sparsity of the coefficients
can be increased by the propagation of low-rank structure, we propose an
effective integration strategy for the local and global structure, which
correspond to sparse coding and low-rank representation, leading to effective
LGFs. Then, it is possible to determine responses of any arbitary oriented
filters without explicitly applying those filters. We believe that LGFs is
complementary with the existing deep neural networks methods since it is able
to incorporate the merits of exiting methods into DCNNs. To the best of our
knowledge, this is the first attempt to integrate LGFs and DCNNs. Our method
is also significantly from [10], which just modulates the convolutional filter
with the empirically selected Gabor filter bank.

The main contributions in this paper are summarized as follows:

  1. 1.

We define LGFs and the corresponding coefficients in a matrix factorization
framework. To achieve effective LGFs, we promote the sparsity of coefficients
by propagating the low-rank structure.

  2. 2.

The proposed LGFs is formulated as an optimization problem with a well-defined
objective function. we then design an effectively iterative algorithm to seek
for the solution of the constrained optimization problem.

  3. 3.

We propose OCNs by modulating the basic element of DCNNs based on LGFs with
existing deep learning architecture, which considers LGFs and deep neural
networks simultaneously, enforcing the network compact while still obtaining
satisfied feature representation ability with orientation changes.

##  III Landmark Gabor Filters

In this section, we introduce our LGFs for manipulating the convolution filter
in DCNNs. The basic purpose of our method is presenting a way for finding
effective oriented filter bank, which can employ a limited amount of oriented
filters for the accuracy.

We first calculate $n$ rotated versions of the Gabor filters $G(u,v)$ based on
Eq. (2) and then re-arrange each version into a column vector and then build
the matrix $X\in R^{m\times n}$. Based on matrix factorization, we propose to
find matrices $U\in R^{m\times p}$ and $V\in R^{p\times n}$ whose product can
approximate $X$ via $\min_{U,V}||X-UV||_{F}^{2}$. To capture the global
structure for the set of oriented filters, we put a constraint of low-rank on
the LGFs and coefficients for a low-rank representation $U$ and $V$. Based on
$V$, we propose to find matrices $Y\in R^{p\times q}$ and $Z\in R^{q\times n}$
whose product can best approximate $V$ via the constraint $V=YZ$ and enforce a
sparsity constraint on $Z$ for a sparse representation $Z$. It is formulated
as follows

$\begin{split}\begin{aligned} \min_{U,V,Y,Z}&||X-UV||_{F}^{2}+\lambda
rank(U)+\mu rank(V)\\\
&+\gamma||Z||_{0}\;\;s.t.\;V=YZ,\end{aligned}\end{split}$ (7)

where $\lambda>0,\;\mu>0,\;\gamma>0$ are the tradeoff parameters. Note that
the objective function in (7) is NP-hard for the rank function $rank(\;.\;)$
and the $l_{0}$-norm $||\;.\;||_{0}$ on their minimizations. As is known, the
nuclear norm $||\;.\;||_{*}$ is widely adopted and has been proved to be the
convex envelop of $rank(\;.\;)$. Meanwhile, the $l_{1}$-norm $||\;.\;||_{1}$
has been demonstrated as an effective approximations of $l_{0}$-norm
$||\;.\;||_{0}$. Then we can obtain:

$\begin{split}\begin{aligned}
\min_{U,V,Y,Z}&||X-UV||_{F}^{2}+\lambda||U||_{*}+\mu||V||_{*}+\gamma||Z||_{1}\\\
&s.t.\;V=YZ,\end{aligned}\end{split}$ (8)

where the nuclear norm $||U||_{*}=\sum_{i}|s_{i}|$. $s_{i}$ is the singular
value of the matrix $U$ and likewise $||V||_{*}$. Obviously, with the
lpropagation, the sparse representation $Z$ can encode with both the global
and local structure. Since $U\in R^{m\times p},Y\in R^{p\times q}$ and $UY\in
R^{m\times q},Z\in R^{q\times n}$, each column of $UY$ can be seen as a LGF
capturing high-level features of the original oriented filter. To better deal
with the noise, _i.e.,_ Gaussian noise, we relax the constraint of (8). This
yields the optimization problem as follows

$\begin{split}\begin{aligned}
\min_{U,V,Y,Z}&||X-UV||_{F}^{2}+\lambda||U||_{*}+\mu||V||_{*}+\gamma||Z||_{1}\\\
&+\rho||V-YZ||_{F}^{2},\end{aligned}\end{split}$ (9)

where least squares, nuclear-minimizations and $l_{1}$-minimizations are
involved in (9). The problem to be solved in (9) is non-convex regarding joint
$(U,V,Y,Z)$. We then employ the efficient iterative algorithm [35] to seek for
the solution of the optimization problem in (9). We solve the nuclear-
minimization and $l_{1}$-minimization in (9) by the Singular Value Shrinkage
Thresholding algorithm (SVT) [36] and the Iterative Shrinkage Thresholding
algorithm (IST) [37], respectively, which are defined in the following.

Lemma 1 (Iterative Shrinkage Thresholding [37]) Given a matrix M and a
positive number $\tau$, the following minimization

$\min_{X}\tau||X|_{1}+1/2||X-M||^{2}_{F}$ (10)

has a global optimal solution in the closed form

$X=sign(M)max(0,|M|-\tau).$ (11)

Lemma 2 (Singular Value Shrinkage Thresholding[36]) Given a matrix $M=USV^{T}$
and a positive number $\tau$, the following minimization

$\min_{X}\tau||X|_{*}+1/2||X-M||^{2}_{F}$ (12)

has a global optimal solution in the closed form

$X=Usign(S)max(0,|S|-\tau)V^{T}.$ (13)

Based on the Alternating Direction Minimization (ADM) strategy, we divide the
objective function into the following subproblems:

(1) $Y$-Subproblem: The $Y$-subproblem is shown as

$\begin{split}\begin{aligned}
Y^{*}=\arg\min\limits_{Y}\;\rho||V-YZ||_{F}^{2},\end{aligned}\end{split}$ (14)

Taking the derivative with respect to $U$ and then enforcing it to be zero, we
obtain the closed-form solution as:

$Y^{*}=V^{T}Z(Z^{T}Z)^{-1}.$ (15)

(2) Updating $U$: The LGFs $U$ is updated by

$\begin{split}\begin{aligned}
U^{*}=\arg\min\limits_{U}\;||X-UV||_{F}^{2}+\lambda||U||_{*},\end{aligned}\end{split}$
(16)

(3) Updating $V$: The coefficient $V$ with respect to $U$ is updated by

$\begin{split}\begin{aligned}
V^{*}=\arg\min\limits_{V}\;||X-UV||_{F}^{2}+\lambda||V||_{*}+\rho||V-YZ||_{F}^{2},\end{aligned}\end{split}$
(17)

(4) Updating $Z$: The coefficient $Z$ regarding $Y$ is updated by

$\begin{split}\begin{aligned}
Z^{*}=\arg\min\limits_{Z}\;\gamma||Z||_{1}+\rho||V-YZ||_{F}^{2},\end{aligned}\end{split}$
(18)

We solve the subproblems corresponding to $U$ and $V$ by the SVT algorithm.
IST can be adopted to deal with the subproblem corresponding to $Z$. For
clarification, we summarize the optimization procedure in Algorithm 1.

Input : Set of oriented filters $X=\left[x_{1},x_{2},...,x_{n}\right]\in
R^{m\times n}$, regularization parameters $\rho,\;\lambda,\;\mu,\;\gamma$.

Output : Basis matrix $U,Y$ and coefficient matrix $V,\;Z$.

Initialize : Randomly initialize $U,\;V,\;Y,\;Z$ and set $i=1$.

repeat

Fix $U,\;V$ and $Z$, update $Y$ by updating rule (15);

Fix $V,\;Y$ and $Z$, update $U$ according to Eq. (16);

Fix $U,\;Y$ and $Z$, update $V$ according to Eq. (17);

Fix $U,\;V$ and $Y$, update $Z$ according to Eq. (18);

$i=i+1$.

until  _$convergence$_ ;

Algorithm 1 Algorithm of the proposed LGFs

Convergence Analysis:  The problem in (9) is not convex regarding the joint
variables $\left(U,V,Y,Z\right)$. The objective function in (9) is convex
regarding each single variable in $\left(U,V,Y,Z\right)$ since only
$l_{1}$-norm, Frobenius-norm and nuclear-norm are involved and all of them are
convex functions.

In the following, we theoretically show that the optimal solution
$\left(V,Z\right)$ to (9) is the final clustering result. Inspired by [38]
that we can explicitly pursue the block-diagonal structure in the matrix
factorization framework, we give a theoretical proof of the optimal solution
$\left(V,Z\right)$ is block diagonal. Meanwhile, we also show why $Z$ is more
proper as the coefficient.

Theorem 1. Let $X\in R^{m\times n}$ be a set of oriented filters whose columns
are sampled from a union of subspaces which are independent. Assume that $X$
has been sorted based on the subspaces, i.e.,
$X=\left[x_{1},x_{2},...x_{n}\right]\Gamma$, where $\Gamma$ is an permutation
matrix specifying subspaces of original oriented filters. Then there is the
optimal solution  $\left(V,Z\right)$ to  (9) with $V$ and $Z$ being both block
diagonal, i.e., there are just connections within subspace and no connection
between subspaces.

Proof. Let $\left(V,Z\right)$ be the optimal solution to (9). Note that
different subspaces distinguish each other with different oriented basis
filters, _i.e.,_ LGFs, we can use the oriented basis filter to represent the
subspace for clarity. Since $V$ is a solution of the nuclear norm problem in
(9), then $v_{i}$ is a vector of minimum nuclear norm satisfying
$x_{i}=Uv_{i}$. Let us decompose $v_{i}$ as $v_{i}=v_{i}^{{}^{\prime}}+h_{i}$,
where $v_{i}^{{}^{\prime}}$ denotes the coefficient recovered from the true
basis filter of $x_{i}$ and $h_{i}$ denotes the coefficient recovered from the
other basis filter. Now we just need to prove $h_{i}=0$. Then, for any
oriented filter $x_{i}$, since $v_{i}=v_{i}^{{}^{\prime}}+h_{i}$, we have
$x_{i}=Uv_{i}=U(v_{i}^{{}^{\prime}}+h_{i})=Uv_{i}^{{}^{\prime}}+Uh_{i}$.
Moreover, since $x_{i}\in U_{i}$ , $Uv_{i}\in U_{i}$ and $Uh_{i}\notin U_{i}$,
we can obtain $Uh_{i}=0$. Based on the fact that $v_{i}^{{}^{\prime}}$ and
$h_{i}$ have disjoint subset of indices, if $h_{i}\notin 0$ is assumed, we can
obtain
$||v_{i}^{{}^{\prime}}||_{*}<||v_{i}^{{}^{\prime}}+h_{i}||_{*}=||v_{i}||_{*}$,
which contradicts the optimality of $v_{i}$. Then, $h_{i}=0$ and $V$ is block
diagonal.

Moreover, based on $V$ , if $Z$ is the solution of the $l_{1}$-norm problem in
(9), then $z_{i}$ is a vector of the minimum $l_{1}$-norm satisfying
$x_{i}=Wz_{i}$. Let us decompose $z_{i}$ as $z_{i}=z_{i}^{{}^{\prime}}+o_{i}$,
where $z_{i}^{{}^{\prime}}$ denotes the coefficient recovered from the true
basis filter of $x_{i}$ and $o_{i}$ denotes the coefficient recovered from the
other basis filter. As above, we just need to prove $o_{i}=0$. Since
$z_{i}=z_{i}^{{}^{\prime}}+o_{i}$, then
$x_{i}=Wz_{i}=W(z_{i}^{{}^{\prime}}+o_{i})=Wz_{i}+Wo_{i}$ can be obtained.
Besides, since $x_{i}\in W_{i},Wz_{i}\in W_{i}$ and $Wo_{i}\notin W_{i}$, we
can obtain $Wo_{i}=0$. Since $z_{i}^{{}^{\prime}}$ and $o_{i}$ have the
disjoint subset of indices, if $o_{i}\notin 0$ is assumed, we can obtain
$||z_{i}^{{}^{\prime}}||_{1}<||z_{i}^{{}^{\prime}}+o_{i}||_{1}=||z_{i}||_{1}$,
which contradicts the optimality of $z_{i}$. Therefore, we can obtain
$o_{i}=0$ and $Z$ is block diagonal, which concludes the proof.

Since the solution $V$ may not be unique, which has been proved in [39],
Theorem 1 is not able to ensure that any optimal $V$ is block diagonal with an
unknown permutation $\Gamma$. However, according to Theorem 1, any optimal $Z$
can be guaranteed to be block diagonal under an unknown permutation $\Gamma$.
This also explains why we choose $Z$ instead of $V$ as our coefficient.
Moreover, we can observe that $Z$ incorporates both the global and local
structure of original oriented filters. However, $V$ just encodes the global
structure. Then, $UY\in R^{m\times q}$ with the coefficient $Z\in R^{q\times
n}$ is used as our LGFs.

##  IV Orientation Convolutional Networks

In order to incorporate the orientation and scale information into OCNs, we
modulate the convolutional filters in standard CNNs with LGFs. Then, the
corresponding convolutional features in OCNs can be enhanced. Unlike standard
DCNNs, which express the dimensions of the weight per layer as $C_{out}\times
C_{in}\times W\times W$, OCNs denote it as $C_{out}\times C_{in}\times
N^{{}^{\prime}}\times W\times W$, where $W\times W$ is the size for the
filter, $N^{{}^{\prime}}$ is the channel, $C_{out}$ and $C_{in}$ refer to the
channel for the corresponding feature maps. Here, $N^{{}^{\prime}}$ is set by
the number of orientations $U$. For the given scale $v$, we define the
modulated procedure using LGFs in DCNNs as:

$\begin{split}\begin{aligned} C_{i,u}^{v}=C_{i,o}\odot
LG(u,v),\end{aligned}\end{split}$ (19)

where $C_{i,o}$ denotes the learned filter, and $\odot$ is a operation for the
element-by-element product between each 2D filter of $C_{i,o}$ and $LG(j,v)$.
$C_{i,j}^{v}$ is the modulated one. The obtained modulated convolution filters
are defined as:

$\begin{split}\begin{aligned}
C_{i}^{v}=\left(C_{i,1}^{v},C_{i,2}^{v},...,C_{i,U}^{v}\right).\end{aligned}\end{split}$
(20)

For each scale, we can achieve enhanced features in this way. In the
following, we omit the scale $v$ to simplify the description process.

The obtained modulated convolution filters are basic element of OCNs, which
can generate feature maps encoding the orientation and scale information. The
output feature map $\widehat{F}$ in OCNs is defined as:

$\begin{split}\begin{aligned}
\widehat{F}_{i,k}=\sum\limits_{n=1}^{N^{{}^{\prime}}}F^{n}\otimes
C_{i,u=k}^{n}\end{aligned}\end{split}$ (21)

where $N^{{}^{\prime}}$ refers to the channel, $F$ is the input feature map,
$n$ is the n-th channel of $F$. $C_{i,u}$. $\widehat{F}_{i,k}$ refers to the
k-th response of $F_{i}$.

In OCNs, the weights contained in the forward convolution process are the
modulated convolution filters. However, the weights needed to be saved are
just the learned filters. Then, we update the learned filter $C_{i,o}$ in the
back-propagation (BP) process. By summing up the gradient of all filters in
the modulated convolution filters, we can obtain:

$\begin{split}\begin{aligned} &C_{i,o}=C_{i,o}-\eta\upsilon,\\\
&\upsilon=\frac{\partial Loss}{\partial C_{i,o}}=\sum_{u=1}^{U}\frac{\partial
Loss}{\partial C_{i,u}}\odot LG(u,v),\end{aligned}\end{split}$ (22)

where $Loss$ refers to the loss function and $\eta$ is the learning rate in
the BP process. It can be found that the BP process in our method is the same
as the standard BP process and we just need to update the learned convolution
filters $C_{i,o}$, resulting in a more compact and efficient model.

##  V Experiments and Analysis

In this section, we implement OCNs based on the conventional CNNs and the
ResNet, which are listed in Fig. 2. We evaluate the proposed OCNs on MNIST
digital recognition dataset, the rotated version MNIST-rot dataset, CIFAR-10
dataset, CIFAR-100 dataset. NVIDIA GeForce GTX 1080Ti is used in our
experiment. We first investigate how parameters in (9) influence the
performance and show the selection of parameters in the experiment.

###  V-A Parameters Selection

To investigate how parameters $(\rho,\mu,\lambda,\gamma)$ in (9) influence the
performance, we use $D=\frac{||\hat{X}||_{F}^{2}}{||X||_{F}^{2}}$ as a
measurement of the approximation degree of original oriented filters. We seek
for $(\rho,\mu,\lambda,\gamma)$ corresponding to the maximal degree of
approximation. Then the problem becomes seeking for the maximal $D$.

For clarity, we employ the original set of oriented filters in the following.
The set of oriented filters are based on the Gabor filter and generated as:
the central frequency $F=16$ and the orientation sample interval of $5^{o}$
are used. For Gabor filters with central frequency 16, 5 LGFs are used. Since
the initial result shows that parameters should be set to small values, we
choose in $\left\\{0.001,0.005,0.01,0.05,0.1,0.5\right\\}$ for investigations.
To reduce the computation complexity, we fist set $\rho=\mu=\gamma$ and find
the optimal $\lambda$. Then we find the optimal $\rho,\mu,\gamma$ with the
fixed optimal $\lambda$. Likewise, we set $\rho=\gamma$ and find the optimal
$\mu$. Then we find the optimal $\rho,\gamma$ with the fixed optimal $\mu$.
According to Fig. 1, we can set
$\rho=0.05,\;\mu=0.05,\;\gamma=0.05,\;\lambda=0.05$ for the four parameters
for simplicity.

Figure 1: Degree of approximation of oriented filters: (a) $\lambda$ vs
$(\rho,\gamma,\mu)$; (b) $\mu$ vs $(\rho,\gamma)$; (c) $\gamma$ vs $\rho$.

Figure 2: Network structure of CNN, ORN and OCNs. TABLE I: Error rate $(\%)$
on MNIST when V=1.
Size | 10-20-40-80 | 20-40-80-160
---|---|---
$5\times 5$ | 0.58 | 0.59
$3\times 3$ | 0.62 | 0.60
TABLE II: Error rate $(\%)$ on the MNIST when V=4.
Size | 10-20-40-80 | 20-40-80-160
---|---|---
$5\times 5$ | 0.57 | 0.51
$3\times 3$ | 0.62 | 0.59
TABLE III: Error rate $(\%)$ on the MNIST of different orientations.
Size | 2 | 3 | 4 | 5 | 6 | 7
---|---|---|---|---|---|---
$5\times 5$ | 0.56 | 0.54 | 0.51 | 0.43 | 0.44 | 0.48
$3\times 3$ | 0.623 | 0.58 | 0.59 | 0.54 | 0.54 | 0.59
TABLE IV: Performace comparison on the MNIST. Method | Error$(\%)$
---|---
| network stage kernel | params (M) | times (s) | MNIST | MNIST-rot
Baseline CNN | 80-160-320-640 | 3.08 | 6.50 | 0.73 | 2.82
STN | 80-160-320-640 | 3.20 | 7.33 | 0.61 | 2.52
TIpooling$(\times 8)$ | (80-160-320-640)$(\times 8)$ | 3.08 | 50.21 | 0.97 | -
ORN4(ORAlign) | 10-20-40-80 | 0.49 | 9.21 | 0.57 | 1.69
ORN8(ORAlign) | 10-20-40-80 | 0.96 | 16.01 | 0.59 | 1.42
ORN4(ORPooling) | 10-20-40-80 | 0.25 | 4.60 | 0.59 | 1.84
ORN8(ORPooling) | 10-20-40-80 | 0.39 | 6.56 | 0.66 | 1.37
GCN4$(3\times 3)$ | 10-20-40-80 | 0.25 | 3.45 | 0.63 | 1.45
GCN4$(3\times 3)$ | 20-40-80-160 | 0.78 | 6.67 | 0.56 | 1.28
GCN4$(5\times 5)$ | 10-20-40-80 | 0.51 | 10.45 | 0.49 | 1.26
GCN4$(5\times 5)$ | 20-40-80-160 | 1.86 | 23.85 | 0.48 | 1.10
GCN4$(7\times 7)$ | 10-20-40-80 | 0.92 | 10.80 | 0.46 | 1.33
OCN4$(7\times 7)$ | 20-40-80-160 | 3.17 | 25.17 | 0.42 | 1.20
OCN4$(3\times 3)$ | 10-20-40-80 | 0.25 | 3.43 | 0.62 | 0.81
OCN4$(3\times 3)$ | 20-40-80-160 | 0.78 | 6.51 | 0.59 | 0.70
OCN4$(5\times 5)$ | 10-20-40-80 | 0.51 | 9.47 | 0.57 | 0.55
OCN4$(5\times 5)$ | 20-40-80-160 | 1.86 | 22.79 | 0.51 | 0.65
OCN4$(7\times 7)$ | 10-20-40-80 | 0.92 | 12.70 | 0.41 | 0.56
OCN4$(7\times 7)$ | 20-40-80-160 | 3.47 | 29.02 | 0.40 | 0.55

Figure 3: The residual block. The former two are used for ResNet. The third
one is small kernel for OCNs. The last one in large kernel for OCNs.

Figure 4: Sample images which contain rotated objects/parts falsely classified by ResNet but correctly classified by the proposed OCNs in CIFAR-10. TABLE V: Performace comparison on the CIFAR-10 and the CIFAR-100. Method | Error on CIFAR-10$(\%)$ | Error on CIFAR-100$(\%)$
---|---|---
NIN | 8.81 | 35.67
VGG | 6.32 | 28.49
| network stage kernel | params (M) |  |
ResNet-110 | 16-16-32-64 | 1.7 | 6.43 | 25.16
ResNet-1202 | 16-16-32-64 | 10.2 | 7.83 | 27.82
GCN2-40 | 16-32-64-128 | 4.5 | 4.95 | 24.23
GCN2-110 | 12-12-24-45 | 1.7/3.3 | 6.34/5.62 | -
GCN2-110 | 16-16-32-64 | 3.4/6.5 | 5.65/4.96 | 26.14/25.3
GCN4-110 | 8-8-16-32 | 1.7 | 6.19 | -
GCN4-40 | 16-16-32-64 | 2.2 | 5.34 | 25.65
OCN2-110 | 12-12-24-45 | 1.7/3.3 | 6.32/5.34 | -
OCN2-110 | 16-16-32-64 | 3.4/6.5 | 5.65/5.95 | 26.35/25.5
OCN4-110 | 8-8-16-32 | 1.7 | 6.07 | -
OCN2-40 | 16-32-64-128 | 4.5 | 4.83 | 22.34
OCN4-40 | 16-16-32-64 | 2.2 | 5.21 | 24.16
WRN-40 | 64-64-128-256 | 8.9 | 4.53 | 21.18
WRN-28 | 160-160-320-640 | 36.5 | 4.00 | 19.25
GCN2-40 | 16-64-128-256 | 17.9 | 4.41 | 20.73
GCN4-40 | 16-32-64-128 | 8.9 | 4.65 | 21.75
GCN3-40 | 64-64-128-256 | 17.6 | 3.88 | 20.13
OCN2-40 | 16-64-128-256 | 17.9 | 4.21 | 20.76
OCN4-40 | 16-32-64-128 | 8.9 | 4.75 | 22.04
OCN3-40 | 64-64-128-256 | 17.6 | 3.76 | 19.45

###  V-B MNIST and MNIST-rot

MNIST dataset has 60,000 data points used for training and 10,000 data points
used for testing. For our method, we set the batch size as 128 and the weight
decay as 0.00005. Meanwhile, the initial learning rate is set as 0.001 and the
learning rate is reduced to half per 10 epochs. In the following, we show the
performance of our method on the testing set based on the average 5 runs after
50 epochs. For baselines, we tune their parameters to achieve the best
results. For comparision, we use different algorithms including STN[32], TI-
Pooling[30], GCNs[10] and ORNs[9]. For clarity, in OCNs, we use OConv (OC) as
the name of the enhanced convolution filter which is modulated with LGFs
generated by our method.

Note that network structures of conventional CNNs, ORNs and ours are shown in
Fig. 2. It can be seen that Max-pooling and ReLu are used after convolutional
layers for all network structures. Meanwhile, dropout layer is used for
avoiding over-fitting. For comparison with the other CNNs, the size of similar
model is guaranteed by reducing the width of the convolutional kernel [33] as
done in ORNs. We first test several scales for different OCNs layers, _i.e.,_
$V=1,V=4$. According to Tables I-II, we can observe that the performance of
$V=4$ is better than that of $V=1$ in terms of error rate. The number of
orientations is also evaluated in Table III when $V=4$, which shows that OCNs
can achieve satisfied performance when 3 to 7 orientations are used. It can be
found that enough orientation channels are needed to capture the oriented
information.

We also compare the proposed OCNs with the baseline methods on MNIST and
MNIST-rot dataset. As is shown in Table IV, the last two columns show the
performance comparison in terms of the error rate. When compared with the
baseline conventional CNNs, OCNs show better performance with $3\times 3$
kernel size but using fewer parameters of CNNs. Besides, it can be observed
that the testing error of OCNs with $5\times 5$ and $7\times 7$ kernel size is
$0.55\%$ on MNIST-rot and $0.40\%$ on MNIST dataset. It is due to the fact
that larger kernel sizes have more oriented information. Besides, we list the
corresponding computation time of compared methods, which demonstrate that
OCNs are more effective than other state-of-art baseline models. Table IV also
shows that a larger OCNs can indeed produce better performance, which
validates that oriented filters approximated by our method help to achieve the
robustness to rotation variations.

TABLE VI: Performance comparison on the SVHN without using the additional
training set.
Method | VGG | ResNet-110 | ORN4-40 | ORN4-28 | GCN4-40 | GCN4-28 | OCN4-40 | OCN4-28
---|---|---|---|---|---|---|---|---
Params(M) | 20.3 | 1.7 | 2.2 | 1.4 | 2.2 | 1.4 | 2.2 | 1.4
Accuracy$(\%)$ | 95.66 | 95.8 | 96.35 | 96.19 | 96.9 | 96.86 | 97.1 | 96.98

###  V-C CIFAR10 and CIFAR100

The CIFAR dataset which contains CIFAR-10 and CIFAR-100 is used for
classification issue of the natural image. It has 50,000 images used for
training and 10,000 images used for testing. CIFAR-10 consists of 10 classes
with 6,000 images contained per class and CIFAR-100 has 100 classes with 600
images contained per class. The size of each colored image is $32\times 32$.

The network structure used is based on ResNet [34]. For clarity, with the
enhanced convolution filter as above, we use OCN-ResNets as the name of the
network. For baselines, we tune their related parameters to achieve the best
results. Table V depicts results of different algorithms such as NIN [40] and
ResNet. On CIFAR-10, Table IV indicates that OCN-ResNets indeed improve the
performance both in the different numbers of parameters and kernel size
compared with the baseline ResNet. Furthermore, the result compared with the
Wide Residue Network (WRN)[33] shows that OCN-ResNets can still obtain a
better performance when OCN-ResNets are half the size of WRN, demonstrating
its advantage in terms of the model efficiency. Due to the huge parameters of
WRN and more categories in CIFAR-100 data set, we can still observe that WRN
better performs on CIFAR-100 data set. In addition, according to Fig. 4, we
can observe that the top improved categories in CIFAR10 dataset are truck
$(3.5\%)$, bird $(2.6\%)$, and deer $(3.0\%)$, indicating significant scale
variations exist within category. It demonstrates that OCN-ResNet indeed
enhances the capability of dealing with scale variation.

###  V-D SVHN dataset

The Street View House Numbers (SVHN) dataset [41] is an image dataset. It has
$32\times 32$ images centered around a single character like MNIST. There are
531,131 additional images in this dataset. In the experiments, we do not use
the additional images for all models. For such dataset, we implement OCNs
based on ResNet. Specifically, the spatial convolutional layers are replaced
with the OCconv layer based on modulated convolution filters, resulting in
OCN-ResNet. Besides, the bottleneck structure is not adopted due to the fact
that $1\times 1$ kernel does not propagate any information of modulated
convolution filters. The whole network is divided into 4 stages by ResNet. To
ensure that the proposed OCNs method has the similar model size as the
compared ResNet, we adjust the depth and width of the network. In our
experiment, the hyper-parameters for 40-layer and 28-layer OCN-ResNets are set
the same as ResNet. The network stage is set as 16-16-32-64. In Table VI, the
performances of different methods are shown. OCN-ResNet has much smaller
parameter size than VGG model, yet an improved performance with $1.32\%$ is
obtained. Without an increased parameter size, the OCN-ResNet shows a better
performance than ResNet and GCNs respectively, which demonstrates the
effectiveness of OCN-ResNet for real world images.

##  VI Conclusion

This paper studies the problem of obtaining effective oriented filters,
_i.e.,_ LGFs, and proposes a new deep learning model by combining LGFs and
DCNNs for enhanced representations. The main contribution is designing LGFs,
as well as improving state-of-the-art DCNNs architectures on the
generalization ability towards orientation and scale variations. LGFs can be
easily implemented on the exiting architectures and the whole process of
designing LGFs is flexible and extensible, which makes it better modulate the
standard filter in DCNNs. Experimentally, OCNs significantly outperform the
baselines, obtaining the sate-of-the-art performance over benchmarks.

## References

  * [1] P. Perona, “Deformable kernels for early vision,” in _1991 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR)_ , 1991, pp. 222–227.
  * [2] G. M. Haley and B. S. Manjunath, “Rotation-invariant texture classification using modified gabor filters,” in _1995 International Conference on Image Processing (ICIP)_ , vol. 1, 1995, pp. 262–265 vol.1.
  * [3] T. N. Tan, “Rotation invariant texture features and their use in automatic script identification,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 20, no. 7, pp. 751–756, 1998.
  * [4] Y. Tang and X. Wu, “Salient object detection using cascaded convolutional neural networks and adversarial learning,” _IEEE Transactions on Multimedia_ , vol. 21, no. 9, pp. 2237–2247, 2019.
  * [5] H. Xiao, J. Feng, Y. Wei, M. Zhang, and S. Yan, “Deep salient object detection with dense connections and distraction diagnosis,” _IEEE Transactions on Multimedia_ , vol. 20, no. 12, pp. 3239–3251, 2018.
  * [6] W. Freeman and E. Adelson, “The design and use of steerable filters,” _IEEE Transactions on Pattern Analysis and Machine Intelligence,_ , vol. 13, pp. 891–906, 10 1991.
  * [7] E. Simoncelli and H. Farid, “Steerable wedge filters for local orientation analysis,” _IEEE transactions on image processing_ , vol. 5, pp. 1377–82, 02 1996.
  * [8] S. C. Taco and W. Max, “Steerable cnns,” in _2016 International Conference on Learning Representations (ICLR)_ , 2016.
  * [9] Y. Zhou, Q. Ye, Q. Qiu, and J. Jiao, “Oriented response networks,” in _2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2017, pp. 4961–4970.
  * [10] S. Luan, C. Chen, B. Zhang, J. Han, and J. Liu, “Gabor convolutional networks,” _IEEE Transactions on Image Processing_ , vol. 27, no. 9, pp. 4357–4366, 2018.
  * [11] D. D. Lee and H. S. Seung, “Learning the parts of objects by nonnegative matrix factorization,” _Nature_ , vol. 401, no. 6755, pp. 788–791, 1999\.
  * [12] D. L. Donoho and M. Elad, “Optimally sparse representation in general (nonorthogonal) dictionaries via l 1 minimization,” _Proceedings of the National Academy of Sciences of the United States of America_ , vol. 100, no. 5, pp. 2197–2202, 2003.
  * [13] B. Olshausen and D. Field, “Emergence of simple-cell receptive field properties by learning a sparse code for natural images,” _Nature_ , vol. 381, pp. 607–609, 07 1996.
  * [14] M. Elad and M. Aharon, “Image denoising via learned dictionaries and sparse representation,” in _2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR)_ , vol. 1, 2006, pp. 895–900.
  * [15] J. Mairal, M. Elad, and G. Sapiro, “Sparse representation for color image restoration,” _IEEE Transactions on Image Processing_ , vol. 17, no. 1, pp. 53–69, 2008.
  * [16] Y. Zhou, A. Mao, S. Huo, J. Lei, and S. Kung, “Salient object detection via fuzzy theory and object-level enhancement,” _IEEE Transactions on Multimedia_ , vol. 21, no. 1, pp. 74–85, 2019.
  * [17] G. Ma, C. Chen, S. Li, C. Peng, A. Hao, and H. Qin, “Salient object detection via multiple instance joint re-learning,” _IEEE Transactions on Multimedia_ , vol. 22, no. 2, pp. 324–336, 2020.
  * [18] H. Lee, C. Ekanadham, and A. Ng, “Sparse deep belief net model for visual area v2,” in _Advances in Neural Information Processing Systems (NIPS)_ , vol. 20, 2008, pp. 873–880.
  * [19] P. O. Hoyer, “Non-negative matrix factorization with sparseness constraints,” _Journal of Machine Learning Research_ , vol. 5, no. 3, 2004.
  * [20] “Sparse principal component analysis,” _Journal of Computational and Graphical Statistics_.
  * [21] K. Gregor and Y. Lecun, “Learning fast approximations of sparse coding,” in _2010 International Conference on International Conference on Machine Learning (ICML)_ , 08 2010, pp. 399–406.
  * [22] B. Scholkopf, J. Platt, and T. Hofmann, “Efficient sparse coding algorithms,” in _Advances in Neural Information Processing Systems (NIPS)_ , 2007, pp. 801–808.
  * [23] Y. Wang, H. Xu, and C. Leng, “Provable subspace clustering: When lrr meets ssc,” _IEEE Transactions on Information Theory_ , vol. 65, no. 9, pp. 5406–5432, 2019.
  * [24] V. M. Patel, H. Van Nguyen, and R. Vidal, “Latent space sparse and low-rank subspace clustering,” _IEEE Journal of Selected Topics in Signal Processing_ , vol. 9, no. 4, pp. 691–701, 2015.
  * [25] D. Gabor, “Theory of communication,” _Electrical Engineers Part I General Journal of the Institution_ , vol. 93, pp. 429–459, 01 1946.
  * [26] Baochang Zhang, Yongsheng Gao, Sanqiang Zhao, and Jianzhuang Liu, “Local derivative pattern versus local binary pattern: Face recognition with high-order local pattern descriptor,” _IEEE Transactions on Image Processing_ , vol. 19, no. 2, pp. 533–544, 2010.
  * [27] H. Liu, G. Yang, Z. Wu, and D. Cai, “Constrained concept factorization for image representation,” _IEEE Transactions on Cybernetics_ , vol. 44, no. 7, pp. 1214–1224, 2014.
  * [28] J. Feng, Z. Lin, H. Xu, and S. Yan, “Robust subspace segmentation with block-diagonal prior,” in _2014 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2014, pp. 3818–3825.
  * [29] D. A. V. Dyk and X.-L. Meng, “The art of data augmentation,” _Journal of Computational and Graphical Statistics_ , vol. 10, no. 1, pp. 1–50, 2001.
  * [30] D. Laptev, N. Savinov, J. M. Buhmann, and M. Pollefeys, “Tipooling: transformation-invariant pooling for feature learning in convolutional neural networks,” in _2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2016.
  * [31] Y. Wei, “Deformable convolutional networks,” in _2017 European Conference on Computer Vision (ECCV)_ , 2017.
  * [32] M. Jaderberg, K. Simonyan, A. Zisserman, and K. Kavukcuoglu, “Spatial transformer networks,” in _Advances in Neural Information Processing Systems (NIPS)_ , 2015, pp. 2017–2025.
  * [33] S. Zagoruyko and N. Komodakis, “Wide residual networks,” _arXiv_ , 2016.
  * [34] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)_ , 06 2016, pp. 770–778.
  * [35] C. Lu, J. Feng, S. Yan, and Z. Lin, “A unified alternating direction method of multipliers by majorization minimization,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 40, no. 3, pp. 527–541, 2018\.
  * [36] A. Beck and M. Teboulle, “A fast iterative shrinkage-thresholding algorithm for linear inverse problems,” _SIAM Journal on Imaging Sciences_ , vol. 2, no. 1, pp. 183–202, 2009.
  * [37] J. Cai, E. Candes, and Z.Shen, “A singular value thresholding algorithm for matrix completion,” _SIAM journal on control and optimization_ , vol. 20, no. 4, pp. 1956–1982, 2010.
  * [38] Z. Li, J. Tang, and X. He, “Robust structured nonnegative matrix factorization for image representation,” _IEEE Transactions on Neural Networks and Learning Systems_ , vol. 29, no. 5, pp. 1947–1960, 2018.
  * [39] G. Liu, Z. Lin, S. Yan, J. Sun, Y. Yu, and Y. Ma, “Robust recovery of subspace structures by low-rank representation,” _IEEE Transactions on Pattern Analysis and Machine Intelligence_ , vol. 35, no. 1, pp. 171–184, 2013.
  * [40] M. Lin, Q. Chen, and S. Yan, “Network in network,” in _2014 International Conference on Learning Representations (ICLR)_ , 2014.
  * [41] A. C. A. B. B. W. Y. Netzer, T. Wang and A. Y. Ng, “Reading digits in natural images with unsupervised feature learning,” in _Advances in Neural Information Processing Systems (NIPS)_ , 2011.