# Infrared colour properties of nearby radio-luminous galaxies

Xiao-hong Yang1,3, Pei-sheng Chen2, and Yan Huang1
1 Department of Physics and Institute of Theoretical Physics, Chongqing
University, Chongqing 400044, China
2 National Astronomical Observatories/Yunnan Observatory, Chinese Academy of
Sciences, P.O. Box 110 Kunming, Yunnan Province 650011, China
<EMAIL_ADDRESS>
(Accepted , Received , in original form )

###### Abstract

By combining the data of the Two Micron All Sky Survey, the Wide Field
Infrared Survey Explorer and the AKARI satellite, we study the infrared colour
properties of a sample of 2712 nearby radio-luminous galaxies (RLGs). These
RLGs are divided into radio-loud (RL) active galactic nuclei (AGNs), mainly
occurring at redshifts of $0.05<z<0.3$ and star-forming-dominated RLGs (SFGs),
mainly occurring at redshifts of $0.01<z<0.15$. RL AGNs and SFGs are
separately distributed in the ([3.4]$-$[4.6])$-$([4.6]$-$[12]) two-colour
diagram, in which the RL AGNs display a double-core distribution, and the SFGs
display a single-core distribution. SFGs have a redder [4.6]$-$[12] colour
than RL AGNs due to the significant contribution from the dust component of
SFGs. We find simple criteria of MIR colour separation between RL AGNs and
SFGs such that: 95$\%$ of RL AGNs have [4.6]$-$[12] $<$ 3.0 and 94$\%$ of SFGs
have [4.6]$-$[12] $>$ 3.0. We also analyse the MIR colours of RL AGNs divided
into low- and high-excitation radio galaxies (LERGs and HERGs, respectively).
The ([3.4]$-$[4.6])$-$([4.6]$-$[12]) diagram clearly shows separate
distributions of LERGs and HERGs and a region of overlap, which suggests that
LERGs and HERGs have different MIR properties. LERGs are responsible for the
double-core distribution of RL AGNs on the ([3.4]$-$[4.6])$-$([4.6]$-$[12])
diagram. In addition, we also suggest 90$-$140$\mu$m band spectral index
$\alpha(90,140)<-1.4$ as a criterion of selecting nearby active galaxies with
non-thermal emissions at far-infrared wavelengths.

###### keywords:

surveys-galaxies:active-galaxies:photometry-galaxies:star formation-
infrared:galaxies

††pagerange: Infrared colour properties of nearby radio-luminous
galaxies–LABEL:lastpage††pubyear: 2014

##  1 Introduction

Radio-luminous galaxies (RLGs) form a subclass of active galaxies. The radio
emissions from galaxies are thought to originate from star formation and the
accretion process around supermassive black holes in active galactic nuclei
(AGNs). AGNs emit photons over the entire electromagnetic spectrum and the
radio emissions from AGNs are produced by synchrotron radiation of the
relativistic jets and the lobes formed by interaction of the jets with the
surrounding media (e.g. Antonucci 1984; Urry & Padovani 1995; Cleary et al.
2007). The radio emissions from the star-forming region are believed to
originate from free$-$free emission in the ionized hydrogen (H II) region and
the synchrotron radiation by relativistic electrons in supernova remnants
(Harwit & Pacini 1975). Most active galaxies may contain two components: AGNs
and star-forming regions. For example, NGC 1068 shows an AGN at the centre
surrounded by a ring-shaped star-forming region (Telesco et al. 1984).

Both AGNs and star-forming regions require a large supply of dusts and gases
as fuel. AGNs have different observed subclasses that can be explained by a
unification model containing a supermassive black hole surrounded by an
accretion disc with jets and a torus of dust and gas (Antonucci 1993). The
dusty torus not only supplies the fuel of the accretion disc, but also
reprocesses X-ray emissions from a hot corona above the accretion disc and
then emits infrared radiation. Therefore, the infrared bands are important for
understanding the accretion environment of AGNs. In star-forming regions, dust
and gas clouds are heated by young massive stars and also emit mid-infrared
(MIR) and far-infrared (FIR) radiation.

Active galaxies display various properties at various infrared wavelengths. At
near-infrared (NIR) wavelengths, emission of AGNs is significant, although the
contribution of host galaxies to the spectral energy distribution (SED) cannot
be neglected. For example, the NIR spectra of quasars are roughly represented
by a power-law (PL) continuum superposed by a composite blackbody (BB)
spectrum of the stellar population (Elvis et al. 1994; Richards et al. 2006;
Park et al. 2010). The composite BB spectrum peaks at $\sim 1.6\mu$m.
Therefore, NIR colour is also used to distinguish AGNs from normal stars
(Kouzuma & Yamaoka 2010). At MIR wavelengths, dusts in H II regions (or tori),
polycyclic aromatic hydrocarbons (PAHs) in photodissociation regions and AGNs
comprise potential sources of MIR emission of active galaxies (e.g. Lacy et
al. 2004; Ichikawa et al. 2014). At FIR wavelengths, it is reported to observe
cooler ($\sim 30$ K) interstellar regions heated by stars in host galaxies or
the outer part of the torus in the AGN (e.g. Ichikawa et al. 2012).

In the present study, we summarize previously reported NIR, MIR and FIR
properties of radio galaxies. Strong infrared emission lines have been
commonly detected in high-redshift radio galaxies (Rawlings et al. 1989; Eales
& Rawlings 1993; Iwamuro et al. 1996; Evans 1998). For example, a low-
resolution NIR spectrum (1.14$-$1.70$\mu$m) of 4C 40.36 ($z=2.269$) shows
distinctively strong [O III]$\lambda 4959/5007$ doublet emission lines that
significantly contribute to the $H$-band image (Iwamuro et al. 1996). A
concentric distribution of strong oxygen line-emitting clouds is suggested by
the $H$-band image of 4C 40.36 (Iwamuro et al. 1996).

Benefiting from observations implemented by the Infrared Space Observatory
(ISO) and the Spitzer Space Telescope , many previous reports have focused on
the MIR properties of high-redshift powerful radio galaxies. Siebenmorgen et
al. (2004) used ISOCAM data recorded on board the ISO to construct the SED of
3CR radio galaxies at redshifts of z$\leq$2.5. They determined that the MIR
properties of 3CR radio sources may include synchrotron radiation from AGN,
stars of the host galaxy or dust. Cleary et al. (2007) measured the MIR
radiation from a sample of extremely powerful 3CRR radio galaxies at redshifts
of 0.4$\leq$z$\leq$1.2 based on Spitzer observations. They hypothesized that
Fanaroff$-$Riley type II quasars are more luminous than galaxies in the MIR
range because of a combination of Doppler-boosted synchrotron emissions in
quasars and dust extinction in galaxies, both of which are orientation-
dependent effects. De Breuck et al. (2010) presented a Spitzer survey of 70
radio galaxies at redshifts of 1$<$z$<$5.2 and measured the rest-frame 1.6
$\mu$m emission from the stellar population and the hot dust emission
associated with the active nucleus. Leipski et al. (2010) obtained rest-frame
9$-$16 $\mu$m spectra of 11 quasars and 9 radio galaxies from the 3CRR
catalogue at redshifts of 1$<$z$<$1.4 based on Spitzer observations. They
determined that the mean radio-galaxy spectrum shows a silicate absorption
feature, whereas the mean quasar spectrum shows a silicate emission feature
(Leipski et al. 2010).

Based on Infrared Astronomical Satellite (IRAS) data, 3CR radio sources were
measured at MIR and FIR wavelengths. The results confirm that MIR and FIR
emissions from 3CR quasars and radio galaxies are anisotropic and that the
non-thermal beamed emission of the core component partly contributes to the
FIR emission of the 3CR radio sources (Heckman et al. 1992; Hes et al. 1995).
Dicken et al. (2009) presented the deep MIR (24 $\mu$m) to FIR (70 $\mu$m)
Spitzer photometric observations of a 2Jy sample of powerful radio galaxies
(0.05$<$z$<$0.7). They determined that direct AGN heating is energetically
feasible for the dust producing both MIR and FIR continua (Dicken et al.
2009). Hardcastle et al. (2010) used Herschel-Astrophysical Terahertz Large
Area Survey data to analyse the FIR properties of radio galaxies. They
reported that these properties are similar to those of a comparison population
of radio-quiet galaxies matched in redshift and K-band absolute magnitude.

However, little attention has been paid to the infrared colour of nearby RLGs,
the activities of which are dominated by either AGNs or star formation (Best
et al. 2005; Best & Heckman 2012). This paper will focus on the infrared
colour properties of nearby RLGs, including that reported by Best et al.
(2005) based on sample of 2712 RLGs. The main objective of this study is to
examine the differences in infrared properties of AGN-dominated RLGs and star-
forming region-dominated RLGs by using this large known sample. Additionally,
AGN-dominated RLGs include two different activity modes (Hardcastle et al.
2007; Best & Heckman 2012): high-excitation radio galaxies (HERGs) and low-
excitation radio galaxies (LERGs). HERGs are powered by radiatively efficient
standard thin discs at high accretion rates, whereas LERGs are fuelled at low
accretion rates by radiatively inefficient accretion flows, i.e. advection-
dominated accretion flows (ADAFs; Narayan & Yi 1995; Hardcastle et al. 2007;
Best & Heckman 2012; Yuan & Narayan 2014). Gürkan et al. (2014) analysed the
MIR properties of HERGs and LERGs by using four complete samples (3CRR, 2Jy,
6CE and 7CE). They pointed out that HERGs are separated from LERGs in the MIR-
radio plane (Gürkan et al. 2014). We further analyse the MIR colours of HERGs
and LERGs by using a sample of nearby RLGs.

Throughout this paper, we adopt a flat universe with a Hubble constant
$H_{0}=70\text{ km s}^{-1}\text{Mpc}^{-1}$, $\Omega_{\Lambda}=0.7$ and
$\Omega_{\text{m}}=0.3$.

The paper is organized as follows: Section 2 describes our data; Sections
3$-$5 discuss the colour properties; and Section 6 provides a summary.

Table 1: Infrared photometric data of nearby radio-luminous galaxies (RLGs)

RA(2000) Dec.(2000) | $z$ |  $E$($B-V$) |  $J$ (mag) |  $H$ (mag) |  $K_{s}$ (mag) |  $W1$ (mag) |  $W2$ (mag) |  $W3$ (mag) |  $W4$ (mag) |  $90\mu m$ (Jy) |  $140\mu m$ (Jy)
---|---|---|---|---|---|---|---|---|---|---|---
(1) | (2) | (3) | (4) | (5) | (6) | (7) | (8) | (9) | (10) | (11) | (12)
146.9561 -0.3423 | 0.1347 | 0.1343 | 15.882(0.109) | 15.120(0.160) | 14.526(0.119) | 14.054(0.035) | 13.829(0.044) | 12.040(0.000) | 8.452(0.000) | / | /
146.1436 -0.7416 | 0.2039 | 0.0606 | 15.654(0.115) | 14.982(0.133) | 14.416(0.135) | 13.477(0.025) | 13.217(0.030) | 12.156(0.000) | 9.056(0.000) | / | /
146.7371 -0.2522 | 0.1305 | 0.1327 | 15.482(0.094) | 14.798(0.104) | 14.209(0.093) | 13.505(0.026) | 13.320(0.032) | 12.031(0.274) | 8.996(0.000) | / | /
146.3738 -0.3684 | 0.0529 | 0.1118 | 15.137(0.094) | 14.474(0.130) | 13.948(0.087) | 12.655(0.024) | 12.492(0.025) | 8.622(0.029) | 6.205(0.068) | / | /
145.6012 -0.0014 | 0.1459 | 0.0625 | 15.958(0.115) | 14.970(0.099) | 14.158(0.079) | 13.337(0.024) | 12.710(0.025) | 8.381(0.024) | 6.086(0.053) | 0.5394E+0(0.572E-1) | /
146.4630 0.6387 | 0.0303 | 0.1962 | 15.412(0.143) | 14.725(0.200) | 14.232(0.149) | 12.511(0.023) | 12.313(0.024) | 8.115(0.022) | 5.770(0.044) | 0.4890E+0(0.127E+0) | /
146.8068 0.6656 | 0.0201 | 0.1484 | 13.764(0.099) | 12.936(0.125) | 12.150(0.064) | 10.861(0.022) | 10.639(0.020) | 6.588(0.014) | 4.592(0.022) | 0.2862E+1(0.192E+0) | 0.4211E+1(0.174E+0)
146.7991 0.7027 | 0.0305 | 0.1495 | 15.785(0.154) | 15.201(0.177) | 14.700(0.167) | 13.204(0.024) | 12.950(0.028) | 9.005(0.033) | 6.099(0.058) | 0.5410E+0(0.374E-1) | 0.2857E+1(0.519E+0)
146.7815 0.7380 | 0.2618 | 0.1485 | 16.337(0.172) | 15.473(0.174) | 14.664(0.144) | 14.134(0.042) | 13.850(0.054) | 11.885(0.000) | 8.268(0.000) | / | /
147.0805 0.7880 | 0.2111 | 0.1039 | 15.820(0.097) | 14.956(0.109) | 14.284(0.097) | 13.734(0.026) | 13.446(0.035) | 11.844(0.000) | 8.758(0.000) | / | /
149.1699 -0.0233 | 0.1392 | 0.0299 | 15.396(0.090) | 14.640(0.107) | 14.175(0.086) | 13.305(0.025) | 13.108(0.029) | 11.888(0.268) | 8.592(0.000) | / | /
148.4325 -1.0264 | 0.1103 | 0.0426 | 15.413(0.086) | 14.569(0.107) | 13.851(0.074) | 12.767(0.023) | 11.460(0.022) | 8.173(0.022) | 5.372(0.036) | 0.5702E+0(0.239E-1) | /
148.2377 -0.7920 | 0.0898 | 0.0334 | 14.466(0.073) | 13.847(0.094) | 13.373(0.077) | 12.570(0.023) | 12.507(0.025) | 11.690(0.219) | 8.416(0.000) | / | /
147.7086 -0.8878 | 0.2715 | 0.0549 | 16.059(0.113) | 15.369(0.145) | 14.596(0.138) | 13.881(0.026) | 13.544(0.032) | 12.440(0.490) | 8.950(0.000) | / | /
147.4283 -0.8401 | 0.0809 | 0.0651 | 15.132(0.080) | 14.426(0.082) | 13.978(0.076) | 13.298(0.025) | 13.016(0.028) | 11.140(0.131) | 8.647(0.000) | / | /

  * Col. 1: coordinates in the epoch of 2000 from Best et al. (2005); Col. 2: redshifts of sources; Col. 3: derived $E$($B-V$); Cols. 4$-$10: magnitudes at $J$, $H$, $K_{s}$, $W$1, $W$2, $W$3 and $W$4 bands, respectively, with their uncertainty given by the values in parentheses; Cols. 11 and 12: fluxes in units of Jansky at 90 and 140$\mu$m bands, respectively, with their uncertainty given by the values in parentheses. When the value in parentheses is zero, the listed magnitudes are the upper limits. We list only the sources in which 90- and 140-$\mu$m fluxes have high quality such that the source is confirmed and flux is reliable. Only the first 15 sources are listed here; the full table is available in the electronic version.

Figure 1: Redshift distribution of all sources studied. Black line represents
radio-loud AGNs (RL AGNs); red line represents star-forming galaxies (SFGs).

(A color version of this figure is available in the online journal.)

##  2 Data

###  2.1 Sample

All sources studied here were selected from a catalogue of 2712 RLGs (Best et
al. 2005). This large sample of RLGs was derived by Best et al. (2005), who
used the second data release of the Sloan Digital Sky Survey (York et al.
2000; Stoughton et al. 2002) to cross-correlate with combined data of the
National Radio Astronomy Observatories Very Large Array Sky Survey (Condon et
al. 1998) and the Faint Images of the Radio Sky at 20 cm (Becker et al. 1995)
survey. Both radio surveys achieved higher angular resolutions than those
recorded by their predecessors and covered substantial fractions of the sky,
down to flux densities at the millijansky scale. To remove the very extended
galaxies, Best et al. (2005) excluded galaxies with redshifts of $z<0.01$; the
2712 RLGs are located at redshifts of $0.01<z<0.56$. These samples are from
the SDSS main galaxy catalogue, which means that objects classified as quasars
by the automated SDSS classification pipeline are excluded (Best et al. 2005).

Table 2: Number of detections and upper limits for low-excitation radio
galaxies (LERGs), high-excitation radio galaxies (HERGs) and star-forming
galaxies (SFGs).

Subsample |  $W$1(d) |  $W$1(u) |  $W$2(d) |  $W$2(u) |  $W$3(d) |  $W$3(u) |  $W$4(d) |  $W$4(u)
---|---|---|---|---|---|---|---|---
LERGs | 1745 | 3 | 1748 | 0 | 1032 | 716 | 220 | 1528
HERGs | 52 | 0 | 52 | 0 | 45 | 7 | 32 | 20
SFGs | 502 | 0 | 502 | 0 | 501 | 1 | 490 | 12

  * Cols 2$-$3, 4$-$5, 6$-$7 and 8$-$9 give the number of detections and upper limits for each class at four bands of $W$1, $W$2, $W$3 and $W$4, respectively. The ‘d’ in parenthesis indicates the number of detections, whereas the ‘u’ indicates the number of upper limits.

###  2.2 Cross-identification

In recent years, many infrared surveys have been completed by various
instruments at various infrared bands. The Two Micron All Sky Survey (2MASS)
has swept 99.998$\%$ of the whole sky at the $J$ (1.25$\mu$m), $H$
(1.65$\mu$m) and $K_{s}$ (2.16$\mu$m) bands (Skrutskie et al. 2006). The Wide-
Field Infrared Survey Explorer (WISE) has completed an all-sky survey at the
$W$1 (3.4$\mu$m), $W$2 (4.6$\mu$m), $W$3 (12$\mu$m) and $W$4 (22$\mu$m) bands
and issued two public data releases in April 2011 and March 2012. The WISE
achieves 5$\sigma$ point-source sensitivities better than 0.08, 0.11, 1 and 6
mJy in the four bands, corresponding to Vega magnitudes of 16.5, 15.5, 11.2
and 7.9, respectively; the sensitivity of the WISE $W$3 (12$\mu$m) band is
more than 100 times better than that of IRAS (Wright et al. 2010). The AKARI
(Ishihara et al. 2010), a Japanese infrared space mission, has carried MIR to
FIR surveys at six bands centred at 9, 18, 65, 90, 140 and 160 $\mu$m by using
two instruments: an infrared camera (Onaka et al. 2007) and a FIR surveyor
(FIS; Kawada et al. 2007). The AKARI achieves higher sensitivity, higher
spatial resolution and wider wavelength coverage than the IRAS.

To obtain the NIR, MIR and FIR photometric data of the studied radio sources,
we primarily used the point source catalogues (PSCs) of 2MASS, WISE and AKARI
(Skrutskie et al. 2006; Ishihara et al. 2010; Yamamura et al. 2010). The
catalogue of the 2712 RLGs is cross-identified with the 2MASS, WISE and AKARI-
FIS PSCs. Most 2MASS PSC sources have spatial resolutions of less than $\sim$4
arcsec in all of the three wavelength bands (see 2MASS
homepage111http://www.ipac.caltech.edu/2mass/). The spatial resolutions of
WISE are 6.1, 6.4, 6.5 and 12.0 arcsec at $W$1, $W$2, $W$3 and $W$4,
respectively (Wright et al. 2010). The spatial resolutions of AKARI are 7 and
48 arcsec for the MIR and FIR instruments, respectively (e.g. Matsuta et al.
2012). We used position offsets of 4, 6 and 20 arcsec to search the 2MASS,
WISE and AKARI-FIS PSC counterparts, respectively, for the 2712 RLGs. If more
than one object was within the position offset radius, the closest object was
accepted as the counterpart. In practice, only a small number of RLGs fell
into the category. 96$\%$ of the 2MASS PSC counterparts and 90$\%$ of the WISE
PSC counterparts were within a 1 arcsec radius of the RLGs; 91$\%$ of the
AKARI-FIS counterparts were within a 10 arcsec radius.

Finally, we excluded RLGs without 2MASS counterparts and obtained a sample of
2663 RLGs. About 42$\%$ of the studied samples were consistent with a point
source. Figure 1 shows the redshift distribution of two types of RLGs: radio-
loud (RL) AGNs and star-forming galaxies (SFGs). The classification of RLGs is
described in section 2.4. The RL AGNs studied are located mainly at redshifts
of $0.05<z<0.3$, whereas the SFGs studied are located mainly at redshifts of
$0.01<z<0.15$. The infrared photometric data of the 2663 RLGs are listed in
Table 1, in which Col. 1 gives the coordinates (in the epoch of 2000) from
Best et al. (2005); Col. 2 gives the redshifts of the sources; Col. 3 gives
the derived E(B$-$V); Cols. 4$-$10 give the vega magnitudes at the $J$, $H$,
$K_{s}$, $W$1, $W$2, $W$3 and $W$4 bands, respectively, with their
uncertainties given by the values in parentheses (Skrutskie et al. 2006;
Ishihara et al. 2010); Cols. 11 and 12 give the flux values in units of
Janskies at the 90 and 140$\mu$m bands, respectively, with their uncertainties
given by the values in parentheses (Yamamura et al. 2010). Only the first 15
sources are listed here; the full table is available in the electronic version
of this paper. Table 2 also gives number of detections and upper limits for
LERGs, HERGs and SFGs.

###  2.3 Galactic interstellar extinction correction

The interstellar extinction corrections caused by our Galaxy, $E$($B-V$), are
considered in five bands: $J$, $H$, $K_{s}$, $W$1 and $W$2\. We obtain the
values of $E$($B-V$) for all samples through the NASA/IPAC extragalactic
database (NED Home Page222http://irsa.ipac.caltech.edu/applications/DUST/),
which gives the Galactic dust reddening for a line of sight based on Schlegel,
Finkbeiner and Davis (1998). The extinction of a given wavelength band $a$ is
usually estimated by $A(a)=R(a)\times E(B-V)$. Yuan et al. (2013) reported new
empirical extinction coefficients $R(a)$. According to Table 2 of Yuan et al.
(2013), we employed $R(a)=0.72$, $0.46$, $0.306$, $0.19$, and $0.15$ for the
bands $J$, $H$, $K_{s}$, $W$1 and $W$2, respectively, to correct the
magnitudes of $J$, $H$, $K_{s}$, $W$1 and $W$2\. In fact, the extinctions were
smaller than the uncertainties of the measured magnitudes for the majority of
the sources. Therefore, we applied interstellar extinction corrections for
only about 2 % of our samples.

###  2.4 Classification

At NIR wavelengths, thermal emissions from host galaxies and non-thermal
emissions from the central nuclei were observed. The orientation of AGNs may
affect non-thermal emissions from the central nuclei. Therefore, to
investigate the effect of orientation, we used the 13th version of Quasars and
AGNs (Véron-Cetty & Véron 2010) to search the spectral types of our sample and
obtain the sources sub-classified as Seyfert 1 (Sy1) and Setfert 2 (Sy2)
galaxies.

At MIR and FIR wavelengths, thermal emissions from dust play a very important
role in RLGs. To further investigate the MIR properties of different active
types of RLGs, we also distinguished the active types of RLGs according to
Best et al. (2005) and Best & Heckman (2012). Based on the location of a
galaxy in a plane of 4000-Å break strength ($D_{n}(4000)$) versus radio
luminosity per unit stellar mass ($L_{1.4GHz}/M_{*}$), the RLGs were divided
into RL AGNs and SFGs (Best et al. 2005). The radio emissions of RL AGNs are
mainly generated by jets or lobes, whereas those of SFGs are dominated by star
formation. The majority of the samples used by Best et al. (2005) are also
included in Best & Heckman (2012), although a small number of samples were
inconsistently classified. For the samples overlapping both works, we employed
the classification from Best & Heckman (2012). Best & Heckman (2012) further
divided RL AGNs into HERGs and LERGs; we employed this classification as well.
HERGs and LERGs are believed to belong to different accretion modes, i.e.
standard thin discs and ADAFs, respectively (Hardcastle, Evans & Croston
2007). HERGs have higher accretion rates than LERGs (Best & Heckman 2012).

##  3 NIR properties

Figure 2 shows a $(J-H)-(H-K_{s})$ two-colour diagram (top panel) and the
normalized distributions of the colours $H-K_{s}$ (middle panel) and $J-H$
(bottom panel). In order to easily understand the spectral shapes of the RLGs,
the colours were also transformed to their corresponding spectral indices
$\alpha(\lambda_{1},\lambda_{2})$. Following Sekiguchi (1987), the spectral
index is defined as,

$\alpha(\lambda_{1},\lambda_{2})=\text{log}(F_{\lambda_{1}}/F_{\lambda_{2}})/\text{log}(\lambda_{2}/\lambda_{1}),$
(1)

where $F_{\lambda_{1}}$ and $F_{\lambda_{2}}$ are the measured flux densities
at wavelengths $\lambda_{1}$ and $\lambda_{2}$ ($\lambda_{2}>\lambda_{1}$),
respectively. It is noted that the flux densities are expressed throughout
this paper by using frequency units. The corresponding spectral indices of the
colours $J-H$ and $H-K_{s}$ are also marked in the top panel of Figure 2. In
the two-colour diagram, a PL ( $F_{\lambda}\propto\lambda^{-\alpha}$) line
(dashed line) and a BB line (dotted line) are also shown. The PL line can be
obtained by calculating the change in colour with the spectral index
($\alpha$). For a PL spectrum, the colour in a certain wavelength range
$\Delta\lambda=\lambda_{2}-\lambda_{1}$ is derived by

$m_{\lambda_{1}}-m_{\lambda_{2}}=-2.5\text{
log}[(\frac{\lambda_{1}}{\lambda_{2}})^{-\alpha}(\frac{F_{0\lambda_{2}}}{F_{0\lambda_{1}}})],$
(2)

where $m_{\lambda_{1}}$ and $m_{\lambda_{2}}$ are magnitudes at wavelengths
$\lambda_{1}$ and $\lambda_{2}$, respectively, and $F_{0\lambda_{1}}$ and
$F_{0\lambda_{2}}$ are absolute flux calibrations at wavelengths $\lambda_{1}$
and $\lambda_{2}$, respectively. Similarly, we calculate the change in colour
with temperature ($T$) to draw the BB line. For a BB spectrum, the colour is
derived by

$m_{\lambda_{1}}-m_{\lambda_{2}}=-2.5\text{
log}[(\frac{B_{\lambda_{1}}}{B_{\lambda_{2}}})(\frac{F_{0\lambda_{2}}}{F_{0\lambda_{1}}})],$
(3)

where $B_{\lambda_{1}}$ and $B_{\lambda_{2}}$ are the Planck function with
same temperature at wavelengths $\lambda_{1}$ and $\lambda_{2}$, respectively.

Figure 2: ($J-H$)$-$($H-K_{s}$) two-colour diagram (top panel) and the
normalized distribution of the colour $H-K_{s}$ (middle panel) and J-H (bottom
panel). The power-law spectral indices $\alpha(J,H)$ and $\alpha(H,K_{s})$ are
also marked in the top panel. A BB line with temperatures and a PL line with
spectral indices are plotted as the dotted and dashed line, respectively. In
the middle and bottom panels, black and red lines represent RL AGNs and SFGs,
respectively.

In addition, following Kouzuma & Yamaoka (2010), the top panel of Figure 2
shows the intrinsic loci of dwarf stars (magenta line), giant stars (red line)
and Classical T Tauri Stars (CTTS; blue line) according to Bessell & Brett
(1988) and Meyer et al. (1997). The reddening vector from Rieke & Lebofsky
(1985), which indicates the shifting direction of the stellar and CTTS loci on
the $(J-H)-(H-K_{s})$ diagram, is also plotted. The infrared colour of the
RLGs is different from the colours of various types of stars. Kouzuma &
Yamaoka (2010) proposed the following NIR colour criteria for selecting AGNs:

$(J-H)\leq 1.7(H-K_{s})-0.118,$ (4) $(J-H)\leq 0.61(H-K_{s})+0.5.$ (5)

Equations 4 and 5 are also indicated in the two-colour diagram by dot$-$dashed
and double-dot-dashed lines, respectively. 64$\%$ of RL AGNs and 48$\%$ of
SFGs achieved these selection criteria, which is a lower percentage than that
given by Kouzuma & Yamaoka (2010) for low-redshift AGNs. It is noted that the
sample used in the present study was located at redshifts of 0.01$<z<$0.56
where the NIR radiation from the centre of the AGN should be contaminated by
host galaxies with intrinsic colours similar to those of normal stars. On the
contrary, the majority of the samples studied by Kouzuma & Yamaoka were
quasars that had more luminous cores than those of our sample. Hence, the NIR
selection was less effective for the RL AGNs than for other types of AGNs.

It should be noted that most of the sources achieving the selection criteria
were distributed in the region around the PL line, which indicates that their
NIR spectra can be approximately described by a PL spectrum. The top panel of
Figure 2 also shows that the majority of the RL AGNs and SFGs had indices of
$-2<\alpha(H,K_{s})<1$ and $-2<\alpha(J,H)<0.4$, which is less than the value
of 4 supposed by the Rayleigh$-$Jeans spectra. AGNs produced emissions at
nearly all wavelengths, and their emissions were dominated by non-thermal
radiation. The ultraviolet to MIR continuum of AGNs is represented by a PL
(Stern et al. 2005). However, host galaxies may contaminate the NIR colour of
AGNs. The spectra from the star population in the host galaxy were determined
by using the BB spectra and peak at approximately 1.6 $\mu$m (Elvis et al.
1994; Stern et al. 2005; Richards et al. 2006; Park et al. 2010). For example,
Dultzin-Hacyan & Benitz (1994) pointed out that the dominant NIR emissions of
Sy2 galaxies are primarily produced by evolved massive red giants and
supergiants in the host galaxies; De Breuck et al. (2010) used the spectrum of
a stellar population as a component in fitting the NIR and MIR spectra of
high-redshift radio galaxies. To study the degree in which objects with non-
thermal emissions and stellar populations affect the NIR colour of active
galaxies, Kouzuma & Yamaoka (2010) first examined the distribution of four
types of objects in the $(J-H)-(H-K_{s})$ diagram including microquasar
candidates, low-mass X-ray binaries, cataclysmic variables and massive young
stellar objects and found that most of the objects were not close to the PL
line and did not achieve the NIR colour criteria of AGNs. Based on these
results, they determined that the dominant NIR radiation of the four objects
was likely thermal emission. They further examined the loci of normal galaxies
without considering the contribution from the nuclei in the $(J-H)-(H-K_{s})$
diagram and concluded that the normal galaxies with $0.6<\textit{z}<0.8$ have
intrinsic colours similar to those of normal stars, whereas galaxies with
$\textit{z}\sim 1$ achieve the NIR colour criteria of AGNs. Therefore, the
probable objects with non-thermal radiation and the stellar populations in the
host galaxies cannot cause active galaxies with $\textit{z}\lesssim 0.8$ to
approach the PL line. Thus, if active galaxies with $\textit{z}\lesssim 0.8$
are close to the PL line in the $(J-H)-(H-K_{s})$ diagram, their NIR emissions
should be dominated by non-thermal emissions associated with the nuclei. For
the SFGs achieving the selection criteria (48$\%$ of SFGs), the non-thermal
emissions from the nuclei dominate their NIR emissions. It should also be
noted that a great number of sources are distributed in the region around the
BB line, which indicates that their NIR emissions may be dominated by thermal
radiation because the BB line is considered as an indicator of thermal
radiation from the starlight in the host galaxy. The contribution ratio of the
centre nuclei and their host galaxies to NIR emissions determines the position
of active galaxies in the $(J-H)-(H-K_{s})$ diagram.

Figure 3: ([3.4]$-$[4.6])$-$([4.6]$-$[12]) (top panel) and
([12]$-$[22])-([4.6]$-$[12]) (bottom panel) two-colour diagrams. A PL line
marked with spectral indices and a BB line are also plotted by dashed and
dotted lines, respectively. The upper limits are indicated as arrows.

Figure 4: Probability distribution of LERGs and SFGs. Grey-scale and 30
isodensity contour lines (black solid lines: LERGs; black dotted lines: SFGs)
indicate the probability distribution of LERGs and SFGs. HERGs are
superimposed as red circles in this figure for comparison. A PL line marked
with different spectral indices and a BB line are also plotted with dashed and
dotted lines, respectively. Blue solid lines indicate the loci of models with
$T=3500K$ and $\alpha=-2$, $-3$ and $-4$. Red solid lines indicate the loci of
models with $T=5500K$ and $\alpha=-2$, $-3$ and $-4$. Numbers along the loci
are the fraction of contribution of the PL component to the $W$2-band flux.
Objects with upper limits are excluded.

Figure 5: Normalized histograms of three colours of HERGs (red) and LERGs
(black). Top panel is for colour $[3.4]-[4.5]$, middle panel is for colour
$[4.6]-[12]$, and bottom panel is for colour $[12]-[22]$. The objects with
upper limits are excluded.

Chen & Shan (2009) also reported that almost all type 1 quasars are
distributed in the region around the PL line in the $(J-H)-(H-K_{s})$ diagram
due to their predominantly non-thermal properties, whereas the majority of Sy2
galaxies are distributed in the region around the BB line due to the obscuring
of the non-thermal component of AGNs by the dust torus. Here the
$(J-H)-(H-K_{s})$ diagram shows that many Sy1 RLGs tend to lie close to the BB
line. The difference in distribution of Sy1 RLGs and type 1 quasars occurs
because that type 1 quasars have more luminous nuclei; therefore, the NIR
emissions from the nuclei are significantly stronger than those from their
host galaxies. It is noted that a great number of Sy2 RLGs are distributed in
the region around the PL line. A similar case also exists for type 2 quasars
(Chen & Shan 2009). Chen & Shan (2009) determined that about 43$\%$ of type 2
quasars are distributed in the region around the BB line, whereas about 57$\%$
of type 2 quasars are distributed near the PL line. They cited the possibility
that some NIR emissions from the central nuclei can penetrate the dust torus
because it is not substantially thick and the viewing angle is not very large.
Another possible reason is that the torus is clumpy; therefore, the radiation
from the central nuclei can penetrate the clumpy torus at a certain angle.

The middle and bottom panels of Figure 2 show the normalized histograms of the
colours $J-H$ and $H-K_{s}$ of RL AGNs and SFGs, respectively. In addition, we
performed a Kolmogorov-Smirnov (K-S) test to evaluate the similarity in the
distributions of these colours for RL AGNs and SFGs. We determined that the
similarity of $J-H$ has a significance level of less than 5$\%$, whereas that
of $H-K_{s}$ is larger than 5$\%$. These results indicate that the
distributions of $J-H$ for RL AGNs and SFGs are essentially different, whereas
those of $H-K_{s}$ are essentially the same. Thus, a systematic difference
exists in J-band emissions of RL AGNs and SFGs, the reason for which needs to
be investigated.

##  4 MIR colour properties

###  4.1 Colour properties

Figure 3 shows $([3.4]-[4.6])-([4.6]-[12])$ and $([12]-[22])-([4.6]-[12])$
two-colour diagrams. In each diagram, a PL line and a BB line are plotted with
a dashed and dotted line, respectively.

From Figure 3, we can see that RL AGNs and SFGs are clustered in distinctly
separate regions. Compared to the RL AGNs, the SFGs have a redder $[4.6]-[12]$
colour and a slightly redder [3.4]-[4.6] colour. The PL spectral index
$\alpha(3.4,4.6)$ of the RL AGNs changes from approximately 0 to 2,
$\alpha(4.6,12)$ changes from approximately $-2$ to 2 and $\alpha(12,22)$
changes from approximately $-4$ to 0. This suggests that their observed
spectra behave differently within the wavelength range of 4.6$-$22$\mu$m. One
type is $\alpha(4.6,12)<0$ and $\alpha(12,22)<0$, showing an increase of
$F_{\lambda}$ with $\lambda$ from 4.6 to 22 $\mu$m, and the other type is
$\alpha(4.6,12)>0$ and $\alpha(12,22)<0$, showing a ‘turning point’ in the
spectra, i.e. $F_{\lambda}$ decreases with $\lambda$ from 3.4 to 12$\mu$m
while $F_{\lambda}$ increases with $\lambda$ from 12 to 22$\mu$m. However, the
SFGs show red spectra, i.e. $F_{\lambda}$ increases with $\lambda$ from 4.6 to
22$\mu$m.

To understand the MIR colours of the RLGs, a BB component and a PL component
are used. The BB component roughly models the component of the stellar
population, whereas the PL component represents the contributions of multi-
temperature hot dusts or non-thermal emissions associated with the centre
nuclei. The ultraviolet to MIR continuum of AGNs is described by a PL (Stern
et al. 2005). The thermal emissions of multi-temperature hot dusts are also
described by a PL, as seen in Figure 3 of Brandl et al. (2006). For example,
Smith et al. (2007) used the modified blackbodies at fixed temperatures (35,
40, 50, 65, 90, 135, 200 and 300 K) as thermal dust continuum components to
decompose the MIR spectra of SFGs. The two components are composed according
to the fraction of their contribution to the $W$2-band flux, so we can
calculate the flux densities of the composed model at different wavelengths by

$F_{\lambda}=B_{\lambda}(T)+A_{1}\times\lambda^{-\alpha}.$ (6)

Here, $B_{\lambda}(\text{T})$ represents the Planck function of a BB with
temperature $T$ and
$A_{1}=\frac{\gamma_{1}}{1-\gamma_{1}}\frac{B_{\lambda_{4.6\mu
m}}(T)}{\lambda_{4.6\mu m}^{-\alpha}}$, where $\gamma_{1}$ is the ratio of the
radiation flux density of the PL component to the total flux density at the
$W$2 band. For given values of temperature $T$ and spectral index $\alpha$, we
can obtain the dependence of the colour of the composed model on $\gamma_{1}$
and plot the loci of the composed model with blue and red lines, as in Figure
4. Here, for simplicity, we assign the BB temperatures $T$ = 3500 and 5500 K,
according to the later stellar effective temperature. We use different
spectral indices $\alpha$ to model the potential energy distributions of
emissions from multi-temperature dusts or non-thermal emissions associated
with the centre nuclei. From the ([12]-[22])$-$([4.6]-[12]) diagram, we can
see that the spectral indices $\alpha(12,22)$ of few sources are less than -4,
so we set $\alpha\geq-4$, such as $\alpha=-2$, $-3$, and $-4$, respectively.
Numbers along the loci are the fraction of the contribution of the PL
component to the $W$2-band flux. Figure 4 also shows the probability
distribution of LERGs and SFGs with grey-scale and 30 isodensity contour lines
(black solid lines for LERGs and black dotted lines for SFGs). The probability
distribution is evaluated using the kernel density estimation technique, which
is an effective method of estimating the probability function of a
multivariate variable without any assumption. For comparison, we also
superimpose HERGs as red circles in Figure 4. From Figure 4, we can see that:
the smaller the spectral index $\alpha$, the redder the colour [4.6]-[12]; the
larger the contribution of the PL component to the W2-band flux, the closer to
the PL line the distribution of the RLGs in the ([4.6]-[12])-([12]-[22])
diagram is. Although our parameters cannot cover the distribution of all the
RLGs, the composed model helps us to understand the role of the BB and PL
components in the MIR colours of RLGs.

Figure 4 also shows that LERGs have a distribution of a double-core structure,
which means that LERGs do not have singular infrared properties. The main
difference between the two cores is their different $[3.4]-[4.6]$ colour. More
spectral data are required to understand this difference. It is noted that the
$W$1 band covers the PAH emission at 3.3 $\mu$m and the $W$3 band covers the
9.8-$\mu$m silicate absorption and the 11.3-$\mu$m PAH emission, while there
are no PAH emissions in the $W$2 and $W$4 bands (e.g. Brandl et al. 2006;
Smith et al. 2007). Therefore, many LERGs display a bluer $[3.4]-[4.6]$ colour
than the red line in Figure 4, due to the 3.3-$\mu$m PAH emission.

Figure 6: Plots of the logarithms of 4.6-$\mu$m luminosity versus the colour
$[3.4]-[4.6]$ (top panel), the logarithms of 12-$\mu$m luminosity versus the
colour $[4.6]-[12]$ (middle panel), and the logarithms of 22-$\mu$m luminosity
versus the colour $[12]-[22]$ (middle panel).

Similarly composed models are used to mimic the distribution of most RL AGNs
and SFGs in the $([12]-[22])-([22]-[12])$ diagram. Although the
$([3.4]-[4.6])-([4.6]-[12])$ and $([12]-[22])-([22]-[12])$ diagrams cannot
definitively confirm the nature of thermal or non-thermal emissions of RL AGNs
and SFGs at MIR wavelengths, they help us understand the spectral shapes of RL
AGNs and SFGs. For SFGs, we presume that multi-temperature hot dusts dominate
MIR emissions, as what Brandl et al. (2006) pointed out. The
$([12]-[22])-([22]-[12])$ diagram also shows that the $[12]-[22]$ colour of RL
AGNs is more widely distributed than that of the SFGs, which suggests that the
MIR radiation components of RL AGNs is more diverse than that of SFGs. MIR
radiation of RL AGNs may include the non-thermal emissions associated with
AGNs and the thermal emissions from torus or host galaxies. The two
responsible factors need be further quantitatively studied for RL AGNs. Stern
et al. (2012) and Assef et al. (2013) presented a simple MIR colour criterion
of selecting AGNs $[3.4]-[4.6]$ $>$ 0.8. In our studied samples, 0.06$\%$ of
RL ANGs (14 RL AGNs) and 10$\%$ of SFGs (50 SFGs) achieve this criterion.

In addition, from the $([3.4]-[4.6])-([4.6]-[12])$ and
$([12]-[22])-([4.6]-[12])$ diagrams, we can see the obvious difference in the
distribution of LERGs and HERGs, though LERGs and HERGs partly overlap. To
show the difference, we plot normalized histograms of three MIR colours
($[3.4]-[4.6]$, $[4.6]-[12]$, and $[12]-[22]$) of LERGs and HERGs in Figure 5.
This plot shows that HERGs have redder $[3.4]-[4.6]$ and $[4.6]-[12]$ colours
on average. LERGs are mainly located in the region of $[3.4]-[4.6]$ $<$ 0.4
and $[4.6]-[12]$ $<$ 0.8, whereas a large number of HERGs have $[3.4]-[4.6]$
$>$ 0.4 and $[4.6]-[12]$ $>$ 0.8. In particular, few HERGs have $[3.4]-[4.6]$
$<$ 0.1 and $[4.6]-[12]$ $<$ 0.8. HERGs have a narrower colour [12]-[22]
distribution than LERGs. Figures 3 and 4 show that HERGs are distributed in
the region where MIR emissions are dominated by the power-law component.
Figures 4 also shows that a large number of LERGs are distributed in the
region dominated by the steller population and a small number of LERGs have
redder $[4.6]-[12]$ colour. The difference between LERGs and HERGs may be
concerned with their different accretion modes. It is suggested that HERGs and
LERGs have different accretion modes (Hardcastle et al. 2007). LERGs are
suggested to be fuelled via ADAFs (hot-mode accretion) with low accretion
rates, whereas HERGs are associated with a standard thin accretion disc (cold-
mode accretion) at high accretion rates (Shakura & Sunyaev 1973; Hardcastle et
al. 2007; Best & Heckman 2012). HERGs with high accretion rate have luminous
core, while LERGs with low accretion rate have dim core. Therefore, the MIR
emissions of HERGs are dominated by non-thermal emissions associated with
AGNs; the MIR emissions from central core of LERGs are significantly
contaminated by stars and dusts of host galaxies. Accretion rates play an
important role in the switch between hot- and cold-mode accretions (Esin et
al. 1997). However, accretion rates are also modulated by feedback from
central AGNs, and different accretion modes of the central AGNs have different
effects on this feedback (Ciotti & Ostriker 2001; Gan et al. 2014).
Determining what the key of switch is between hot- and cold-mode accretions is
still an open issue.

Table 3: Results of partial correlation analyses between luminosities and
colours.

Abscissa | Ordinate | Subsample | Number | $\rho_{.z}$ | $\tau/\sigma$
---|---|---|---|---|---
$[3.4]-[4.6]$ | $\text{log}_{10}(\lambda L_{4.6\mu m})$ | LERGs | 1748 | 0.459 | 8.80
|  | HERGs | 52 | 0.585 | 6.50
|  | SFGs | 502 | 0.460 | 11.94
$[4.6]-[12]$ | $\text{log}_{10}(\lambda L_{12\mu m})$ | LERGs | 1748 | 0.284 | 27.80
|  | HERGs | 52 | 0.617 | 13.0
|  | SFGs | 502 | 0.197 | 11.39
$[12]-[22]$ | $\text{log}_{10}(\lambda L_{22\mu m})$ | LERGs | 1748 | 0.027 | 7.90
|  | HERGs | 52 | 0.179 | 3.35
|  | SFGs | 502 | 0.388 | 17.29

  * Notes. $\rho_{.z}$ is Kendall’s partial correlation coefficient with its associated null probability. $\tau/\sigma>3$ means that the correlation is significant at the $3\sigma$ level.

###  4.2 Luminosity versus colour

Here, we check possible correlations between MIR luminosities and colours for
different types of RLGs. We calculate the luminosity distance ($D_{\text{L}}$)
by using the redshift listed in Table 1 (cosmology parameters are assumed in
Section 1) and then obtain the absolute luminosities. The K correction may
become important for high-redshift objects. We apply the K correction to our
sample. For simplicity, MIR spectra of our sample are approximated by simple
PL form ($\propto\nu^{\alpha}$). The intrinsic luminosities $L(\nu)$ are
calculated as follows: $L(\nu)=4\pi D_{\text{L}}^{2}f(\nu)(1+z)^{-\alpha-1}$
(e.g. Bourne et al. 2011 ), where $f(\nu)$ is the observed flux density at the
observing frequency $\nu$. Here, $\alpha$ can be calculated by using Equation
1. In Figure 6, we plot the logarithms of 4.6-$\mu$m, 12-$\mu$m, and 22-$\mu$m
luminosities versus the $[3.4]-[4.6]$, $[4.6]-[12]$, and $[12]-[22]$ colours,
respectively. We can see that the 12-$\mu$m and 22-$\mu$m luminosities of
HERGs are on average higher than those of LERGs. Similar results are also
shown in Figure 7 of Gürkan et al. (2014). Gürkan et al. (2014) remarked that
a division of LERGs and HERGs clearly stands out in the 22-$\mu$m luminosity.
However, based on our studied sample, HERGs cannot be separated from LERGs
using the 22-$\mu$m luminosity.

In order to test physical correlations between luminosities and colours, we
further calculate Kendall’s partial correlation coefficient ($\rho._{z}$) with
censored data using the methods by Akritas & Siebert (1996). $\rho._{z}$
measures the degree of association between two random variables, with the
effect of a set of controlling random variables removed. $\rho._{z}$ is
especially important for excluding the effect of redshift. The values of
$\rho._{z}$ are within $-1$ and 1. The larger their absolute values, the
stronger the correlations. The positive values indicate the positive
correlation and the negative values indicate the negative correlation. Table 3
gives results of partial correlation analyses. For LERGs, a physical
relationship presents between $\text{log}_{10}(\lambda L_{4.6\mu m})$ and
$[3.4]-[4.6]$ and the correlation of luminosities vs. colours becomes weak
with increasing wavelength. As shown in the bottom panel of Figure 6, no
correlation between $\text{log}_{10}(\lambda L_{22\mu m})$ versus $[12]-[22]$
is seen for LERGs. For HERGs, good physical correlations exist between
$\text{log}_{10}(\lambda L_{4.6\mu m})$ versus $[3.4]-[4.6]$ and between
$\text{log}_{10}(\lambda L_{12\mu m})$ versus $[4.6]-[12]$, but the
correlation of $\text{log}_{10}(\lambda L_{22\mu m})$ versus $[12]-[22]$ is
weaker. For SFGs, weak correlations present between $\text{log}_{10}(\lambda
L_{4.6\mu m})$ versus $[3.4]-[4.6]$ and between $\text{log}_{10}(\lambda
L_{22\mu m})$ versus $[12]-[22]$.

Figure 7: Histogram of colour $[4.6]-[12]$ of RL AGNs (black dotted line) and
SFGs (red dashed line). The objects with upper limits are excluded.

###  4.3 Distinguishing RL AGNs and SFGs by MIR colours

The activity in galaxies is thought to be influenced by two components, star
formation and AGNs. Some galaxies are dominated by either star formation or
the AGN, while others exhibit comparable contributions from both components.
The AGN of NGC 1068 is surrounded by a ring of star formation that contributes
half of the total luminosity of NGC 1068 (Telesco & Decher, 1988). The two
components are important in the understanding of galaxy energetics and galaxy
evolution. It is necessary to distinguish the dominant component of the
activity for active galaxies. Both starburst and Sy2 AGN galaxies exhibit
emission lines with FWHM$<$ 1000 km s-1. However, Baldwin et al. (1981),
Veilleux & Osterbrock (1987) and Kauffmann et al. (2003) demonstrated that Sy2
AGN galaxies may be distinguished from normal SFGs based on the diagnostic
diagram of [O III]/H$\beta$ versus [HII]/H$\alpha$, though not all galaxies
can be easily segregated. Machalski & Condon (1999) separated the RL AGN and
star-forming populations on the basis of FIR-radio flux ratios, FIR spectral
indices and radio-optical flux ratios. Best et al. (2005) pointed out that RL
AGNs are fairly well separated from SFGs in the plane of $D_{n}(4000)$ versus
$L_{1.4GHz}/M_{*}$.

Based on the classification provided by Best et al. (2005) and Best & Heckman
(2012), we plot a histogram of the $[4.6]-[12]$ colour in Figure 7. Figure 7
shows the separation of RL AGNs from SFGs; 95$\%$ of RL AGNs have $[4.6]-[12]$
$<$ 3.0, and 94$\%$ of SFGs have $[4.6]-[12]$ $>$ 3.0. The colours
$[3.4]-[12]$, $[3.4]-[22]$, and $[4.6]-[22]$ have similar functions in the
separation of the two types of galaxies, which is attributed to the fact that
the 12-$\mu$m and 22-$\mu$m emissions of SFGs are redder than those of RL
AGNs.

##  5 FIR colour properties

We search AKARI-FIS PSC and obtain AKARI-FIS data with high quality. AKARI-FIS
data have four-level flux quality indicator. High quality, i.e. the flux
quality indicator equals 3, means that the source is confirmed and the flux is
reliable. We neglect the data with the flux quality indicator equal to 1 and
2, which means that the source is not confirmed or the flux is not reliable.
15 and 282 AKARI 90-$\mu$m counterparts are obtained for RL AGNs and SFGs,
respectively. The detection rate of RL AGNs at the 90-$\mu$m band is $\sim
0.7\%$, which is much less than $\sim 56\%$ of SFGs. The majority of the RL
AGNs studied here are not included in the AKARI-FIS PSC, though the AKARI-FIS
survey covered more than 94$\%$ of the entire sky, and the AKARI-FIS PSC
includes 0.4 million infrared sources brighter than the detection limit
(approximately 0.6 Jy) at 90 $\mu$m (Yamamura et al. 2010). This suggests that
the 90 $\mu$m flux density of most of RL AGNs is lower than 0.6 Jy. In
addition, 15 AKARI 90-$\mu$m counterparts of RL AGNs include nine LERGs and
one HERG, respectively. The detection rate of LERGs and HERGs reach $\sim$0.5
and $\sim$2 per cent at 90-$\mu$m band, respectively. The FIR emissions at 90
and 140 $\mu$m are thought be associated with cold dust in host galaxies (e.g.
Netzer et al. 2007; Mullaney et al. 2011), and then HERGs tend to have more
abundant cold dust than LERGs, on average.

Figure 8: The $\alpha(22,90)-\alpha(12,22)$ and $\alpha(90,140)-\alpha(22,90)$
diagrams. A BB line marked with temperature and a PL line are also plotted as
the dashed and dotted line, respectively. A set of solid lines in the
$\alpha(90,140)-\alpha(22,90)$ diagrams indicate the loci of models with a
cold dust component of 30 K and a warm dust component of 80$-$100 K. Circles
represent RL AGNs and triangles represent SFGs.

The $\alpha(22,90)-\alpha(12,22)$ and $\alpha(90,140)-\alpha(22,90)$ diagrams
are also shown in Figure 8 with PL and BB lines plotted as dashed and dotted
lines, respectively. A set of solid lines in the
$\alpha(90,140)-\alpha(22,90)$ diagram indicate the loci of the composed
model, with a cold disc component of $\sim$30 K and a warm dust component of
80$-$100 K. The two components are composed according to the fractional
contribution of the warm dust component to the 90 $\mu$m-band flux. We can
calculated the flux densities at a given wavelength by

$F_{\lambda}=B_{\lambda}(T_{\text{c}})+A_{2}\times B_{\lambda}(T_{\text{w}}).$
(7)

Here, $B_{\lambda}(T_{\text{c}})$ and $B_{\lambda}(T_{\text{w}})$ represent
the Planck functions of the cold component with temperature $T_{\text{c}}$ and
the warm component with $T_{\text{w}}$, respectively, and
$A_{2}=\frac{\gamma_{2}B_{90\mu m}(T_{\text{c}})}{(1-\gamma_{2})B_{90\mu
m}(T_{\text{w}})}$, where $\gamma_{2}$ is the ratio of the radiation flux
density of the warm component to the total flux density at 90 $\mu$m. We can
then draw the set of black line in the $\alpha(90,140)-\alpha(22,90)$ diagram
by computing the spectral index between two bands from Equation 1 when
$\gamma_{2}$ changes.

Figure 8 shows $\alpha(22,90)<0$ for all RL AGNs detected at 90$\mu$m, which
indicates that some RL AGNs have a red FIR colour. In combination with Figure
3, we can see that the spectral indices for the majority of SFGs are
$\alpha(3.4,4.6)>$0, $\alpha(4.6,12)<$0, $\alpha(12,22)<$0, $\alpha(22,90)<$0
, and $\alpha(90,140)<$0, i.e. their spectral indices do not cross zero, which
suggests that SFGs have similar spectra from 3.4 to 90$\mu$m. The spectra may
be described by a PL with a similar spectral index, as shown by the
5$-$38$\mu$m MIR spectra of 22 starburst nuclei presented by Brandl et al.
(2006, see Figure 3 in their paper). However, the bottom panel in Figure 8,
the $\alpha(90,140)-\alpha(22,90)$ diagram, shows that a small number of SFGs
with $\alpha(90,140)<-1.4$ are distributed below the black solid lines and
close to the PL line. A set of black solid lines, plotted in the
$\alpha(90,140)-\alpha(22,90)$ diagram, indicates the composed model of a cold
disc component ($\sim$30 K) and a warm dust component. As shown in the
$\alpha(90,140)-\alpha(22,90)$ diagram, the sources above the black solid
lines and close to the BB line may be explained by thermal emissions from dust
at MIR wavelengths, whereas the MIR properties of the source distributed in
the region around the PL line cannot be explained by thermal emissions of the
composed model, meaning that non-thermal emissions may be used to understand
their MIR properties. In general, the FIR emissions of SFGs are thought to be
caused by the heating of dust by young, massive stars (Condon et al. 1991;
Helou & Bicay 1993; Machalski & Condon 1999). The
$\alpha(90,140)-\alpha(22,90)$ diagram suggests that there are many SFGs whose
FIR emissions may not come from dust. We also suggest the criteria
$\alpha(90,140)<-1.4$ to select active galaxies with non-thermal emissions at
MIR wavelengths.

##  6 Summary

The radio emissions of RLGs are dominated either by AGNs or by supernova
remnants in their star-forming regions; however, both the dusty tori
associated with the AGNs and the dust and gas clouds heated by young stars
emit MIR or FIR emissions. Here, we analyse the infrared colour properties of
a large sample of RLGs based on the 2MASS, WISE and AKARI survey data. RL AGNs
and SFGs are separately distributed in a $([3.4]-[4.6])-([4.6]-[12])$ diagram,
which provides the colour $[4.6]-[12]$ to separate RL AGNs from SFGs. 95$\%$
of RL AGNs have $[4.6]-[12]$$<$ 3.0, and 94$\%$ of SFGs have
$[4.6]-[12]$$>$3.0. This difference may be due to the significant contribution
of the dusts of SFGs to MIR emission. It is noted that a small number of RL
AGNs are distributed in the region of SFGs in the $([3.4]-[4.6])-([4.6]-[12])$
diagram, which indicates that the RL AGNs in the region of SFGs have redder
MIR and FIR colours than ordinary RL AGNs. The 3.4-90$\mu$m observed spectra
of SFGs are described by similar spectra, like a PL, which are indicated by
the distribution of SFGs around the PL line in $([12]-[22])-([4.6]-[12])$ and
$\alpha(90,140)-\alpha(22,90)$. However, the 3.4$-$22$\mu$m spectra of RL AGNs
shows different types, due to the variation of $\alpha(4.6,12)$ from $-2$ to
2. One type has the flux density increasing with wavelength from 3.4 to
22$\mu$m; the other type has the flux density decreasing with wavelength from
3.4 to 12$\mu$m, whereas flux density increases with wavelength from 12 to
22$\mu$m.

The MIR colours of RLGs are described by a model with a BB component and a PL
component. However, the PL component cannot exclude the contribution from
thermal emissions, because the PL component may be described by multi-colour
BB spectra, indicating dusts with various temperatures.

LERGs and HERGs have different distributions of the $[3.4]-[4.6]$,
$[4.6]-[12]$ and $[12]-[22]$ colours, which means that LERGs and HERGs have
intrinsically different MIR properties.

The $\alpha(90,140)-\alpha(22,90)$ diagram indicates that many SFGs with
$\alpha(90,140)<-1.4$ are distributed in the region around the PL line. They
cannot be described by the composed model with a cold disc component ($\sim$30
K) and a warm dust component involved in star formation. The FIR emissions of
the SFGs with $\alpha(90,140)<-1.4$ may not originate from dusts but instead
from non-thermal processes. Therefore, the criterion $\alpha(90,140)<-1.4$
provides a way to select active galaxies, for which non-thermal emissions may
contribute to FIR wavelength.

## Acknowledgements

We thank the anonymous referee for useful comments to improve this paper. We
also thank J.H. He for valuable discussions. This work was supported by the
Fundamental Research Funds for the Central Universities (no.CQDXWL-2014-004).
Chen, P.S. is supported by the Natural Science Foundation of China (grants
11173056).

This publication makes use of data products from the 2MASS, which is a joint
project of the University of Massachusetts and the Infrared Processing and
Analysis Center/California Institute of Technology, funded by the National
Aeronautics and Space Administration and the National Science Foundation, and
from the WISE, which is a joint project of the University of California, Los
Angeles, and the Jet Propulsion Laboratory/California Institute of Technology,
funded by the National Aeronautics and Space Administration. This research is
based on observations with AKARI, a JAXA project with the participation of
ESA.

## References

  * [1] Antonucci R., 1984, ApJ, 281, 112
  * [2] Antonucci R., 1993, ARA&A, 31, 473
  * [3] Akritas M. G., & Siebert J., 1996, MNRAS, 278, 919
  * [4] Assef R. J., Stern D., Kochanek C. S., Blain A. W., et al., 2013, ApJ, 772, 26
  * [5] Becker R. H., White R. L., Helfand D. J., 1995, ApJ, 450, 559
  * [6] Baldwin J., Kirhakos S., Terlevich R., 1981, PASP, 93, 5
  * [7] Bessell M. S., & Brett J. M., 1988, PASP, 100, 1134
  * [8] Best P. N., Kauffmann G., Heckman T. M., & Ivezić Ž. 2005, MNRAS, 362, 9
  * [9] Best P. N., & Heckman T. M., 2012, MNRAS, 421, 1569
  * [10] Bourne N., Dunne L., Ivison R. J., et al., 2011, MNRAS, 410, 1155
  * [11] Brandl B. R., Bernard-Salas J., Spoon H. W. W., Devost D., et al. 2006, ApJ, 653, 1129
  * [12] Condon J. J., Cotton W. D., Greisen E. W., et al., 1998, AJ, 115, 1693
  * [13] Chen P. S., Shan H. G., 2009, MNRAS, 393, 1408
  * [14] Ciotti L., Ostriker J. P., 2001, ApJ, 551, 131
  * [15] Cleary K., Lawrence C. R., Marshall J. A., Hao L., & Meier D., 2007, ApJ, 660, 117
  * [16] Condon J. J., Anderson M. L., Helou G., 1991, ApJ, 376, 95
  * [17] De Breuck C., Seymour N., Stern D., Willner S. P., et al., 2010, ApJ, 725, 36
  * [18] Dicken D., Tadhunter C., Axon D., et al., 2009, ApJ, 694, 268
  * [19] Dultzin-Hacyan D., Benitz E., 1994, A&A, 291, 720
  * [20] Eales S. A., Rawlings S., 1993, ApJ, 411, 67
  * [21] Elvis M., Wilkes B. J., McDowell J. C., et al.,1994, ApJS, 95, 1
  * [22] Esin A. A., McClintock J. E., Narayan R., 1997, ApJ, 489, 865
  * [23] Evans A. S., 1998, ApJ, 498, 553
  * [24] Gan Z. M., Yuan F., Ostriker J. P., Ciotti L., Novak G., 2014, ApJ, 789, 150
  * [25] Gürkan G., Hardcastle M. J., Jarvis M. J., 2014, MNRAS, 438, 1149
  * [26] Hardcastle M. H., Evans D. A., Croston J. H., 2007, MNRAS, 376, 1849
  * [27] Hardcastle et al., 2010, MNRAS, 409, 122
  * [28] Harwit M., Pacini F., 1975, ApJL, 200, 127
  * [29] Heckman T., Chambers K. C., Postma, M., 1992, ApJ, 391, 39
  * [30] Helou G., Bicay M. D., 1993, ApJ, 415, 93
  * [31] Hes R. Barthel P. D., Hoekstra H., 1995, A&A, 303, 8
  * [32] Ichikawa K., Ueda Y., Terashima Y., et al., 2012, ApJ, 754, 45
  * [33] Ichikawa K., Imanishi, M., Ueda, Y., Nakagawa, T., et al., 2014, ApJ, 794, 139
  * [34] Ishihara D., Onaka T., Kataza H., Salama A., et al., 2010, A&A, 514, A1
  * [35] Iwamuro F., Oya S., Tsukamoto H., Maihara T., 1996, ApJ, 466, L67
  * [36] Kauffmann G., et al., 2003, MNRAS, 346, 1005
  * [37] Kawada M., Baba H., Barthel P. D., Clements D., et al., 2007, PASJ, 59, S389
  * [38] Kouzuma S., Yamaoka H., 2010, A&A, 509, 64
  * [39] Leipski C., Haas M., Willner S. P., et al., 2010, ApJ, 717, 766
  * [40] Lacy M., Storrie-Lombardi L. J., Sajina A., et al., 2004, ApJS, 154, 166
  * [41] Matsuta K., Gandhi P., Dontani T., et al., 2012, ApJ, 753, 104
  * [42] Machalski J., Condon J. J., 1999, ApJS, 123, 41
  * [43] Meyer M. R., Calvet N., & Hillenbrand L. A., 1997, AJ, 114, 288
  * [44] Mullaney J. R., Alexander D. M., Goulding A. D., Hickox R. C., 2011, MNRAS, 414, 1082
  * [45] Narayan R., Yi I., 1995, ApJ, 452, 710
  * [46] Netzer H., Lutz D., Schweitzer M., et al., 2007, ApJ, 666, 806
  * [47] Park S. Q., Barmby P., Willner S. P., et al., 2010, ApJ, 717, 1181
  * [48] Onaka T., Matsuhara H., Wada T., Fujishiro N., et al., 2007, PASJ, 59, S401
  * [49] Rawlings S., 1989, Saunders R., Eales S. A., Mackay C. D., 1989, MNRAS, 240, 701
  * [50] Rieke G. H., Lebofsky M. J., 1985, ApJ, 288, 618
  * [51] Richards G. T., Lacy M., Storrie-Lombardi L. J., et al., 2006, ApJS, 166, 470
  * [52] Schlegel D. J., Finkbeiner D. P., Davis M., 1998, ApJ, 500, 525
  * [53] Sekiguchi K., 1987, ApJ, 316, 145
  * [54] Shakura N. I., & Sunyaev R. A., 1973, A&A, 24, 337
  * [55] Siebenmorgen R., Freudling W., Krügel E., Hass M., 2004, A&A, 421, 129
  * [56] Skrutskie M. F., Cutri R. M., Stiening R., Weinberg M. D., et al., 2006, AJ, 131, 1163
  * [57] Smith J.D.T., Draine B.T., Dale D.A., Moustakas J., et al., 2007, ApJ, 656, 770
  * [58] Stern D., Eisenhardt P., Gorjian V., Kochanek C. S., et al., 2005, ApJ, 631, 163
  * [59] Stern D., Assef R. J., Benford D. J., Blain A., et al., 2012, ApJ, 753, 30
  * [60] Stoughton C., et al., 2002, AJ, 124, 485
  * [61] Telesco C. M., Becklin E. E., Wynn-Williams C. G., Harper D. A., 1984, ApJ, 282, 427
  * [62] Telesco C. M., & Decher R., 1988, ApJ, 334, 573
  * [63] Urry C. M., Padovani P., 1995, PASP, 107, 803
  * [64] Veilleux S., Osterbrock D., 1987, ApJS, 63, 295
  * [65] Véron-Cetty M. P., & Véron P., 2010, A&A, 518, 10
  * [66] Wright E. L., Eisenhardt P.R.M., Mainzer A. K., et al., 2010, AJ, 140, 1868
  * [67] Yamamura I., Makiuti S., Ikeda N., Fukuda Y., et al., 2010, VizierR Online Data Catalog, 2298, 0
  * [68] York D. G., Adelman, J., Anderson, J. E., et al., 2000, AJ, 120, 1579
  * [69] Yuan H. B., Liu X. W., Xiang M. S., 2013, MNRAS, 430, 2188
  * [70] Yuan F., Narayan R., 2014, ARA&A, 52, 529


# An Integrated Framework for Two-pass Personalized Voice Trigger

###### Abstract

In this paper, we present the XMUSPEECH system for Task 1 of 2020 Personalized
Voice Trigger Challenge (PVTC2020). Task 1 is a joint wake-up word detection
with speaker verification on close talking data. The whole system consists of
a keyword spotting (KWS) sub-system and a speaker verification (SV) sub-
system. For the KWS system, we applied a Temporal Depthwise Separable
Convolution Residual Network (TDSC-ResNet) to improve the system’s
performance. For the SV system, we proposed a multi-task learning network,
where phonetic branch is trained with the character label of the utterance,
and speaker branch is trained with the label of the speaker. Phonetic branch
is optimized with connectionist temporal classification (CTC) loss, which is
treated as an auxiliary module for speaker branch. Experiments show that our
system gets significant improvements compared with baseline system.

Index Terms: Keyword spotting, Speaker verification, TDSC-ResNet, Multi-task,
CTC

##  1 Introduction

Voice wake-up is becoming popular in people everyday life as smart devices are
widely used. Usually, detecting a trigger phrase including two stages, the
first is a KWS system to detect the prefixed keywords, and the second is a SV
system to conform the identity of the speaker. PVTC2020 [1] includes joint
wake-up word detection with speaker verification on close talking data (task
1) and joint wake-up detection with speaker verification on far-field multi-
channel microphone array data (task 2). More details about the dataset, task
and evaluation metrics refer to [1]. In this paper, we mainly focus on task 1.
From a practical standpoint, the whole system should satisfy the requirements
of low latency. Typically, SV task needs lager networks than KWS task, an
always-on SV model will lead to unnecessary computation and high latency. So
we employed a cascaded 2-stage architecture, where a low cost KWS system
decides whether triggered and then the keyword segment is fed to a larger SV
system to do further identification.

Currently, the Deep-KWS [2] system has become more popular for its simpler
implementation and higher accuracy. Many Convolutional Neural Networks (CNNs)
architectures [3, 4, 5, 6, 7, 8, 9] have been explored for KWS system to
achieve high accuracy with limited model size and computational resources.
Temporal Convolution (TConv) [6] extracts the whole level frequency features
to address the problem that conventional 2D CNNs struggle with concentrating
the relation between high and low frequencies. Depthwise Separable Convolution
Neural Network [10] considers the channel realm and space realm separately,
reduces the number of the parameters of the standard convolution without
significant performance degradation, and is well applied on KWS[5]. To achieve
a fast and accurate KWS system, we applied TConv and depthwise separable
convlolution on our Deep-KWS system.

In SV field, x-vector framework proposed by Snyder et al. [11] is the most
popular framework recently. X-vector can make full use of deep neural network
and significantly improve the performance of SV systems. In recent years,
multi-task (MT) frameworks have been applied to combine speech with speaker
information to improve the performance of SV systems [12, 13, 14]. These works
showed that we can add frame-level phonetic information learned from
multitasking before the pooling layer. It can help the SV system to
distinguish speaker-specific information more easily. [15, 16] attempted to
jointly optimize KWS and SV with a single network. In the speaker dependent
voice trigger task, the SV system aims to verify the speaker identity of
utterance containing prefixed keywords. Due to the above, we reproduced the
frame-level MT framework [13] to better utilize the phonetic information in SV
system. However, this method needs to get the phoneme labels of the inputs on
frame-level. Usually the alignment is generated by a well trained automatic
speaker recognition (ASR) model. The procedure is complex and the accuracy of
labels is subject to the performance of ASR model. What’s more, the ASR model
might have decoding failure when processing the audio which is domain
mismatched or noisy. CTC [17] can automatically label unsegmented sequence
data and has achieved great success in many end-to-end applications [18, 19,
20]. Motivated by [15], we considered that CTC could be a simpler approach to
embed frame-level phonetic information into SV system via MT learning. Thus,
we proposed a multi-CTC x-vector network for PVTC2020 task 1 and significantly
improved the performance compared with the official baseline.

We demonstrate the whole system in Section 2. Experimental details are
presented in Section 3, and results are reported and analyzed in Section 4.
Finally, we conclude this work in Section 5.

##  2 System description

Figure 1: Schematic diagram of the difference between Conventional Convolution
and Temporal Convolution.

###  2.1 KWS sub-system

The KWS system consists of three modules:(i) a feature extraction module, (ii)
a deep neural network, and (iii) a posterior handling module. The features are
80-dimensional log-mel filterbank (FBank) computed with a 25ms window and
shifted every 10ms. And we apply a window of 40 frames, which contain context
information of sub-keywords, to generate training samples as the input of the
model. We choose CNN as the acoustic model of KWS system. In the posterior
handling module, the sequence of acoustic features is projected to a posterior
probabilities sequences of the sub-words after the acoustic model. We smooth
the posteriors over a fixed window of size 50 by taking average, and the
confidence score is computed within a sliding window of size 150. The method
of confidence computation is proposed in [21], in which the confidence score
subject to the constraint that the sub-keywords are uttered in the same order
as in the specified keyword. The system triggers whenever the confidence score
is higher than a predefined threshold.

####  2.1.1 Temporal convolution

The Temporal Convolution ResNet (TC-ResNet) model [6] is composed of sequence
of residual blocks which use one dimensional convolution named Temporal
Convolution (TConv). We treat the input FBank as time series. Conventionally,
the FBank features are denoted as $\mathbf{I}\in\mathbb{R}^{F\times T}$, where
F represents the dimension of FBank feature, and T denotes the number of
frames. As shown in Figure 1. Conventional CNNs reshape the input from
$\mathbf{I}$ to $\mathbf{X_{2d}}\in\mathbb{R}^{C\times F\times T}$, where
$C=1$ means the channel dimension. In TConv, the feature dimension is equal to
the channels of the input feature map, so the input feature map $X$ is
transformed to $\mathbf{X_{1d}}\in\mathbb{R}^{F\times C\times T}$, where
$F=80$ and $C=1$. Therefore, all convolutions in the model are along the
temporal dimension, avoiding stacking a large number of layers to capture
high-level frequency features. In addition, TConv requires a smaller number of
computations compared with the conventional 2d convolution. What’s more, it
can decrease the output feature map (i.e., the input feature map of the next
layer) of the TConv and dramatically reduce the computations in the following
layers.

####  2.1.2 Model architecture overview

Figure 2: Schematic diagram of TDSC-ResNet.

Our Temporal Depthwise Separable Convolution ResNet (TDSC-ResNet) is based on
MatchboxNet [22]. As shown in Figure 2, the model uses TConv and depthwise and
pointwise convolution. We first apply SpecAugment [23] on the bottom of the
architecture thus enable the acoustic model to become more robust. After
SpecAugment, the input feature map $\mathbf{I}\in\mathbb{R}^{F\times T}$ is
transformed to $\mathbf{X_{1d}}\in\mathbb{R}^{F\times 1\times T}$ as
illustrated in Sec. 2.1.1. Then TConv and Squeeze-and-Excitation (SE) [24]
block can better accommodate feature maps. Inside the repeated blocks are
separable convolution, 1$\times$1 pointwise convolutions, batch norm, ReLU,
and dropout (see the right part of Figure 2), and we set the $n=3$ which means
there is 3 blocks in the architecture. Finally we position two layers of
additional TConv followed by an average pooling layer and then a fully
connected layer and a softmax activation layer are applied to obtain the sub-
words occurrence probability of the keyword. The detail settings of the model
is listed in Table 1. The receptive field of the model is designed according
to the length of input samples.

Table 1: The configuration of TDSC-ResNet. The number after Block denotes the
index of the repeated blocks.

Layers | Kernel size | Channels
---|---|---
TConv① | 1x6 | 128
Block① Depthwise Conv | 1x7 | 128
Block① Pointwise Conv | 1x1 | 64
Block② Depthwise Conv | 1x8 | 64
Block② Pointwise Conv | 1x1 | 64
Block③ Depthwise Conv | 1x9 | 64
Block③ Pointwise Conv | 1x1 | 64
TConv② | 1x14 | 128
TConv③ | 1x1 | 128

###  2.2 SV sub-system

The standard x-vector system supports the input to have fixed length. We first
modified the x-vector system to allow various lengths input to better fit the
CTC loss training. The feature is 40-dim MFCCs with 3-dim Pitch, 25ms frame
length and 10ms shift. Cepstral mean-normalization (CMN) with a sliding window
of 3 seconds and voice active detection (VAD) are also applied. In the
training phase, the training data contains keyword and other contents. But in
the evaluation phase, the SV system extract discriminative speaker embeddings
of the keyword segments, which are much shorter than training data. Hence, we
optimize the model through two steps. First the model is trained with all the
training data. Then only the keyword segments are used to fine-tune the model
again. SpecAugment is applied during training. In enrollment stage, the
average vector of the three utterances’ embedding extracted from the target
speaker is saved as the enrollment speaker embedding vector.

####  2.2.1 Connectionist temporal classifier

Unlike cross entropy, in which the alignments between input and target
sequences should be given, the CTC loss automatically learn the alignments.
Suppose there are $N$ modeling label units, the softmax layer before CTC
consists of $N$ units and a blank unit. The introduction of blank unit enable
outputs to define the probabilities of all possible alignments of the input
sequence. Given a T frames utterance, the CTC path
$\pi=(\pi_{0},...,\pi_{T-1})$ contains repetitions of no-blank labels and the
blank unit. After a many-to-one mapping function $\beta$, which means removing
the repeated labels and blanks, the CTC path can be mapped to the
corresponding label sequence (e.g. $\beta$(a-ab-) = $\beta$(-aa- -abb) = aab).
Summing all the possibilities of CTC paths has heavy computing burden, and a
forward-backward dynamic programming algorithm [17] was proposed to solve this
problem. Given the label sequence, CTC naturally tends to align each label
prediction. In this way, the sequence of frame-level posterior probability
distribution before CTC loss is similar to that before CE loss, while the
later requires a predetermined alignment.

####  2.2.2 The frame-level multi-task learning

Figure 3: Frame-level phonetic MT x-vector framework.

We adopt time delay neural network (TDNN) with SE-block as the speaker
embedding extractor, which is optimized by AAM-Softmax [25] loss. The detailed
configuration is shown in Table 2. The MT system is depicted in Figure 3 and
the configuration of TDNNs in phonetic branch is the same as that in speaker
part. Unlike the MT system in [15], the phonetic branch and the speaker branch
in our MT system are trained jointly by a total loss function (1) for each
examples, and the phonetic branch is more inclined to assist SV branch to
achieve a better performance. After training, the phonetic part is removed
from the model. For the back-end, the $embedding\ far$ in Figure 3 is for PLDA
scoring and $embedding\ near$ for cosine scoring. PLDA model is trained
corresponding to the train set. Submean and normalization are applied during
scoring.

$L_{loss}=L_{speaker}+\alpha L_{speech}$ (1)

Table 2: The configuration of the speaker embedding extractor.

Layer-index | Layer | Context | SE-ratio | Size
---|---|---|---|---
1 | TDNN-SE | {t-2:t+2} | 4 | 512
2 | TDNN-SE | {t - 2,t,t + 2} | 8 | 512
3 | TDNN-SE | {t - 3,t,t + 3} | 8 | 512
4 | TDNN | {t} |  | 512
5 | TDNN | {t} |  | 1500
6 | Stats pooling | [0,T] |  | 2 x 1500
7 | TDNN | {0} |  | 512
8 | TDNN | {0} |  | 512
9 | AAM-Softmax | {0} |  | Num.spk

For the multi-Cross-Entropy x-vector (MT-CE-XV) system, the phonetic branch in
Figure 3 is trained by CE loss. To get the aligned label, we used the PVTC
training data to train a lattice-free MMI based ASR [26] refer to Kaldi [27]
recipe111https://github.com/kaldi-asr/kaldi/blob/master/egs/aishell/s5/run.sh.
Then, the frame-level phoneme alignment is obtained through the ASR model.
Assume an utterance with T frames, the $L_{speech}$ in (1) is defined as (2)

$L_{speech}=\frac{1}{T}\sum_{t=1}^{T}CE(O^{p}_{t},Y^{p}_{t})$ (2)

where $O^{p}_{t}$ denotes the output of the phonetic branch at time t,
$Y^{p}_{t}$ is its corresponding frame-level phoneme label.

For the multi-CTC x-vector (MT-CTC-XV) system, the $L_{speech}$ is (3)

$L_{speech}=L^{\pi}_{CTC}(Y_{l})$ (3)

where $\pi$ is the possibility distribution permutation before the CTC and
$Y_{l}$ is the label sequence.

A multi-CE-CTC x-vector (MT-CTC-CE-XV) system is a combination of CE and CTC
loss in MT learning. That is, there are two phonetic branches, one for CE loss
and another for CTC. Here the $L_{speech}$ is (4)

$L_{speech}=\frac{1}{T}\sum_{t=1}^{T}CE(O^{p}_{t},Y^{p}_{t})+L^{\pi}_{CTC}(Y_{l})$
(4)

Table 3: Results of different systems.

System index | KWS system | SV system | SV-dev-EER% | SV-dev-minDCF | $\bm{score_{dev}}$ | $\bm{score_{test}}$
---|---|---|---|---|---|---
S1 |  KWS-baseline[1] |  SV-baseline[1] | 1.32 | 0.16 | 0.10 | 0.37
S2 [30] |  |  |  |  |  | 0.075
S3 [31] |  |  |  |  | 0.042 | 0.081
S4 | XMU-KWS | SV-baseline | 1.368 | 0.162 | 0.090 | 0.161
S5 | XMU-KWS | x-vector | 1.572 | 0.195 | 0.099 | 0.189
S6 | XMU-KWS | MT-CE-XV | 1.159 | 0.152 | 0.073 | 0.168
S7 | XMU-KWS | MT-CTC-XV-phoneme | 1.291 | 0.143 | 0.078 | 0.162
S8 | XMU-KWS | MT-CTC-XV-char | 0.974 | 0.112 | 0.062 | 0.161
S9 | XMU-KWS | MT-CTC-CE-XV | 1.237 | 0.123 | 0.073 | 0.168
S10 | XMU-KWS | MT-CTC-XV-char + score-fusion with PLDA | 0.808 | 0.086 | 0.051 | 0.140
S11 |  | \+ Data augmentation | 0.786 | 0.083 | 0.049 | 0.126
S12 |  | \+ model-fusion with SV-baseline | 0.570 | 0.061 | 0.038 | 0.098

##  3 Experimental setup

All of the models are implemented based on our open source toolkits:ASV-
Subtool [28]. In the KWS system, The validation set is randomly selected 10
out of 300 speakers’ data in the training set. The keyword ‘xiao le xiao le’
is split to three sub-keywords (i.e., ‘1_xiao le’, ‘2_le xiao’, ‘3_xiao le’),
and each sub-keyword contains an interval between Chinese characters inside
the keyword. The labels for each input is determined by force-alignment with a
large ASR system as the official pipeline. In the evaluation period, a simple
VAD is applied to detect voice activity, and only the audio clips which
trigger the KWS system are saved to be further transformed to speaker
embeddings.

To train the SV system, the model is first pre-trained by the all training
data and then fine-tuned by the keyword segments. Here are some details about
training strategy:

  * •

KWS: Stochastic gradient descent (SGD) is used with a batch size of 128 for
100 epochs. The initial learning rate is set as 0.01 and decreases by a factor
of 0.5 when the model reaches a valid loss plateau. Early stopping is employed
when the valid loss is not decreasing.

  * •

Pre-train SV: The Radam optimizer [29] is used to optimize the models with a
batch size of 32 for 20 epochs.

  * •

Fine-tune SV: SGD optimizer with a batch size of 128 for 20 epochs. The
learning rate is initialized as 0.03 and decreases by a factor of 0.5 when the
model reaches a valid loss plateau.

##  4 Results and analysis

Table 4: Performance of the KWS models on dev set (the false rejection (FR) rate [%] under one false alarm (FA) per hour). Models |  Params |  FR |  RTF
---|---|---|---
Baseline [1] |  240K |  2.00 |  0.076
TDSC-ResNet |  231K |  0.72 |  0.043

###  4.1 Performance of KWS sub-system

We choose the false rejection rate under one false alarm per hour as the KWS
system performance criterion. In a real scenario, the main cost of time comes
from the always-on KWS system. We first calculate the process time $T_{kws}$
of KWS system which consists of the time to trigger the system and the time to
generate the keyword clips. The real time factor (RTF) is calculated as
follows:

$\textit{RTF}=T_{kws}/T_{total\\_dev}$ (5)

where $T_{total\\_dev}$ is the total duration of the development set audios,
and here $T_{total\\_dev}=20.13$. The KWS systems are evaluated on an Intel(R)
Xeon(R) E5-2643 v4 CPU clocked at 3.4 GHz. As shown in Table 4, our KWS system
has higher accuracy and lower latency compared with the baseline.

###  4.2 Performance of SV models

After KWS system, the keyword segments of the whole utterance in development
set compose a new dataset, which will determine the threshold of the following
SV system. We choose the mean threshold of Equal Error Rate (EER) and minDCF
[32] as the threshold of the SV system.

The statistic $score$ is calculated from the miss rate and the FA rate
according to the following equation:

$score=Miss+alpha*\textit{FA}$ (6)

where $Miss$ represents the proportion of errors in all positive label
samples, and FA refers to the rate of errors in all negative label samples.
The alpha constant is set as 19, which is calculated by the assumption that
the probability of the positive samples is 0.05.

The results of different systems are shown in Table 3. From System S4 to S10,
all SV models apply a cosine scoring except the one in S10, which also uses
PLDA scoring with the $embedding\ far$ in Figure 3. As the official baseline
selected data on OpenSLR222http://openslr.org/resources.php as pre-training
data, our x-vector system in S5 performs worse than S4. We observe that all
the multi-task learning SV models from S6 to S10 achieve better performance
than S5. It proves the point that the frame-level phonetic information
obtained by multi-task benefits SV system. We choose the MT-CTC-XV-char SV
system and use PLDA to estimate scores, then the cosine scores and PLDA scores
of S10 are fused. Compared with SV-baseline, the single SV system in S10 gets
better performance without other pre-training data. To make SV models more
robust, MUSAN[33] and RIRs[34] noises are added to the training data as a
method of data augmentation (S11). Finally, with the score fusion on SV
systems in S11 and SV-baseline (S4), we get a competitive result (S12)
compared with the top 2 teams (S2,S3), and the score gap on the evaluation set
may stem from the additional information of other pre-training data.

###  4.3 Effect of multi-task learning

Meanwhile, different types of multi-task learning are also investigated.
Compared with MT-CE-XV (S6), MT-CTC-XV (S7,S8) trains with the label sequence
directly and automatically utilizes phonetic information. We find that S8
outperforms S7, and the difference between them is the CTC modeling unit. The
Chinese char unit contains more context information compared with phoneme
unit, and contributes better performance. However, combining CTC loss and CE
loss in multi-CE-CTC (S9) does not lead to further improvement, suggesting
that the two phonetic branches may have learned duplicate frame-level phonetic
information.

##  5 Conclusions

This paper presents a two-pass system for personalized voice trigger. We
applied TDSC-ResNet to Deep-KWS system and made it more accurate and faster.
In addition, various multi-task learning framework are explored to jointly
learn frame-level phonetic information in SV system. What’s more, a MT-CTC-XV
system, which is simpler than MT-CE-XV system, is proposed to utilize the
phonetic information for SV task. The system can be easily applied to transfer
learning and get further improvement. After score fusion, the single MT-CTC-XV
sub-system gets a better performance compared with official SV system. At
last, the fusion system achieves a score of 0.098 on evaluation set.

##  6 Acknowledgements

The work is supported by the National Natural Science Foundation of China
(Grant No.61876160 and No.62001405).

## References

  * [1] Y. Jia, X. Wang, X. Qin, Y. Zhang, X. Wang, J. Wang, and M. Li, “The 2020 personalized voice trigger challenge: Open database, evaluation metrics and the baseline systems,” _arXiv preprint arXiv:2101.01935_ , 2021.
  * [2] G. Chen, C. Parada, and G. Heigold, “Small-footprint keyword spotting using deep neural networks,” in _2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2014, pp. 4087–4091.
  * [3] R. Tang and J. Lin, “Deep residual learning for small-footprint keyword spotting,” in _2018 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2018, pp. 5484–5488.
  * [4] A. Coucke, M. Chlieh, T. Gisselbrecht, D. Leroy, M. Poumeyrol, and T. Lavril, “Efficient keyword spotting using dilated convolutions and gating,” in _ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2019, pp. 6351–6355.
  * [5] M. Xu and X.-L. Zhang, “Depthwise separable convolutional resnet with squeeze-and-excitation blocks for small-footprint keyword spotting.” in _Interspeech 2020_ , 2020, pp. 2547–2551.
  * [6] S. Choi, S. Seo, B. Shin, H. Byun, M. Kersner, B. Kim, D. Kim, and S. Ha, “Temporal convolution for real-time keyword spotting on mobile devices.” in _Interspeech 2019_ , 2019, pp. 3372–3376.
  * [7] C. Yang, X. Wen, and L. Song, “Multi-scale convolution for robust keyword spotting,” _Proc. Interspeech 2020_ , pp. 2577–2581, 2020.
  * [8] X. Li, X. Wei, and X. Qin, “Small-footprint keyword spotting with multi-scale temporal convolution,” _arXiv preprint arXiv:2010.09960_ , 2020.
  * [9] T. Higuchi, M. Ghasemzadeh, K. You, and C. Dhir, “Stacked 1d convolutional networks for end-to-end small footprint voice trigger detection,” _arXiv preprint arXiv:2008.03405_ , 2020.
  * [10] A. G. Howard, M. Zhu, B. Chen, D. Kalenichenko, W. Wang, T. Weyand, M. Andreetto, and H. Adam, “Mobilenets: Efficient convolutional neural networks for mobile vision applications,” _arXiv preprint arXiv:1704.04861_ , 2017.
  * [11] D. Snyder, D. Garcia-Romero, D. Povey, and S. Khudanpur, “Deep neural network embeddings for text-independent speaker verification.” in _Interspeech_ , 2017, pp. 999–1003.
  * [12] Y. Liu, L. He, J. Liu, and M. T. Johnson, “Speaker embedding extraction with phonetic information,” _arXiv preprint arXiv:1804.04862_ , 2018.
  * [13] S. Wang, J. Rohdin, L. Burget, O. Plchot, Y. Qian, K. Yu, and J. Cernockỳ, “On the usage of phonetic information for text-independent speaker embedding extraction.” in _Interspeech_ , 2019, pp. 1148–1152.
  * [14] M. Zhao, R. Li, S. Yan, Z. Li, H. Lu, S. Xia, Q. Hong, and L. Li, “Phone-aware multi-task learning and length expanding for short-duration language recognition,” in _2019 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC)_. IEEE, 2019, pp. 433–437.
  * [15] S. Sigtia, E. Marchi, S. Kajarekar, D. Naik, and J. Bridle, “Multi-task learning for speaker verification and voice trigger detection,” in _ICASSP 2020-2020 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2020, pp. 6844–6848.
  * [16] M. Jung, Y. Jung, J. Goo, and H. Kim, “Multi-task network for noise-robust keyword spotting and speaker verification using ctc-based soft vad and global query attention,” _arXiv preprint arXiv:2005.03867_ , 2020.
  * [17] A. Graves, S. Fernández, F. Gomez, and J. Schmidhuber, “Connectionist temporal classification: labelling unsegmented sequence data with recurrent neural networks,” in _Proceedings of the 23rd international conference on Machine learning_ , 2006, pp. 369–376.
  * [18] S. Kim, T. Hori, and S. Watanabe, “Joint ctc-attention based end-to-end speech recognition using multi-task learning,” in _2017 IEEE international conference on acoustics, speech and signal processing (ICASSP)_. IEEE, 2017, pp. 4835–4839.
  * [19] S. Karita, N. E. Y. Soplin, S. Watanabe, M. Delcroix, A. Ogawa, and T. Nakatani, “Improving transformer-based end-to-end speech recognition with connectionist temporal classification and language model integration,” in _20th Annual Conference of the International Speech Communication Association: Crossroads of Speech and Language, INTERSPEECH 2019_ , 2019, pp. 1408–1412.
  * [20] T. N. Sainath, R. Pang, D. Rybach, Y. He, R. Prabhavalkar, W. Li, M. Visontai, Q. Liang, T. Strohman, Y. Wu _et al._ , “Two-pass end-to-end speech recognition,” _arXiv preprint arXiv:1908.10992_ , 2019.
  * [21] B. Liu, S. Nie, Y. Zhang, S. Liang, Z. Yang, and W. Liu, “Focal loss and double-edge-triggered detector for robust small-footprint keyword spotting,” in _ICASSP 2019-2019 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2019, pp. 6361–6365.
  * [22] S. Majumdar and B. Ginsburg, “Matchboxnet–1d time-channel separable convolutional neural network architecture for speech commands recognition,” _arXiv preprint arXiv:2004.08531_ , 2020.
  * [23] D. S. Park, W. Chan, Y. Zhang, C.-C. Chiu, B. Zoph, E. D. Cubuk, and Q. V. Le, “Specaugment: A simple data augmentation method for automatic speech recognition.” in _Interspeech 2019_ , 2019, pp. 2613–2617.
  * [24] J. Hu, L. Shen, and G. Sun, “Squeeze-and-excitation networks,” in _2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2018, pp. 7132–7141.
  * [25] J. Deng, J. Guo, N. Xue, and S. Zafeiriou, “Arcface: Additive angular margin loss for deep face recognition,” in _2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)_ , 2019, pp. 4690–4699.
  * [26] D. Povey, V. Peddinti, D. Galvez, P. Ghahremani, V. Manohar, X. Na, Y. Wang, and S. Khudanpur, “Purely sequence-trained neural networks for asr based on lattice-free mmi.” in _Interspeech_ , 2016, pp. 2751–2755.
  * [27] D. Povey, A. Ghoshal, G. Boulianne, L. Burget, O. Glembek, N. Goel, M. Hannemann, P. Motlicek, Y. Qian, P. Schwarz, J. Silovsky, G. Stemmer, and K. Vesely, “The kaldi speech recognition toolkit,” in _IEEE 2011 Workshop on Automatic Speech Recognition and Understanding_ , 2011\.
  * [28] F. Tong, M. Zhao, J. Zhou, H. Lu, Z. Li, L. Li, and Q. Hong, “Asv-subtools: Open source toolkit for automatic speaker verification,” in _ICASSP 2021-2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_. IEEE, 2021, pp. 6184–6188.
  * [29] L. Liu, H. Jiang, P. He, W. Chen, X. Liu, J. Gao, and J. Han, “On the variance of the adaptive learning rate and beyond,” in _ICLR 2020 : Eighth International Conference on Learning Representations_ , 2020.
  * [30] https://www.pvtc2020.org/leaderboard.html.
  * [31] J. Hou, L. Zhang, Y. Fu, Q. Wang, Z. Yang, Q. Shao, and L. Xie, “The NPU system for the 2020 personalized voice trigger challenge,” _arXiv preprint arXiv:2102.13552_ , 2021.
  * [32] C. S. Greenberg, V. M. Stanford, A. F. Martin, M. Yadagiri, G. R. Doddington, J. J. Godfrey, and J. Hernandez-Cordero, “The 2012 nist speaker recognition evaluation.” in _INTERSPEECH_ , 2013, pp. 1971–1975.
  * [33] D. Snyder, G. Chen, and D. Povey, “Musan: A music, speech, and noise corpus,” _arXiv preprint arXiv:1510.08484_ , 2015.
  * [34] T. Ko, V. Peddinti, D. Povey, M. L. Seltzer, and S. Khudanpur, “A study on data augmentation of reverberant speech for robust speech recognition,” in _2017 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)_ , 2017, pp. 5220–5224.