11institutetext: Max-Planck-Institute for Astronomy, Königstuhl 17, D-69117
Heidelberg, Germany
11email<EMAIL_ADDRESS>22institutetext: Main Astronomical Observatory of
the National Academy of Sciences of Ukraine, Akademika Zabolotnoho str. 27,
Kyiv 03143, Ukraine 33institutetext: Department of physics, Taras Shevchenko
Kyiv National University, Glushkova 2 ave., building 1, Kyiv 03680, Ukraine
44institutetext: Niels Bohr Institute, Experimental Particle Physics,
Blegdamsvej 17, 2100 København Copenhagen, Denmark 55institutetext:
Jagiellonian University Faculty of Physics, Astronomy and Applied Computer
Science ul. prof. Stanislawa Lojasiewicza 11, 30-348 Krakow

# What excites the optical emission in X-ray-selected galaxies? ††thanks: The
catalogue of X-ray galaxies selected from SDSS is only available in electronic
form at the CDS via anonymous ftp to cdsarc.u-strasbg.fr (130.79.128.5) or via
http://cdsweb.u-strasbg.fr/cgi-bin/qcat?J/A+A/

N.G. Pulatova 1122 H.-W. Rix 11 A.V. Tugay 33 L.V. Zadorozhna 334455 R.
Seeburger 11 M. Demianenko 11

We present a study of $1347$ galaxies at $z<0.35$ with detected nuclear X-ray
emission and optical emission line diagnostics in the Baldwin-Phillips-
Terlevich (BPT) diagram. This sample was obtained by cross-matching the X-ray
Multi-Mirror Mission Observatory - Newton (XMM-Newton) DR10 catalogue with
Sloan Digital Sky Survey (SDSS) DR17 galaxies with well-measured line ratios.
The distribution of these sources in the BPT diagram covers all three
excitation regimes: Ionized Hydrogen (HII) regions (23%), ‘composites’ (30%),
and Seyfert galaxies with the low ionization nuclear emission line regions
(LINERs) (47%). In contrast, the fraction of objects classified as active
galactic nuclei (AGN) in the SDSS subsample selected for cross-match with XMM-
Newton is only 13%. This fact illustrates that X-ray emission from galaxies
commonly points towards the presence of AGN. Our data show, for the first
time, a clear dependence of the BPT position on the ratio of the X-ray to
$H\alpha$ fluxes. Sources dominated by X-ray emission lie in the Seyfert and
LINER regimes of the BPT diagram. Most sources with a low X-ray-
to-$H\alpha$-luminosity ratio, $log_{10}(L_{X}/L_{H\alpha})<1.0$, lie in the
HII regime. In our sample, there are even 45 galaxies that have
$L^{Star}_{XR}/L^{Total}_{Xray}>0.5$. In contrast, the positions of the sample
members in the BPT diagram exhibit no dependence on the X-ray hardness ratio.
Our finding suggests that the X-ray-to-$H\alpha$ ratio can help us to
differentiate galaxies whose X-ray flux is dominated by an AGN from galaxies
with central X-ray binaries and other stellar X-ray sources.

###### Key Words.:

Astronomical data bases – (Galaxies:) quasars: emission lines – Galaxies:
Seyfert – Galaxies: star formation – X-rays: galaxies

##  1 Introduction

The origin of the ‘nuclear emission’ in galaxies, which is seen across a wide
range of the electromagnetic spectrum, has been the focus of astrophysical
studies for decades. Baldwin et al. (1981) proposed a powerful way to
understand what physical processes dominate gas ionisation based on the flux
ratios of forbidden optical lines in the interstellar medium. This criterion
relies on four reddening-insensitive line ratios: $[OIII]/H\beta$,
$[NII]/H\alpha$, $[SII]/H\alpha$, and $[OI]/H\alpha$.

The Baldwin-Phillips-Terlevich (BPT) diagram was intended to separate two
sources of ionisation: active galactic nuclei (AGN) or accretion discs around
the supermassive black hole (SMBH), and young, massive hot stars, that is the
host galaxy component. Kauffmann et al. (2003) defined the extreme starburst
line to make a semi-empirical fit to the outer bound of the Sloan Digital Sky
Survey (SDSS) galaxy spectra. This outer bound defines the region where
composite starburst–AGN objects are expected to lie on the diagnostic
diagrams. Later, Kewley et al. (2006) proposed an empirical relation to
separate Seyfert galaxies from low ionisation nuclear emission-line regions
(LINERs). Kauffmann et al. (2003) defined these latter as regions with [OIII]
and $H\alpha$ narrow-line region (NLR) emission-line luminosities in the range
of $\sim 10^{5}–10^{6}L\odot$. LINERs are found in galaxies of earlier Hubble
type than Seyfert galaxies, and their nuclear continua are usually dominated
by old stars. According to the BPT diagram, galaxies with emission lines can
be separated into the following categories depending on the type of source of
ionisation that dominates there: Seyfert, LINERs, and HII regions.

Figure 1: Distribution of the sample of 1347 X-ray-selected galaxies in
equatorial coordinates. Figure 2: Distribution of 1347 SDSS X-ray galaxies in
terms of the angular separation between the SDSS optical centre of the galaxy
and the XMM-Newton X-ray source. Figure 3: Distribution of 1347 SDSS galaxies
with X-ray nuclear emission in terms of redshift, z . Figure 4: Distribution
of 1347 SDSS X-ray-selected galaxies in terms of hardness ratio, HR1-HR4,
according to data from the XMM-Newton catalogue.
$HRn=\frac{B_{n+1}-B_{n}}{B_{n+1}+B_{n}}$, where $B$ denotes the count rates
in energy band $n$, and $n$ takes a value from 1 to 4.

We investigated the X-ray emission from star forming regions and AGN
separately. X-ray-selected galaxies can play a crucial role in our
understanding of these sources: if the dominant X-ray emission inevitably
comes from the accretion onto the SMBH, and if it is this process that
dominates the excitation of the optical emission lines, we should expect the
optical line diagnostics of X-ray-selected galaxies to place them
predominantly in the AGN regime in the Kewley version of the BPT diagram.
Torbaniuk et al. (2021) used a sample of X-ray-selected galaxies to quantify
the level of AGN activity by applying multi-wavelength AGN-selection criteria
(optical BPT diagrams, X-ray/optical ratio, etc.). The authors compared
specific black hole accretion rates for AGN-dominated, quiescent galaxies and
galaxies with active star formation. They concluded that the level of AGN
activity decreased with cosmic time, and they found a scenario where both star
formation and AGN activity are fueled by a common gas reservoir.

##  2 SDSS galaxies with nuclear X-ray emission

In this work, we study the X-ray emission and optical emission lines from
X-ray-selected galaxies from SDSS Sky Survey DR17 (Blanton et al., 2017). To
this end, we created a new sample of $1347$ galaxies at z¡0.35 by cross-
matching SDSS DR17 with X-ray Multi-Mirror Mission Observatory - Newton (XMM-
Newton) DR10 (Webb et al., 2020).

###  2.1 Data

In our work, we used data from the XMM-Newton observatory to study the X-ray
emission of galaxies. We chose 4XMM-Newton X-ray Source Catalogue Data Release
10 (hereafter 4XMM-DR10), as it is one of the largest publicly available
repositories of X-ray data from a single observatory (Webb et al., 2020). The
4XMM-DR10 database contains $849\leavevmode\nobreak\ 991$ detections of
$575\leavevmode\nobreak\ 158$ unique sources over $2.85\%$ of the sky. We used
data from the PN camera, which has an angular resolution of 6.6 arcsec (full
width at half maximum (FWHM)) in the energy range of 0.2 to 15 keV (Strüder et
al., 2001). We cross-matched 4XMM-DR10 with 204 895 SDSS-selected galaxies
using the ‘Sky’ algorithm from the TOPCAT software with a separation of
$6.0^{\prime\prime}$. For more information, see TOPCAT Tutorial, and Sect.
2.2). The sample of 204 895 SDSS galaxies was selected using the following
criteria:

1\. Galaxies with spectra with fractional flux uncertainties of $\leq 33$% or
0.13 dex (Bolton et al., 2012)

2\. Narrow-line galaxies with $\sigma_{line}<200\leavevmode\nobreak\ km/s$.
This criterion corresponds to the SDSS limit for narrow-line galaxies. Type 1
AGN galaxies with broad lines ($\sigma_{line}>200\leavevmode\nobreak\ km/s$)
cannot be plotted correctly on the BPT diagram (Bolton et al., 2012)

3\. These criteria were applied to emission lines $[OIII],\leavevmode\nobreak\
[OI]\leavevmode\nobreak\ \lambda\lambda 5007,6300$, $[NII]\leavevmode\nobreak\
\lambda\lambda 6548,6584,$ $[SII]\leavevmode\nobreak\ \lambda\lambda
6717,6731$ doublets, $H\alpha$, $H\beta$

Figure 5: Position of 1347 SDSS X-ray-selected galaxies on the
$[NII]/H{\alpha}$BPT diagram. The grey background circles show 204 895 SDSS-
DR17 galaxies that satisfy the conditions for optical spectra
$\sigma_{\mathrm{log}f}\leq 0.13$ and $\sigma_{line}<200\leavevmode\nobreak\
km/s$. Empirical lines (dashed blue and solid red) that separate LINERs from
Seyfert galaxies are from Kewley et al. (2006). The colour code represents the
hardness ratios among the HR1, HR2, HR3, and HR4 bands. Figure 6: Distribution
on the BPT diagram (Baldwin et al., 1981) of $1347$ SDSS X-ray-selected
galaxies. Empirical lines (dashed blue and solid red) that separate LINERs
from Seyfert galaxies are from Kewley et al. (2006). The grey background
circles show all the $204\leavevmode\nobreak\ 895$ SDSS-DR17 galaxies that
satisfy our basic optical quality cuts: all line fluxes have fractional
uncertainties of $<0.13$ dex (33%) and are ‘narrow’
$\sigma_{line}<200\leavevmode\nobreak\ km/s$. The colour bar represents
X-ray/H$\alpha$ flux ratio (6a) and redshift z (6b) and corresponds only to
$1347$ SDSS X-ray-selected galaxies.

We also cross-matched the X-ray catalogue with the full SDSS sample and then
applied the quality cuts after the matching. We ended up with nearly the same
sample, which implies that source confusion (matching the wrong optical source
to the X-ray source) is not a significant issue.

###  2.2 Sample of X-ray galaxies with emission lines and its general
properties

Our cross-match result is a sample of $1364$ emission line galaxies with
nuclear X-ray emission. Some of these matched sources could be incorrect
because the closest match is not necessarily the counterpart. In our previous
work, we carried out manual inspection of X-ray and optical sources from the
SIMBAD and XMM-IRAP databases (Zadorozhna et al., 2021). In the present study,
we quantified the potential contamination by shifting the positions of the
X-ray sources by random amounts that are larger than the source-matching
radius. After this, we again matched ‘shifted’ X-ray sources with SDSS DR17.
As a result, we only got 51 false matches, which corresponds to 3.7 %. We used
the optical emission-line fluxes from the SDSS DR17 data release to find the
positions of X-ray-selected galaxies on the BPT diagrams. This allows us to
determine the source of excitation in these galactic nuclei. After visual
inspection of the BPT diagram (see Figure A.1), we found that some galaxies
lie below the LINER position for most of the objects from the sample. We
separated these galaxies from the primary sample and reviewed every individual
galaxy (for more details, please see the Appendix). After careful checking, we
removed 17 galaxies from the primary sample. The last two galaxies, SDSS
J015308.29+010150.6 and SDSS J130340.81+534323.6, are located at the end of
the LINER region. We obtained a final sample that contains $1347$ X-ray-
selected galaxies at $z<0.35$ that covers the northern sky. This number
corresponds to $0.6\%$ of the SDSS galaxies with good optical spectra and
$0.2\%$ of the XMM-Newton X-ray sources, respectively.

For each galaxy, we included the following parameters: Right Ascension
($deg$), Declination ($deg$), redshift (z), optical emission line flux with
errors: $[OIII],[OI]\lambda 5007,6300$, $[NII]\lambda\lambda
6548,6584,[SII]\lambda\lambda 6717,6731$ doublets, $H\alpha$, $H\beta$ in
$10^{-17}erg\cdot s^{-1}\cdot cm^{-2}\cdot$Å-1, X-ray flux in the total photon
energy band with errors ($0.2-12keV$) from the XMM-DR10 catalogue
($erg/cm^{2}\cdot s^{-1}$), hardness ratio (HR), and the angular separation
between the position of the centre of a galaxy and the X-ray source.

Figures 1-3 present the general properties of the sample. Figure 1 shows the
distribution of $1347$ galaxies with nuclear X-ray emission in equatorial
coordinates. From this distribution, we see that SDSS X-ray-selected galaxies
cover the northern sky, and follow the distribution of SDSS galaxies. The
equator region with higher spatial density corresponds to Stripe 82 (LaMassa
et al., 2013, 2016). SDSS observed this region intensively during 2005, 2006,
and 2007. The zone of avoidance in the northern sky corresponds to the Milky
Way galaxy. The distribution of X-ray-selected galaxies in equatorial
coordinates is homogeneous over the northern sky. The distance between the
X-ray source and the optical centre of the galaxy is less than 1 arcsec apart,
for more than half of X-ray-selected galaxies (see Figure 2). The number of
X-ray galaxies rapidly decreases with angular separation from 1 arcsec to 3.5
arcsecs from $n=125$ at $r=1$ arcsec to $n=35$ at $r=3.5$ arcsec, and after
$r=3.5$ arcsec and then come to a plateau. We plot the distribution of $1347$
SDSS galaxies with a nuclear X-ray source as a function of redshift in Fig. 3.
The minimum and the maximum values are $\approx 0$ and $0.362$, respectively.
For galaxies located at higher redshift ($z>0.35$), $H_{\alpha}$ optical line
will be outside the SDSS cutoff wavelength of 9200Å. For $z>1.14$, $H_{\beta}$
is redshifted out of the wavelength range. The distribution of SDSS X-ray-
selected galaxies peaks at $z\sim 0.03$, and then the number of galaxies
decreases with increasing z. The mean and median values for the distribution
as a function of redshift are 0.076 and 0.064, respectively.

Figure 4 presents the distribution of the SDSS X-ray-selected galaxies as a
function of HR, showing HR1-HR4. The HR is the equivalent of a photometric
color index, and is calculated as a normalized difference of the exposure
corrected counts in two energy bands.

For HR1, the commonly used bands are 0.2 - 0.5 keV and 0.5 - 1.0 keV. For HR2,
the bands 0.5 - 1.0 keV and 1.0 - 2.0 keV are used. For HR3, 1.0 - 2.0 keV and
2.0 - 4.5 keV are used. Finally, for HR4, 2.0 - 4.5 keV and 4.5 - 12.0 keV are
used. For more information about the hardness ratio, readers can refer to the
XMM-Newton tools website 222https://xmm-
tools.cosmos.esa.int/external/sas/current/doc/emldetect/node3.html. From the
distribution in the BPT diagram of X-ray-selected galaxies with the HR colour
code, we conclude that there is no apparent correlation between the optical
type (star formation (SF), AGN, composite) of X-ray galaxies and their HR.
However, at the same time, we find a correlation between X-ray-to-$H\alpha$
flux ratio and optical type. This could be due to the location of the
$H\alpha$ emission line (accretion disc), while X-ray emission comes from the
corona.

From the general parameters of the sample, we conclude that most galaxies from
the sample belong to the local Universe, and the sample can be considered
homogeneous. A benefit of using this SDSS X-ray-selected galaxy sample is that
it covers almost all of the northern sky, and is therefore representative of
X-ray galaxies in the nearby Universe.

##  3 Results

In this section, we show how we investigated the position of X-ray-selected
galaxies in the BPT diagram as a function of two quantities: the X-ray
hardness ratio, HR1-4, and the X-ray-to-$H\alpha$ flux ratio. Figure 5
displays the distribution of the 1347 X-ray-selected galaxies on the
[NII]/$H\alpha$ version of the BPT diagram. The data points are colour coded
according to X-ray hardness ratios (HR1-HR4). Figure 6 presents the full range
of BPT diagrams ([NII]/$H\alpha$, [SII]/$H\alpha$, and [OI]/$H\alpha$ as the
x-axes). The colour code indicates the X-ray-to-H$\alpha$ flux ratio (Fig. 6a)
and the redshift $z$ (Fig. 6b). We also include a comparison with 204,895
galaxies from the SDSS-DR17 dataset in both figures. These galaxies meet the
criteria of $\sigma_{\mathrm{log}f}\leq 0.13$ for all lines and
$\sigma_{line}<200km/s$; they are plotted as a grey background in the
diagrams.

  * •

We confirm that X-ray-selected galaxies can lie in all regimes of the BPT
diagram: Seyfert 25%, LINERs 22%, composite (30%), and HII (23%). For the
sample of $204\leavevmode\nobreak\ 895$ SDSS galaxies with emission lines
(plotted as a grey background), AGNs make up only 13%, which is consistent
with Kewley et al. (2006). This fact has two immediate implications: X-ray-
selected emission-line galaxies have a much higher ($3.5\times$) incidence of
AGN than the parent sample of galaxies with narrow emission lines; and the
majority of X-ray-detected galaxies with narrow emission lines (at least in
our sample) are not AGN dominated according to their optical emission lines.

  * •

From the distribution in the BPT diagram of X-ray-selected galaxies with the
HR colour code, we conclude that there is no apparent correlation with HR.

  * •

We checked how many galaxies from our sample have the X-ray luminosity
$L_{X}>3\cdot 10^{42}$ erg/s, and can be classified as AGN according to
(Torbaniuk et al., 2021). According to this classification, we find 214 AGNs
among 1347 X-ray-selected galaxies (16%). The percentage of X-ray AGNs grows
from the star forming galaxy category to the Seyfert optical category: 6 from
316 (1.8%) star forming galaxies, 52 from 405 (12.8%) composite galaxies, and
176 from 626 (28%) Seyfert galaxies with LINERs. This means that X-ray AGNs
($L_{X}>3\cdot 10^{42}$ erg/s) can be found in all optical categories.
However, the highest percentage of X-ray AGNs is found in Seyfert galaxies and
LINERs. This result shows that X-ray luminosity is a necessary but
insufficient condition for identifying a source as an AGN.

  * •

We show, for the first time, that the X-ray-to-H$\alpha$ flux ratio is a good
predictor of the BPT regime in which a galaxy located: almost all the objects
with the highest X-ray-to-$H\alpha$ ratios,
$log_{10}({L_{X}/L_{H}}_{\alpha})>1.0,$ are located in the AGN regime. Almost
all the galaxies with low ratios, $log_{10}({L_{X}/L_{H}}_{\alpha})<1.0,$ are
located in the H II regime.

The distinct pattern in the colours of the data points (reflecting
$log_{10}(L_{X}/L_{H}\alpha)$ in Fig. 6a (top) reflects the power of the
$log_{10}(L_{X}/L_{H}\alpha)$ flux ratio as a predictor of the excitation
mechanism. The bottom panel of Fig. 6 explores whether aperture corrections or
the distance to the object play a significant role. The absence of a redshift
dependence in the BPT position implies that such effects are insignificant.

Figure 7: Positions of 1347 SDSS X-ray selected galaxies on X-ray versus
$H\alpha$ luminosities plane. X-ray luminosity is in the range 0.2-12 keV and
in $erg\cdot s^{-1}$, $H\alpha$ luminosity is in $erg\cdot s^{-1}\cdot$. Type
of ionisation (AGNs, Composite, SF) corresponds to the position of X-ray
galaxy on [OIII]/$H\beta$ versus [NII]/$H\alpha$ BPT diagram. AGNs are marked
with blue circles, Composites - with orange crosses, and SF - with green
squares.

To detect the type of ionisation that influences the X-ray and H$\alpha$
luminosities, we plot the distribution of SDSS X-ray-selected galaxies in the
X-ray versus $H\alpha$ plane in Fig. 7. The plot is similar to Fig. 6 but adds
additional information about the distribution of X-ray and $H\alpha$
luminosities for different types of excitation mechanisms in galaxies. In Fig.
7, we separate the whole sample of $1347$ SDSS X-ray-selected galaxies into
three groups (AGN, composite galaxies, and star-forming galaxies) according to
their position on the $[NII]/H\alpha$ diagram (source of ionisation). These
three groups are colour coded in Fig. 7 .

Figure 8: Distribution on the $log_{10}[OIII]$ versus $log_{10}H\alpha$ plane
of SDSS X-ray selected galaxies (left) and 204 895 SDSS galaxies (right). Type
of ionisation (AGNs, Composite, SF) corresponds to the position of X-ray
galaxy on [OIII]/$H\beta$ versus [NII]/$H\alpha$ BPT diagram.

From Fig. 7, we see that the peaks of the distribution of the X-ray-selected
galaxies correspond to different types of ionising sources. The first peak in
$H\alpha$ luminosity at $3\cdot 10^{40}$ corresponds to AGN, and the second
two belong to composite and star-forming galaxies. The distribution of
$H\alpha$ luminosity for composite and star-forming galaxies shows flat peaks
of about $10^{41}\leavevmode\nobreak\ erg/s$. The peak in the $H\alpha$
luminosity distribution for AGNs is one order less than for composite and
star-forming galaxies. The opposite situation is seen for X-ray luminosity.
From Fig. 7, we see that the maximum of the distribution of X-ray-selected
AGNs in the energy range of 0.2-12 keV corresponds to
$10^{42}\leavevmode\nobreak\ erg/s$, while for star-forming and composite
galaxies, the maximum of the distribution matches $10^{41}\leavevmode\nobreak\
\leavevmode\nobreak\ erg/s$. The distribution of X-ray-selected AGNs is almost
symmetrical to the black line in the centre of Figure 7, and star-forming and
composite X-ray-selected galaxies are located mostly below the black line.
AGNs are mostly brighter in X-rays than star-forming and composite galaxies.

Figure 9: BPT diagram for 1262 SDSS X-ray selected galaxies. Empirical lines
(dashed blue and solid red) that separate LINERs from Seyfert galaxies are
from Kewley et al. (2006). The relation between X-ray luminosity from the
stellar component and the total X-ray luminosity from the XMM-Newton
observations is shown with colour coding. Figure 10: BPT diagram for 1262 SDSS
X-ray-selected galaxies. The empirical lines (dashed blue and solid red) that
separate LINERs from Seyfert galaxies are from Kewley et al. (2006). The mass
of the stellar component (left) and the SFR (right) from Chang et al. (2015)
are shown with colour coding in the respective panels.

The distribution on the $log_{10}[OIII]$ versus $log_{10}H\alpha$ luminosity
plane of SDSS X-ray-selected galaxies (left) and 204 895 SDSS galaxies (right)
(See Fig. 8) shows the same general behaviour. The brightest in [OIII]
luminosity are AGNs in both samples with peak values of about
$10^{40}\leavevmode\nobreak\ erg/s$. Composite galaxies then follow, with peak
values of about $8\cdot 10^{39}\leavevmode\nobreak\ erg/s$. The weakest of the
three optical categories is the star-forming galaxies, with a peak in [OIII]
luminosity corresponding to $5\cdot 10^{39}\leavevmode\nobreak\ erg/s$. A
different situation is seen for $log_{10}H\alpha$ luminosity. The brightest
are composite galaxies with peak values of about $10^{41}\leavevmode\nobreak\
erg/s$ for both samples. The distributions of star-forming galaxies and AGNs
both show peaks at $5\cdot 10^{40}\leavevmode\nobreak\ erg/s.$ The main
difference between the two plots is that AGN dominate the distribution of
X-ray-selected galaxies, while star-forming galaxies dominate the SDSS sample.

Table 1: Distribution of $1347$ SDSS X-ray-selected galaxies and 204 895 SDSS galaxies (the last column), according to the their position on the BPT diagram Type of ionisation | $[NII]/H\alpha$ | $[SII]/H\alpha$ | $[OI]/H\alpha$ |  $[NII]/H\alpha$ 204 895 SDSS galaxies
---|---|---|---|---
Star-forming | 316 (23%) | 797 (59%) | 699 (52%) | 116 139 (57%)
Composite | 405 (30%) | 62 039 (30%)
Seyfert | 333 (25%) | 291 (22%) | 347 (26%) | 26 527 (13%)
LINERs | 293 (22%) | 259 (19%) | 301 (22%)

We calculated the part of X-ray emission from the stellar component in SDSS
X-ray-selected galaxies. To this end, we used the method described in Lehmer
et al. (2010):

$L_{HX}^{gal}=\alpha\cdot M_{*}+\beta\cdot SFR,$ (1)

where $\alpha=(9.05\pm 0.37)\cdot 10^{28}\leavevmode\nobreak\ erg\cdot
s^{-1}\cdot M^{-1}_{\odot}$ and $\beta=(1.62\pm 0.22)\cdot
10^{39}\leavevmode\nobreak\ erg\cdot yr\cdot s^{-1}\cdot M_{\odot}^{-1}$

$L_{HX}^{gal}$ traces the combined emission from high-mass X-ray binaries
(HMXBs) and low-mass X-ray binaries (Lehmer et al., 2010). This empirical
equation was obtained based on high-resolution Chandra observations.

For the $M_{*}$ and SFR in Eq. (1), we used data from Chang et al. (2015).

For 1262 SDSS X-ray-selected galaxies, we obtained the relation between X-ray
emission from X-ray binaries and the total X-ray emission from XMM-Newton
data. The results are plotted in Fig. 9.

The AGN component dominates the X-ray emission for almost the entire sample
$(L^{Star}_{XR}<0.5\cdot L^{Total}_{XR})$, except for 45 objects that are
preferentially located at the lower end of the BPT. These galaxies could be
pure X-ray AGN-free galaxies (Tugay & Vasylenko, 2011). The AGN dominance is
stronger for Seyferts than for LINERS and HII regions. To investigate this
question further, we plotted the star-formation rate (SFR) and masses of the
stellar component on the BPT diagram, using a color code (see Fig. 10). By
separating the stellar X-ray components, we see that LINERS have higher
stellar masses and much lower SFRs than Seyferts. In the HII region, objects
with a low SFR also have low masses.

##  4 Discussion and conclusion

It is commonly accepted that most X-ray emission in the Universe originates
from AGN but some amount could come from the stellar population of a host
galaxy. For a detailed study of the physical mechanism inside AGNs and star-
forming galaxies, separating stellar and AGN components in optical spectra is
essential. Contributions to the emission-line spectrum from both star
formation and the AGN are almost inevitable in many SDSS galaxies, given the
relatively large projected aperture size of the fibres ($5.5$ kpc diameter at
$z=0.1$; Kauffmann et al. (2003)).

The connection between star formation and the central SMBH has been
investigated in multiple studies. Stellar feedback from a young nuclear star
cluster has been shown to influence the interstellar medium at the centres of
AGN (Schartmann et al., 2009; Costa et al., 2014; Hopkins et al., 2016;
Shangguan et al., 2020). However, AGN located inside the galaxy can also
influence star formation in the host galaxy. These effects are called negative
and positive AGN feedback. Through such processes, AGN can suppress or trigger
star formation in the host galaxy (Croton et al., 2006; Ishibashi & Fabian,
2012; Cresci & Maiolino, 2018; Stemo et al., 2020).

The interdependence of the emission in X-rays and optical lines was discovered
at the end of the 20th century. For example, in luminous sources, strong
correlations exist between $H\alpha$, $H\beta$, $[OIII]\lambda 5007$
luminosities, and X-ray luminosities (Elvis et al., 1984; Ward et al., 1988;
Mulchaey et al., 1994). Many authors have used the X-ray-to-optical(continuum)
flux ratio to identify AGN in their host galaxies (Brandt & Hasinger, 2005;
Fitriana & Murayama, 2022; Torbaniuk et al., 2021). Brandt & Hasinger (2005)
list three AGN criteria:

\- an intrinsic X-ray luminosity, $L_{X,int}>=3\cdot
10^{42}\leavevmode\nobreak\ erg\cdot s^{-1}$;

\- an X-ray-to-optical flux ratio of $log_{10}\leavevmode\nobreak\
(f_{X}/f_{opt})>-1;$

\- an X-ray-to-IR flux ratio of $log_{10}(f_{X}/f_{K_{s}})>-1.2$.

The interdependence of X-ray and optical flux in the r-band was revealed based
on a study of a sample of 6181 X-ray sources from the Stripe-82X survey
(Ananna et al., 2017). The authors claim that objects with
$log_{10}\leavevmode\nobreak\ X/O>1$ and R-K(Vega magnitude)¿ 5 are candidates
for heavily obscured AGNs. Our investigation partially confirms this result.
Namely, we find that galaxies with $log_{10}(L_{X}/L_{H}\alpha)>1.0$ are
linked to LINER or Seyfert-type nuclear activity.

A series of works is devoted to BPT diagnostic diagrams for X-ray AGNs.
Birchall et al. (2022) present a sample of $917$ X-ray-selected AGN within the
local galaxy population. Similarly to the present work, these authors created
their sample from galaxies listed in SDSS DR8 and 3XMM DR7. The BPT diagnostic
diagram can be used to confirm an AGN inside a host galaxy. The BPT diagnostic
is more effective at reliably identifying sources as AGN in higher-mass
galaxies. These latter authors used this sample to calculate the incidence of
AGN as a function of stellar mass and redshift. Their main finding is that the
fraction of galaxies hosting AGN above a fixed specific accretion rate limit
of $10^{-3.5}$ is constant (at $\sim 1$ %) over stellar masses of
$8<\log_{10}M_{*}/M_{\odot}<$ 12 and increases (from $\sim 1$% to 10%) with
redshift.

Torbaniuk et al. (2021) used the sample of X-ray-selected galaxies to quantify
the level of AGN activity by applying multi-wavelength AGN-selection criteria
(optical BPT-diagrams, X-ray/optical ratio, etc.).

The present work is different because Torbaniuk et al. (2021) used flux in
only one line from two doublets, [NII] $\lambda 6583$ and [SII] $\lambda
6717,$ to find the position of a galaxy on the BPT diagrams. In our work, we
consider both doublet lines [NII] $\lambda\lambda$ 6548, 6584, [SII]
$\lambda\lambda$ 6717, 6731, as was done in the original work of Baldwin et
al. (1981). We restrict our sample to SDSS spectra whose fractional flux
uncertainties in all lines are $\leq 0.13$ dex.

For a more detailed investigation of the relationship between X-ray and
$H\alpha$ flux for AGNs and star-forming galaxies, we separated the sample of
1347 X-ray-selected galaxies into four groups according to their position on
the BPT diagram and the type of ionisation that dominates; these are star-
forming galaxies, composite galaxies, Seyfert galaxies, and LINERs (See Table
1). As Table 1 shows, 23% of the 1347 X-ray-selected galaxies belong to pure
galaxies with star-forming regions. Seyfert galaxies and LINERs make up less
than 50% of all X-ray galaxies and can be divided into almost equal parts of
about 20% each. In the original work of Kewley et al. (2006), the authors used
the sample of $85\leavevmode\nobreak\ 224$ galaxies, finding
$63\leavevmode\nobreak\ 893$ (75%) star-forming galaxies, $2411$ (3%)
Seyferts, $6005$ (7%) LINERs, and $5870$ (7%) composite galaxies, as well as
$7045$ (8%) galaxies of an ambiguous nature. We find that the fraction of
galaxies with an AGN type of nuclear activity is significantly higher among
X-ray-selected galaxies (47%) compared to that of the total sample of
$204\leavevmode\nobreak\ 895$ SDSS galaxies (about 13%, or three times lower;
see Table 1). This means that a major part of X-ray emission in galaxies comes
from central SMBHs. However, SMBHs are not the only source of X-ray emission
from galaxies; it can arise from star forming regions as well. When we compare
the sample of $204\leavevmode\nobreak\ 895$ SDSS galaxies with 1347 X-ray-
selected galaxies, we see that the two samples differ in their fraction of
star-forming galaxies. More than half (57%) of the SDSS-selected galaxies are
star-forming galaxies as defined based on their nuclear activity, while for
the X-ray-selected galaxies, this fraction is less than one-quarter (23%). We
conclude that most galaxies in the HII regime have no detectable X-ray
emission (weak X-ray emission from star formation may be present). We explore
the star-formation activity in a galaxy using the optical spectra, and the
$H_{\alpha}$ line in particular; from $[OIII]$, we can obtain information
about AGN activity Panessa et al. (2006).

In all these analyses involving the BPT diagram, we need to define a minimum
quality cut for the flux measurements of all emission lines to ensure
sufficient precision on the locations of all entries within the BPT diagram.
Therefore, we chose the weakest line with the lowest signal-to-noise ratio
(S/N) for the sample, namely [SII] (see Fig. 11). Following convention (e.g.
Bolton et al., 2012), we adopt a maximal fractional flux uncertainty of 33% or
0.13 dex (see Eq. 2 if $\frac{\sigma_{flux}}{f}<0.33,$) for the $H\alpha$,
[NII], [SII], and [OIII] lines. We find that at high S/N (S/N¿30, Fig. 12),
the fraction of composite galaxies increases significantly compared to SF and
Seyfert+LINER. Therefore, increasing the S/N ¿ 3 may influence the selection
criteria, as we will get a higher fraction of composite galaxies, which are
prevalent in the local Universe. The right panel of Figure 12 shows that the
probability density function for $\log(X-ray/H\alpha)$ has a more symmetrical
form with S/N ¿3 compared to S/N¿10. We conclude that S/N¿3 is an optimal
criterion that provides results of sufficient quality and allows us to create
the most homogeneous sample of objects with which to study their properties.

$\sigma_{\log f}=\frac{d\log f}{df}\cdot\sigma_{flux}=\frac{1}{\ln
10}\cdot\frac{\sigma_{flux}}{f}=\frac{0.3}{\ln 10}=0.13.$ (2)

Figure 11: Probability density function of S/N for [NII] and [SII] doublets
for 204 895 SDSS galaxy spectra. We choose the weakest line for S/N selection
criteria; that is, [SII].  Figure 12: Distribution of SDSS X-ray-selected
galaxies Left: Galaxies with S/N¿30 on the [SII] version of the BPT diagram
(Baldwin et al., 1981). Compared to Fig. 6, the contribution of composite-type
galaxies increases. The colour bar represents the X-ray/$H\alpha$ flux ratio.
Right: PDF of $\log(X-ray/H\alpha)$ for S/N 3 and 10, respectively.

Summarising our findings and conclusions, we find a clear dependence of the
BPT position on the X-ray-to-$H\alpha$ flux ratio. Sources dominated by X-ray
emission lie in the AGN and LINER regime of the BPT diagram. Almost all
sources with low X-ray-to-H$\alpha$ luminosity ratio,
$log_{10}(L_{X}/L_{H\alpha})<1.0$, lie in the HII regime. Our findings suggest
that the X-ray-to-$H\alpha$ ratio can be used to differentiate galaxies whose
X-ray flux is dominated by an AGN from those with central X-ray binaries and
other stellar X-ray sources. We find that from XMM-Newton observations of
Seyfert galaxies, the fraction of the total X-ray emission coming from the
stellar component is less than $1/30$. The galaxies with more than 50% of
their total X-ray emission coming from the stellar component
($log_{10}(L^{Star}_{XR}/L^{Total}_{Xray})>-0.3$ for 45 galaxies of our
sample) are characterised by a dominance of X-ray emission from the stellar
component. The galaxies for which the highest fraction of X-ray emission is
from the stellar component are located at the bottom of the BPT diagram and
are the galaxies that follow the green line that separates LINERs from
Seyferts. These galaxies could be candidates for pure X-ray AGN-free galaxies.

###### Acknowledgements.

Pulatova Nadiia is grateful to the Deutsche Forschungsgemeinschaft for
supporting the project and managing director of the Max Planck Institute for
Astronomy, Thomas Henning, for his kind invitation to work at the Institute,
which made this work possible. We are grateful to Igor Chilingarian, Andrea
Merloni and Andrey Zhiglo for their useful comments and advice on this work.
L. V. Zadorozhna is thankful to Dr. Chunshan Lin for his hospitality at the
Institute of Theoretical Physics, Jagiellonian University. L.V. Zadorozhna
gratefully acknowledges financial support from the National Science Center
Poland, grants No. UMO-2021/40/C/ST9/00015, UMO-2018/30/Q/ST9/00795. This
research was carried out with the support of the Center for the Collective Use
of Scientific Equipment ”Laboratory of High Energy Physics and Astrophysics”
of the Taras Shevchenko National University of Kyiv. The authors are grateful
to Alexander J. Dimoff (Max-Planck-Institut für Astronomie) for the helpful
suggestion, which has improved the English text of our paper. We thank the
anonymous referee for the helpful comments. This research has made use of data
obtained from the 4XMM XMM-Newton serendipitous source catalogue compiled by
the 10 institutes of the XMM-Newton Survey Science Centre selected by ESA.
Funding for the Sloan Digital Sky Survey IV has been provided by the Alfred P.
Sloan Foundation, the U.S. Department of Energy Office of Science, and the
Participating Institutions. SDSS-IV acknowledges support and resources from
the Center for High Performance Computing at the University of Utah. The SDSS
website is www.sdss.org. SDSS-IV is managed by the Astrophysical Research
Consortium for the Participating Institutions of the SDSS Collaboration
including the Brazilian Participation Group, the Carnegie Institution for
Science, Carnegie Mellon University, Center for Astrophysics — Harvard &
Smithsonian, the Chilean Participation Group, the French Participation Group,
Instituto de Astrofísica de Canarias, The Johns Hopkins University, Kavli
Institute for the Physics and Mathematics of the Universe (IPMU) / University
of Tokyo, the Korean Participation Group, Lawrence Berkeley National
Laboratory, Leibniz Institut für Astrophysik Potsdam (AIP), Max-Planck-
Institut für Astronomie (MPIA Heidelberg), Max-Planck-Institut für Astrophysik
(MPA Garching), Max-Planck-Institut für Extraterrestrische Physik (MPE),
National Astronomical Observatories of China, New Mexico State University, New
York University, University of Notre Dame, Observatário Nacional / MCTI, The
Ohio State University, Pennsylvania State University, Shanghai Astronomical
Observatory, United Kingdom Participation Group, Universidad Nacional Autónoma
de México, University of Arizona, University of Colorado Boulder, University
of Oxford, University of Portsmouth, University of Utah, University of
Virginia, University of Washington, University of Wisconsin, Vanderbilt
University, and Yale University.
Data availability
All the data presented and used in this paper are publicly available. The SDSS
data (DR17 release) are accessible through the online SQL search service on
the web page at http://skyserver.sdss.org/dr17/SearchTools/sql. The XMM-Newton
data are available in the XMM-Newton Survey Science Centre.

## References

  * Ananna et al. (2017) Ananna, T. T., Salvato, M., LaMassa, S., et al. 2017, ApJ, 850, 66
  * Baldwin et al. (1981) Baldwin, J. A., Phillips, M. M., & Terlevich, R. 1981, PASP, 93, 5
  * Birchall et al. (2022) Birchall, K. L., Watson, M. G., Aird, J., & Starling, R. L. C. 2022, MNRAS, 510, 4556
  * Blanton et al. (2017) Blanton, M. R., Bershady, M. A., Abolfathi, B., et al. 2017, AJ, 154, 28
  * Bolton et al. (2012) Bolton, A. S., Schlegel, D. J., Aubourg, É., et al. 2012, AJ, 144, 144
  * Brandt & Hasinger (2005) Brandt, W. N. & Hasinger, G. 2005, ARA&A, 43, 827
  * Brescia et al. (2015) Brescia, M., Cavuoti, S., & Longo, G. 2015, MNRAS, 450, 3893
  * Chang et al. (2015) Chang, Y.-Y., van der Wel, A., da Cunha, E., & Rix, H.-W. 2015, ApJS, 219, 8
  * Costa et al. (2014) Costa, T., Sijacki, D., Trenti, M., & Haehnelt, M. G. 2014, MNRAS, 439, 2146
  * Cresci & Maiolino (2018) Cresci, G. & Maiolino, R. 2018, Nature Astronomy, 2, 179
  * Croton et al. (2006) Croton, D. J., Springel, V., White, S. D. M., et al. 2006, MNRAS, 365, 11
  * Elvis et al. (1984) Elvis, M., Soltan, A., & Keel, W. C. 1984, ApJ, 283, 479
  * Fitriana & Murayama (2022) Fitriana, I. K. & Murayama, T. 2022, PASJ[arXiv:2203.15967]
  * Hopkins et al. (2016) Hopkins, P. F., Torrey, P., Faucher-Giguère, C.-A., Quataert, E., & Murray, N. 2016, MNRAS, 458, 816
  * Ishibashi & Fabian (2012) Ishibashi, W. & Fabian, A. C. 2012, MNRAS, 427, 2998
  * Kauffmann et al. (2003) Kauffmann, G., Heckman, T. M., Tremonti, C., et al. 2003, MNRAS, 346, 1055
  * Kewley et al. (2006) Kewley, L. J., Groves, B., Kauffmann, G., & Heckman, T. 2006, MNRAS, 372, 961
  * LaMassa et al. (2016) LaMassa, S. M., Urry, C. M., Cappelluti, N., et al. 2016, ApJ, 817, 172
  * LaMassa et al. (2013) LaMassa, S. M., Urry, C. M., Cappelluti, N., et al. 2013, MNRAS, 436, 3581
  * Lehmer et al. (2010) Lehmer, B. D., Alexander, D. M., Bauer, F. E., et al. 2010, ApJ, 724, 559
  * Mulchaey et al. (1994) Mulchaey, J. S., Koratkar, A., Ward, M. J., et al. 1994, ApJ, 436, 586
  * Panessa et al. (2006) Panessa, F., Bassani, L., Cappi, M., et al. 2006, A&A, 455, 173
  * Schartmann et al. (2009) Schartmann, M., Meisenheimer, K., Klahr, H., et al. 2009, MNRAS, 393, 759
  * Shangguan et al. (2020) Shangguan, J., Ho, L. C., Bauer, F. E., Wang, R., & Treister, E. 2020, ApJ, 899, 112
  * Stemo et al. (2020) Stemo, A., Comerford, J. M., Barrows, R. S., et al. 2020, The Astrophysical Journal, 888, 78
  * Strüder et al. (2001) Strüder, L., Briel, U., Dennerl, K., et al. 2001, A&A, 365, L18
  * Torbaniuk et al. (2021) Torbaniuk, O., Paolillo, M., Carrera, F., et al. 2021, MNRAS, 506, 2619
  * Tugay & Vasylenko (2011) Tugay, A. V. & Vasylenko, A. A. 2011, Odessa Astronomical Publications, 24, 72
  * Ward et al. (1988) Ward, M. J., Done, C., Fabian, A. C., Tennant, A. F., & Shafer, R. A. 1988, ApJ, 324, 767
  * Webb et al. (2020) Webb, N. A., Coriat, M., Traulsen, I., et al. 2020, A&A, 641, A136
  * Zadorozhna et al. (2021) Zadorozhna, L. V., Tugay, A. V., Shevchenko, S. Y., & Pulatova, N. G. 2021, Kinematics and Physics of Celestial Bodies, 37, 149

##  Appendix A Rejected objects

Figure A.1: The position on $[OIII]/H_{\beta}$ vs $[OI]/H_{\alpha}$ BPT diagram of selected SDSS galaxies with X-ray nuclear emission. Empirical lines (dashed blue and solid red) that separate LINERs from Seyfert galaxies are from Kewley et al. (2006). The green empirical line ($[OIII]/H_{\beta}<0.64+1.18\cdot[OI]/H_{\alpha}$) separates ”true” LINERs from ”doubtful” galaxies.  Table A.1: Information about 19 individual SDSS galaxies with nuclear X-ray emission, located under LINER position on BPT diagram Number | RA | DER | SDSS classification | remarks
---|---|---|---|---
1 | 16.70573 | 1.05625 | STARBURST BROADLINE |
2 | 28.28456 | 1.03075 | STARFORMING |
3 | 117.85459 | 17.51422 | STARFORMING | absent $H_{\beta}$
4 | 121.63670 | 15.26223 | ——- | absent $H_{\beta}$
5 | 155.91514 | 4.18629 | STARBURST BROADLINE |
6 | 167.80496 | 28.69641 | AGN BROADLINE |
7 | 168.96626 | 1.49863 | STARFORMING | absent [OI], S/N = 4
8 | 179.03689 | 56.77642 | ——- | absent $H_{\beta}$, [OIII]
9 | 183.24640 | 27.45127 | STARFORMING | absent [OIII], [OI]
10 | 195.92004 | 53.72324 | STARFORMING |
11 | 202.86815 | 25.06823 | ——- | absent emission lines
12 | 215.99761 | -0.89128 | AGN | absent $H_{\beta}$, [OIII], [OI]
13 | 224.31295 | 22.34289 | STARFORMING BROADLINE | emission lines has double peaks
14 | 233.22408 | 30.34984 | STARBURST | $\sigma_{line}\sim\leavevmode\nobreak\ 200\leavevmode\nobreak\ km/s$
15 | 243.92172 | 47.18661 | ——- | absent $H_{\beta}$, [OIII], [OI]
16 | 260.04183 | 26.62557 | STARFORMING BROADLINE | absent [OIII]
17 | 322.41647 | 0.08921 | STARFORMING | absent [OIII]
18 | 342.90562 | 1.19148 | ———- | absent $H_{\beta}$
19 | 358.52933 | -10.42132 | AGN BROADLINE | absent $H_{\beta}$

We checked all 19 X-ray-selected SDSS galaxies below the LINER’s position on
the BPT diagram. The empirical separation line between LINERs and ‘doubtful
galaxies’ marked with the green line,
$[OIII]/H_{\beta}<0.64+1.18\cdot[OI]/H_{\alpha}$ in Fig. A.1. We find that 12
of all 19 galaxies have no emission lines that we use to plot BPT diagrams
(see Table A.1). The $H_{\beta}$ line was mostly falsely detected, and its
flux was calculated automatically. Visually, this region appears to be a
random fluctuation of noise. We removed these galaxies from the sample.

The other four galaxies are classified automatically by SDSS (Brescia et al.
2015) as BROADLINE galaxies (with $\sigma_{line}>200\leavevmode\nobreak\
km/s$). We removed these galaxies from the sample as the physical conditions
that give rise to the broad lines are different from those that give rise to
the narrow lines, and so comparing these types of objects is misleading.

Finally, the remaining three galaxies were classified as STARBURST or
STARFORMING. The difference between the STARBURST and STARFORMING
classifications of galaxies is worthy of note. A STARBURST galaxy is a
STARFORMING galaxy with an equivalent width of $H\alpha$ of greater than $50$
Å. From these three STARBURST or STARFORMING galaxies (numbers 2, 10, 14 in
Table A.1) we find that the last one, 14, has
$\sigma_{line}>200\leavevmode\nobreak\ km/s$ for all-optical emission lines,
except $H\alpha$, $H\beta$ with $\sigma_{line}\sim 200\leavevmode\nobreak\
km/s$ ). According to this, we decided to remove galaxy 14 from Table A.1 as a
BROADLINE galaxy. The last two STARFORMING galaxies (SDSS J015308.29+010150.6
and SDSS J130340.81+534323.6) are located at the bottom end of the LINER
region and were not investigated further.


# See Beyond Seeing: Robust 3D Object Detection from Point Clouds
via Cross-Modal Hallucination

Jianning Deng2, Gabriel Chan3, Hantao Zhong4, and Chris Xiaoxuan Lu21
1Corresponding author. Email<EMAIL_ADDRESS>of Edinburgh,
United Kingdom. 3Connecticut College, USA. 4University of Cambridge, United
Kingdom.

###### Abstract

This paper presents a novel framework for robust 3D object detection from
point clouds via cross-modal hallucination. Our proposed approach is agnostic
to either hallucination direction between LiDAR and 4D radar. We introduce
multiple alignments on both spatial and feature levels to achieve simultaneous
backbone refinement and hallucination generation. Specifically, spatial
alignment is proposed to deal with the geometry discrepancy for better
instance matching between LiDAR and radar. The feature alignment step further
bridges the intrinsic attribute gap between the sensing modalities and
stabilizes the training. The trained object detection models can deal with
difficult detection cases better, even though only single-modal data is used
as the input during the inference stage. Extensive experiments on the View-of-
Delft (VoD) dataset show that our proposed method outperforms the state-of-
the-art (SOTA) methods for both radar and LiDAR object detection while
maintaining competitive efficiency in runtime.

##  I Introduction

Robust recognition and localization of objects in 3D space is a fundamental
computer vision task and an essential capability for intelligent systems. In
the context of autonomous driving, accurate 3D object detection is vital for
safe motion planning, especially in a complex urban environment. Due to
accurate depth measurement in long-range and robustness to illumination
conditions, ranging sensors such as LiDAR and radar have attracted increasing
attention recently and in turn, make the point clouds from them one of the
most commonly used data representations for 3D object detection.

Despite advancements in LiDAR-based [1, 2, 3, 4, 5, 6, 7, 8, 9, 10] and radar-
based 3D object detection [11, 12], each exhibits inherent drawbacks. LiDAR
excels at producing dense point clouds but lacks per-point semantic
information. Even though some LiDAR sensors can output point-level intensity
values [13, 14], they are often inconsistent across semantic labels as lidar
intensity is complexly determined by the distances between objects and LiDAR,
rather than the object type alone [15]. Conversely, the emerging 4D radars,
while prone to sparse data, present valuable semantic information per point,
such as the Radar Cross Section (RCS) and Doppler velocity. Unlike LiDAR
intensity, RCS is a _distance-independent_ measurement uniquely determined by
the object material and reflection angle. The Doppler velocity measures the
moving speed of a detected point relative to the ego vehicle. Benefiting from
the rich semantic information, fairly reasonable 3D object performance can be
achieved using 4D radars in complex urban environments [12], even if some
objects have less than 10 radar points.

(a) Proposed method for 4D radar object detection

(b) Centerpoint

(c) Pointpillar

(d) PV-RCNN

(e) Ours

Figure 1: Fig. 1(a) illustrates the proposed method with the 4D radar input as
an example. Fig. 1(b) \- Fig. 1(e) are the visualization of radar detection in
the same scene of different methods. Ground truth boxes are denoted in red,
the wrong detections are denoted in yellow and the correct detections are
denoted in green. RGB images are only used for visualization.

Given the complementary nature of these sensors, we explore if one can
assimilate the characteristics of the other while enhancing independent
detection capacities. Indeed, due to the cost consideration, many low-end
vehicles are only equipped with either radars or LiDARs. It is thus valuable
to train a _single-modal_ detection model from the _multimodal_ data collected
by some pilot vehicles equipped with both sensors, yet dispatch the trained
models on low-end vehicles equipped with only one of them. Addressing this
necessitates delving into cross-modal learning. Prior arts achieve this via
either hallucination for feature augmentation [16, 17, 18, 19, 20] or
knowledge distillation for backbone refinement [21, 22, 23, 24]. However, the
learning direction in these prior works is _unidirectional_ passing from the
strong modality to the weak modality (e.g., RGB camera $\rightarrow$ depth
camera), insufficient to meet our goal.

Unlike prior arts, in this work, we aim to realize a new cross-modal
hallucination framework that can work in agnostic learning directions (i.e.,
LiDAR $\leftrightarrow$ radar). To bridge the intrinsic sensing gap between
two modalities, we introduce a novel selective matching module to enable
feature-level alignment in the shared latent space. Our design enhances intra-
modality features through cross-modal learning and supplements inter-modality
features for improved detection robustness (see Fig. 1). Our method
outperforms previous state-of-the-art (SOTA) methods in 3D object detection,
which we demonstrate on the public View-of-Delft dataset in both LiDAR and
radar object detection tasks. Our code will be publicly released upon
acceptance.

Figure 2: Method Overview. The upper figure illustrates the 2-step training
strategy, blocks used in the first step of training are connected with green
line and those for the second step are connected with orange line. Note the
primary and auxiliary data can be interchangeable among two sensor modalities
(radar and LiDAR) depending on the end goal. Only single modal data (primary
modal) will be used during inference as shown in the lower figure connected
with blue line.

##  II Related work

LiDAR object detection. LiDAR 3D object detection is categorized into voxel-
based, point-based, and point-voxel approaches. Voxel-based methods transform
point cloud data into grid cells, suitable for convolution operations [25, 26,
9, 1]. For example, [1] converts point clouds into 2D pseudo-images for faster
inference, while [9] adopts a two-stage process, improving detection at a
higher memory cost. Point-based techniques directly process unstructured point
cloud data, minimizing data loss [2, 8, 7, 27]. Models in this category, such
as [2, 7], utilize methodologies from [28] and [29] to extract features.
Recent research introduces point-voxel networks that combine the best of both
methods for enhanced accuracy [5, 3, 10, 30]. As highlighted by [7], these
networks, like [5], perform well on the KITTI dataset [13], though they
require more computation resources.

Radar object detection. Early efforts in radar-based object detection focused
on 2D object detection [31, 32, 33]. It is only with the recent availability
of 4D radar sensors that radar 3D object detection began receiving attention
from researchers. [11] is an anchor-based 3D detection framework with a
PointNet style backbone. It was only evaluated on short-range radar point
clouds within $\sim$ 10 meters in front of the ego-vehicle, which is far too
close for real-world autonomous scenarios. On the other hand, authors of the
recently published dataset [12] successfully repurposed PointPillars [1] on
radar point clouds. Although the 3D object detection architecture was designed
for LiDAR point clouds, it achieved SOTA performance on their radar 3D object
detection benchmark [12].

Cross-modal feature augmentation. Learning with side information involves
using supplementary data during training to enhance single-modality networks
[17, 19, 20, 16, 18, 22, 23, 21]. [18] pioneered using cross-modal
hallucination in object detection by enhancing RGB image detection using depth
image features. [16] integrated thermal data for RGB pedestrian detection.
Recent works, like [21], utilize multi-modal features via knowledge
distillation to improve 3D detection. However, many of these methods do not
maximize cross-modal information use. For instance, [16, 18] require an extra
backbone for hallucination instead of optimizing the original. Knowledge
distillation techniques [22, 23, 21], while strengthening the backbone,
overlook the potential of hallucinated data.

##  III Method

The proposed method is depicted in Fig. 2. Our framework includes (i) a point-
based backbone for feature extraction, (ii) an instance feature aggregation
module for aligning input modalities, (iii) a hallucination branch for cross-
modal feature alignment, and (iv) a detection head for bounding box
prediction. The selective matching module and shared detection head are
training-specific. We use ‘primary’ and ‘auxiliary’ interchangeably to denote
our two sensor modalities for clarity. At a high-level summary of our method,
a backbone network takes point set $\mathbf{P}$ of the primary modality and
outputs a subset $\mathbf{P}_{f}$ for the sampled foreground points with
extracted features. For the auxiliary-modal data, we use a hat notation, such
as $\hat{\mathbf{P}}$. The instance feature aggregation module processes
$\mathbf{P}_{f}$ to obtain ‘centered points’ $\mathbf{C}$, adjusting point
positions with a spatial offset $\tilde{o_{i}}$. Features from both modalities
are then aligned via non-linear mapping for hallucination, constrained by
instance location, resulting in hallucination features $\mathbf{H}$. These are
then concatenated with $\mathbf{C}$ and fed into the detection head for final
object recognition.

###  III-A Backbone Network

We construct our backbone network based on the Set-Abstraction (SA) layer
proposed in PointNet++ [29] for better efficiency and to avoid information
loss [8, 7]. Additionally, the farthest-point sampling (FPS) operation in SA
layer is replaced with center-aware sampling proposed in [7] for better
performance, which selects top-$k$ points based on the predicted centeredness.
This predicted centeredness is constrained during training as:

$\mathcal{L}_{Ctr}=-\sum^{N}_{k=1}(Mask_{k}\cdot
y_{k}log(\tilde{y}_{k})+(1-y_{k})log(1-\tilde{y}_{k}))$

(1)

with $y_{k}$ as the ground truth centeredness and $\tilde{y}_{k}$ as network
estimated value. $Mask_{k}$ is the corresponding mask value for each point
proposed in [8], which assigns higher weights to points closer to the centroid
of objects, and no weight at all for background points. The output of the
backbone is denoted as
$\mathbf{P}_{f}=\\{p_{i}\\}\in\mathbb{R}^{N_{f}\times(3+D)}$ with $N_{f}$
points and $p_{i}=[t_{i},f_{i}^{D}]$.

(a) Inputs

(b) Backbone results

(c) Centered

Figure 3: Point-level matches are difficult to obtain without centroid
generation in the instance feature aggregation module due to sparsity. The
black bounding boxes are the object ground truth labels. (Best viewed in color
and zooming in).

###  III-B Instance Feature Aggregation

Single-stage detectors face challenges in point matching across different
sensor modalities due to discrepancies between point clouds from co-located
LiDAR and radar, evident in Fig.3(a). This difference, arising from
fundamental sensor characteristics and unsolvable by calibration, is worsened
by sampling randomness, especially in the backbone’s foreground sampling (c.f.
Fig. 3(b)). To mitigate this, we use instance feature aggregation for improved
alignment before cross-modal matching. This process is illustrated in Fig.
3(c). Following [34], we generate ‘centered points’ for context. Each
foreground point, $p_{i}\in\mathbf{P}_{f}$, gets an offset,
$\tilde{o}_{i}\in\mathbb{R}^{3}$, guiding it towards its object center. We
constrain this regression process with a smooth-L1 loss as:

$\mathcal{L}_{O-Reg}=\frac{1}{\sum_{i}\mathbf{I}(p_{i})}\sum_{i=1}^{m}smooth_{L1}(o_{i}-\tilde{o}_{i})\cdot\mathbf{I}(p_{i})$
(2)

here $o_{i}$ is the true offset, and $\mathbf{I}(p_{i})$ checks if $p_{i}$ is
within an object. Points after this instance-aggregation step is denoted as
$\mathbf{C}=\\{c_{i}\\}\in\mathbb{R}^{N_{f}\times(3+D)}$, with each having a
new position $t_{i}^{\prime}=t_{i}+\tilde{o_{i}}$. Since points now cluster
near the instance centroid, the subsequent SA layer’s ball query often groups
the same points, resulting in almost identical instance features after max
pooling. Thus, we can quickly establish instance matches between modalities by
matching points in close spatial locations.

###  III-C Cross-Modal Feature Hallucination

Different sensing principles make LiDAR and radar instance features modality-
specific. For instance, radar measures relative Doppler velocity [35, 36, 14],
while LiDAR provides detailed object geometry. Given these intrinsic
differences, it’s impractical to directly match all features across
modalities. A solution is to ground cross-modal learning in a shared subspace,
focusing on a subset of their features (the magenta block in Fig. 2). This
mandates a feature projection module, allowing feature embedding in a shared
latent space for both modalities. Specifically, given the instance-aggregated
point set $\mathbf{C}\in\mathbb{R}^{N_{f}\times(3+D)}$ of the primary modality
in the domain $X$ and clustered point set
$\hat{\mathbf{C}}\in\mathbb{R}^{\hat{N}_{f}\times(3+\hat{D})}$ of the
auxiliary modality in domain $\hat{X}$, we have two mapping functions acting
as the feature projection: $F_{pri}:X\rightarrow H$ and
$F_{aux}:\hat{X}\rightarrow\hat{H}$ to project both modalities to a shared
common space. We adpot the MLP block for each of the mapping functions which
yields two hallucination feature sets
$\mathbf{H}=\\{h_{i}\\}\in\mathbb{R}^{N_{f}\times F}$ and
$\mathbf{\hat{H}}=\\{h_{i}\\}\in\mathbb{R}^{\hat{N_{f}}\times F}$
respectively. The dimension of the shared common subspace, $F$, is set
empirically.

###  III-D Selective Matching

Using the projected hallucination features, we must match pairs between
modalities for training, as shown by the purple block in Fig.2. As we note
that noisy and incorrectly classified points tend to cluster at random spots,
our approach involves a cross-modal Nearest Neighbor (1-NN) search within a
specific radius to pinpoint the right matches, as shown in Fig. 4(a). Notice
that only foreground points successfully shifted towards object centers can
reach neighbor points from the other modality, as they are close in space.
This proximity not only aids in efficient cross-modal representation learning
but also reduces distractions from noisy data points. The cross-modal
hallucination loss is as follows:

$\centering\mathcal{L}_{FM}=\frac{1}{N_{p}}\sum_{i=0}^{N_{f}}\sum_{j=0}^{\hat{N}_{f}}||h_{i}-\hat{h}_{i}||_{2}\cdot\mathbf{PM}_{ij}\@add@centering$
(3)

where $\mathbf{PM}\in\mathbb{R}^{N_{f}\times\hat{N}_{f}}$ is a binary matrix,
and $\mathbf{PM}_{ij}=1$ when $c_{i}$ and $\hat{c}_{j}$ are a selected
matching pair, otherwise $\mathbf{PM}_{ij}=0$. $N_{p}$ denotes the total
number of matched pairs.

(a) Illustration of selective matching. Points with lighter colors on the
right are ignored in the match-searching process.

(b) Visualization of the training process for selective matching.

Figure 4: Here is the illustration and visualization of the selective matching
during training. In Fig. 4(b), we can see that all matched points (cyan
points) are positioned near the bounding box center for co-visible objects,
which demonstrates the effectiveness of the instance feature aggregation
module. For wrongly sampled centered points, they will be moved to random
positions and will not interfere with the training process (magenta points).

###  III-E Detection Head

Following [8, 7], we encode bounding boxes as multidimensional vectors
comprising locations, scale, and orientation. The detection head, shown as the
blue block in Fig. 2, has branches for confidence prediction and box
refinement. The detection head’s loss function is:

$\mathcal{L}_{Det}=\mathcal{L}_{ref}+\mathcal{L}_{cls}$ (4)

To avoid trival solution for feature matching in Eq. 3, we use an extra
detection head specifically for the shared space features, called shared
detection head (the orange block in Fig. 2). This loss term is denoted as
$\mathcal{L}_{S-Det}$.

###  III-F Training and Inference

We adopt a two-step training strategy for stability. Unlike other methods [17,
16], our hallucination branch is co-optimized with the backbone in the second
phase. In the first step, we train a basic detection network (shown by the
green line in Fig.2) with loss:

$\mathcal{L}_{s1}=\mathcal{L}_{Ctr}+\mathcal{L}_{O-Reg}+\mathcal{L}_{Det}$ (5)

In the second step, parameters from the first phase initialize the backbone
and feature aggregation module. While the auxiliary backbone remains static,
the primary one undergoes joint refinement with the hallucination module. A
new detection head addresses the augmented feature space (represented by the
orange line in Fig.2). The associated loss is:

$\mathcal{L}_{s2}=\mathcal{L}_{s1}+\lambda_{1}\mathcal{L}_{FM}+\lambda_{2}\mathcal{L}_{S-Det}$
(6)

When it comes to inference, only the primary modality data is applied, as
shown by the blue line in Fig. 2.

##  IV Evaluation

###  IV-A Experimental Setup

Dataset. The View-of-Delft (VoD) dataset [12] offers calibrated, synchronized
data from LiDAR, RGB camera, and 4D radar for 3D object detection, comprising
8693 frames captured in Delft’s urban environment. Unique to VoD is its
inclusion of vulnerable road users like pedestrians and cyclists. Notably, VoD
alone provides public access to simultaneous 4D radar recordings and LiDAR
data and is thus selected for evaluation. While datasets such as nuScenes [14]
have both radar and LiDAR data, their radars only offer 2D spatial
measurements, limiting them for 3D object detection.

Implementation details. We follow the design of the backbone network in [7],
which comprises several _SA layers_ with center-aware sampling to remove noisy
points. Next, the _Vote Layer_ predicts an offset for each point to
concentrate them on the corresponding object center for spatial alignment in
centroid generation. After that, another _SA layer_ is employed to aggregate
instance-level features. Once we obtain point clusters for different objects,
we project instance-level features to a shared subspace using a 4-layer MLP.
These projected features are later concatenated back to the instance-level
features for bounding box classification and regression. We use
$\lambda_{1}=\frac{1}{3}$ and $\lambda_{2}=\frac{2}{3}$ in Eqn. 6 for all our
experiments. Only the primary modal data will be used during inference. The
best LiDAR and radar detection models are selected based on the best
validation result for their respective detection tasks.

rowsep=1pt,colsep=2pt  Method Type Car Pedestrian Cyclist mAP (IoU = 0.5) (IoU
= 0.25) (IoU = 0.25) PointPillars†[1] V 35.90 34.90 43.10 38.00 SECOND[26]
35.07 25.47 33.82 31.45 CenterPoint[9] 32.32 17.37 40.25 29.98 PV-RCNN[5] PV
38.30 30.79 46.58 38.56 PointRCNN[2] P 15.99 34.01 26.50 25.50 3DSSD[8] 23.86
9.09 32.20 21.71 IASSD[7] 31.33 23.61 49.58 34.84 Ours 32.32 42.49 50.49 41.77

TABLE I: Test results for radar object detection on VoD dataset. Note that
results for PointPillars with symbol † are reported in [12]. The _‘Type’_
column denotes the data representation used in the method: _‘V’_ denotes
voxel, _‘P’_ denotes point, _‘PV’_ denotes point-voxel. Methods underlined are
all two-stage detectors.

rowsep=1pt,colsep=2pt  Method Type Car Pedestrian Cyclist mAP (IoU = 0.5) (IoU
= 0.25) (IoU = 0.25) PointPillars†[1] V 75.60 55.10 55.40 62.10 SECOND[26]
77.69 59.95 65.50 67.71 CenterPoint[9] 68.29 66.90 64.42 66.54 PV-RCNN[5] PV
75.16 65.24 66.09 68.83 PointRCNN[2] P 61.51 67.36 67.03 65.30 3DSSD[8] 77.34
12.64 37.68 42.55 IASSD[7] 77.29 32.18 57.11 55.53 Ours 79.74 60.58 68.52
69.62

TABLE II: Test results for LiDAR object detection on VoD dataset. Note that
results for PointPillars with symbol † are reported in [12]. The _‘Type’_
column denotes the data representation used in the method: _‘V’_ denotes
voxel, _‘P’_ denotes point, and _‘PV’_ denotes point-voxel. Methods underlined
are all two-stage detectors.

###  IV-B Overall Results

Tab. I and Tab. II show that our proposed framework gives rise to the best
overall performance for both modalities, achieving $41.77$ mAP and $69.62$ mAP
for radar and LiDAR object detection, respectively. Fig. 5 illustrates the
qualitative results of our method.

Radar Object Detection. Despite the challenges of sparse and noisy radar point
clouds, our detection method significantly outperforms SOTA techniques. PV-
RCNN [5] excels in car detection due to reduced size ambiguity in voxel
representation on sparse radar clouds. Yet, our method excels in detecting
smaller objects like cyclists and pedestrians. Specifically, our model scores
42.49 mAP for pedestrians, outpacing the PV-RCNN by roughly 21.7%. This
suggests that better representation for small objects with sparse clouds can
be gleaned from cross-modal data, especially LiDAR. With our hallucination
branch detailed in Sec. III-C, we encourage the radar detection network to
emulate LiDAR’s instance representation, and our results affirm its efficacy.

LiDAR Object Detection. In LiDAR detection, our technique leads in car and
cyclist categories, surpassing the runner-up by 2.6% and 2.2% mAP
respectively. This boost stems from the Radar Cross Section (RCS) features’
semantic cues, distinguishing between metal and human skin[37, 38]. While two-
stage detectors, such as [9, 5, 2], often excel in pedestrian detection
through second-stage ROI refinement, they come with larger memory and slower
speeds.

(a) Radar detection in perspective

(b) Radar detection

(c) LiDAR detection

Figure 5: Qualitative result of our method. Bounding boxes for GTs are denoted
in red, and the predictions are denoted in green. The left images and the
middle figures are the radar detection results. Notice that the RGB images
here are only for visualization purposes but not used in model
training/inference. The right figures visualize the LiDAR point clouds and the
prediction results.

###  IV-C Ablation Study

To be consistent with LiDAR detection work, the more demanding 3D IoU in KITTI
[13] with 0.7, 0.5, and 0.5 is used for LiDAR evaluation and 0.5, 0.25, 0.25
for radar evaluation on the VoD [12] _validation_ set.

Effect of auxiliary-modal attributes. Through experiments with different
auxiliary modal data inputs, we assess their influence on radar object
detection using cross-modal supervision. Tab. III shows that radar detection
improves when incorporating the LiDAR point cloud’s geometric information
($x$, $y$, $z$). The LiDAR intensity attribute has a slight impact, given
radar’s stable semantic cues from RCS measurements. The Car category’s modest
performance gain is due to around 25% of cars lacking radar points, hindering
detection. Moreover, cars’ larger size versus cyclists and pedestrians
complicates bounding box estimation with few points.

rowsep=1pt,colsep=2pt  Auxiliary LiDAR Car Pedestrian Cyclist mAP point
attributes (IoU=0.5) (IoU=0.25) (IoU=0.25) \+ none (baseline) 31.24 32.50
60.69 41.48 \+ $x,y,z$ 31.32 40.04 67.78 46.38 \+ $x,y,z,I$ 32.20 40.42 68.67
47.03

TABLE III: Radar detection results in _validation set_ supervised by different
LiDAR attributes during training. $x,y,z$ for point position, $I$ for
intensity.

Effect of joint optimization. We study the effectiveness of our joint
optimization strategy.

(a). Setup. We use the base network trained without cross-modal supervision
and the hallucination branch as the ablation baseline. Later, we remove the
hallucination branch of our framework shown in Fig. 2, freeze the backbone
weights, and train a new detection head for performance comparison.

(b). Results. Looking at the first two rows in Tab. IV, mAP increases in both
modalities, even without the hallucination branch. Significant improvements
are seen with the Cyclist on radar and Pedestrian on LiDAR. For LiDAR, fewer
car matches might cause a minor decline with only BR (Backbone Refinement),
but other categories enhance. To understand why only refined backbone improves
performance, we collected backbone output statistics. We can see from Fig. 6
that the percentage of instances containing more than five points gets
improved. We propose that cross-modal representation learning enhances the
backbone network’s feature extraction. These robust features are partially
propagated to close points via max pooling, retaining more significant
instance points in the final sampling layer, thus creating a robust instance
feature representation for improved detection. Furthermore, the last row in
each modality shows that the entire network with the hallucination branch
leads to the best performance for all three categories, indicating its
effectiveness and that extra cross-modal features can be encoded in our model
for better detection inference.

(a) Radar backbone statistics

(b) LiDAR backbone statistics

Figure 6: Percentage of instance containing 5-20 points in the backbone
output. Left for radar, right for LiDAR, The BR denotes backbone refinement.

rowsep=1pt,colsep=2pt  Modality BR HA Car Pedestrian Cyclist mAP (IoU=0.5)
(IoU = 0.25) (IoU = 0.25) radar 31.24 32.50 60.69 41.48 $\checkmark$ 31.51
33.48 67.19 43.94 $\checkmark$ $\checkmark$ 32.20 40.42 68.67 47.03 (IoU=0.7)
(IoU = 0.5) (IoU = 0.5) mAP LiDAR 57.94 29.40 68.26 51.87 $\checkmark$ 55.72
48.64 76.22 60.20 $\checkmark$ $\checkmark$ 58.82 56.29 78.17 64.42

TABLE IV: Evaluation on the validation set. BR: network with backbone
refinement which optimizes the backbone in the second step training with
cross-modal supervision, HA: network with hallucination branch.

Effect of two-level alignment. We next study the LiDAR object detection
results to better understand the influence of our alignment strategies
introduced in Sec. III-B and Sec. III-C. Tab. V shows that spatial alignment
in centroid generation is the key to cross-modal learning. When removed, the
discrepancy between modalities results in mismatching pairs. Misleading
supervision signals caused by mismatched pairs degenerate the network
performance, especially for large objects like Cars. When the spatial
alignment in centroid generation is added, the model establishes correct
cross-modal instance matches and improves mAP, as shown in the third row of
Tab. V. We attribute the slight performance drops on cars to fewer matched
instances in this category. As expected, the feature alignment will only
function and positively contribute to the model when the cross-modal points
are spatially aligned first. The last row shows that combining two alignment
operations yields the best performance. Similar conclusions are observed for
radar detection, and we omit its discussion to avoid repetition.

rowsep=1pt,colsep=2pt  Modality Spatial Feature Car Pedestrian Cyclist mAP
(IoU=0.7) (IoU=0.5) (IoU=0.5) LiDAR 57.94 29.40 68.26 51.87 $\checkmark$ 28.88
30.22 64.33 41.15 $\checkmark$ 57.29 48.66 77.70 61.22 $\checkmark$
$\checkmark$ 58.82 56.29 78.17 64.42

TABLE V: Ablation on the effect of spatial alignment in centroid generation
and feature level alignment to the cross-modal hallucination. Notice that we
use the basic model trained without any cross-modal information in Tab. IV as
the baseline performance.

###  IV-D Runtime Efficiency

rowsep=1pt,colsep=2pt  Type Method Memory Parallel Speed Voxel-based
PointPillars[1] 354MB 69 123 SECOND[26] 710MB 34 34 Point-Voxel PV-RCNN[5]
1223MB 17 13 Point-based pointRCNN[2] 560MB 43 14 3DSSD[8] 502MB 48 20 IA-
SSD[7] 120MB 202 23 Ours 133MB 183 23

TABLE VI:  Comparing memory usage and runtime efficiency with LiDAR input.
Memory for each method is measured using the same 16384 points per scan.
’Parallel’ refers to the maximum batch size for one RTX 3090. Speed is gauged
with single scan input and reported in frames per second. Our method’s results
are highlighted in bold.

We evaluate the memory consumption and inference speed compared with SOTA
methods with LiDAR as the primary modal input. As shown in Tab. VI, our method
has the second lowest memory footprint among all methods and the fastest
inference speed in the point-based methods. Together with Tab. II and Tab. I,
our framework has the best balance between efficiency and accuracy.

##  V Conclusions and Future Work

This work introduced a novel framework to improve the robustness of single-
modal 3D object detection via cross-modal supervision. Our method is able to
effectively exploit the side information from auxiliary modality data for a
better-informed backbone network and a robust hallucination branch. Bespoken
spatial and domain alignment strategies are also proposed to address the
fundamental discrepancy across modalities and showed a significant performance
improvement. Experimental results on VoD[12] demonstrate that our method
outperforms SOTA methods in object detection for both radar and LiDAR while
maintaining a competitive memory footprint and runtime efficiency.

## References

  * [1] A. H. Lang, S. Vora, H. Caesar, L. Zhou, J. Yang, and O. Beijbom, “Pointpillars: Fast encoders for object detection from point clouds,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2019, pp. 12 697–12 705.
  * [2] S. Shi, X. Wang, and H. Li, “Pointrcnn: 3d object proposal generation and detection from point cloud,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2019, pp. 770–779.
  * [3] Z. Yang, Y. Sun, S. Liu, X. Shen, and J. Jia, “Std: Sparse-to-dense 3d object detector for point cloud,” in _Proceedings of the IEEE/CVF international conference on computer vision_ , 2019, pp. 1951–1960.
  * [4] Z. Liu, X. Zhao, T. Huang, R. Hu, Y. Zhou, and X. Bai, “Tanet: Robust 3d object detection from point clouds with triple attention,” _Proceedings of the AAAI Conference on Artificial Intelligence_ , vol. 34, no. 07, pp. 11 677–11 684, Apr. 2020. [Online]. Available: https://ojs.aaai.org/index.php/AAAI/article/view/6837
  * [5] S. Shi, C. Guo, L. Jiang, Z. Wang, J. Shi, X. Wang, and H. Li, “Pv-rcnn: Point-voxel feature set abstraction for 3d object detection,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2020, pp. 10 529–10 538.
  * [6] S. Shi, Z. Wang, J. Shi, X. Wang, and H. Li, “From points to parts: 3d object detection from point cloud with part-aware and part-aggregation network,” _IEEE transactions on pattern analysis and machine intelligence_ , vol. 43, no. 8, pp. 2647–2664, 2020.
  * [7] Y. Zhang, Q. Hu, G. Xu, Y. Ma, J. Wan, and Y. Guo, “Not all points are equal: Learning highly efficient point-based detectors for 3d lidar point clouds,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 18 953–18 962.
  * [8] Z. Yang, Y. Sun, S. Liu, and J. Jia, “3dssd: Point-based 3d single stage object detector,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2020, pp. 11 040–11 048.
  * [9] T. Yin, X. Zhou, and P. Krahenbuhl, “Center-based 3d object detection and tracking,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2021, pp. 11 784–11 793.
  * [10] J. Noh, S. Lee, and B. Ham, “Hvpr: Hybrid voxel-point representation for single-stage 3d object detection,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 14 605–14 614.
  * [11] K. Bansal, K. Rungta, S. Zhu, and D. Bharadia, “Pointillism: Accurate 3d bounding box estimation with multi-radars,” in _Proceedings of the 18th Conference on Embedded Networked Sensor Systems_ , 2020, pp. 340–353.
  * [12] A. Palffy, E. Pool, S. Baratam, J. F. Kooij, and D. M. Gavrila, “Multi-class road user detection with 3+ 1d radar in the view-of-delft dataset,” _IEEE Robotics and Automation Letters_ , vol. 7, no. 2, pp. 4961–4968, 2022\.
  * [13] A. Geiger, P. Lenz, and R. Urtasun, “Are we ready for autonomous driving? the kitti vision benchmark suite,” in _2012 IEEE Conference on Computer Vision and Pattern Recognition_ , 2012, pp. 3354–3361.
  * [14] H. Caesar, V. Bankiti, A. H. Lang, S. Vora, V. E. Liong, Q. Xu, A. Krishnan, Y. Pan, G. Baldan, and O. Beijbom, “nuscenes: A multimodal dataset for autonomous driving,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2020, pp. 11 621–11 631.
  * [15] P. Chazette, J. Totems, L. Hespel, and J.-S. Bailly, “5 - principle and physics of the lidar measurement,” in _Optical Remote Sensing of Land Surface_ , N. Baghdadi and M. Zribi, Eds. Elsevier, 2016, pp. 201–247. [Online]. Available: https://www.sciencedirect.com/science/article/pii/B9781785481024500053
  * [16] D. Xu, W. Ouyang, E. Ricci, X. Wang, and N. Sebe, “Learning cross-modal deep representations for robust pedestrian detection,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 5363–5371.
  * [17] M. R. U. Saputra, P. P. de Gusmao, C. X. Lu, Y. Almalioglu, S. Rosa, C. Chen, J. Wahlström, W. Wang, A. Markham, and N. Trigoni, “Deeptio: A deep thermal-inertial odometry with visual hallucination,” _IEEE Robotics and Automation Letters_ , vol. 5, no. 2, pp. 1672–1679, 2020.
  * [18] J. Hoffman, S. Gupta, and T. Darrell, “Learning with side information through modality hallucination,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016, pp. 826–834.
  * [19] J. Lezama, Q. Qiu, and G. Sapiro, “Not afraid of the dark: Nir-vis face recognition via cross-spectral hallucination and low-rank embedding,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 6628–6637.
  * [20] C. Choi, S. Kim, and K. Ramani, “Learning hand articulations by hallucinating heat distribution,” in _Proceedings of the IEEE International Conference on Computer Vision_ , 2017, pp. 3104–3113.
  * [21] W. Zheng, M. Hong, L. Jiang, and C.-W. Fu, “Boosting 3d object detection by simulating multimodality on point clouds,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 13 638–13 647.
  * [22] Z. Yuan, X. Yan, Y. Liao, Y. Guo, G. Li, S. Cui, and Z. Li, “X-trans2cap: Cross-modal knowledge transfer using transformer for 3d dense captioning,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2022, pp. 8563–8573.
  * [23] S. Gupta, J. Hoffman, and J. Malik, “Cross modal distillation for supervision transfer,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016, pp. 2827–2836.
  * [24] S. Ren, Y. Du, J. Lv, G. Han, and S. He, “Learning from the master: Distilling cross-modal advanced knowledge for lip reading,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 13 325–13 333.
  * [25] Y. Zhou and O. Tuzel, “Voxelnet: End-to-end learning for point cloud based 3d object detection,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2018, pp. 4490–4499.
  * [26] Y. Yan, Y. Mao, and B. Li, “Second: Sparsely embedded convolutional detection,” _Sensors_ , vol. 18, no. 10, p. 3337, 2018.
  * [27] W. Shi and R. Rajkumar, “Point-gnn: Graph neural network for 3d object detection in a point cloud,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2020, pp. 1711–1719.
  * [28] C. R. Qi, H. Su, K. Mo, and L. J. Guibas, “Pointnet: Deep learning on point sets for 3d classification and segmentation,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2017, pp. 652–660.
  * [29] C. R. Qi, L. Yi, H. Su, and L. J. Guibas, “Pointnet++: Deep hierarchical feature learning on point sets in a metric space,” _Advances in neural information processing systems_ , vol. 30, 2017.
  * [30] C. He, H. Zeng, J. Huang, X.-S. Hua, and L. Zhang, “Structure aware single-stage 3d object detection from point cloud,” in _Proceedings of the IEEE/CVF conference on computer vision and pattern recognition_ , 2020, pp. 11 873–11 882.
  * [31] Y. Wang, Z. Jiang, X. Gao, J.-N. Hwang, G. Xing, and H. Liu, “Rodnet: Radar object detection using cross-modal supervision,” in _Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision_ , 2021, pp. 504–513.
  * [32] A. Zhang, F. E. Nowruzi, and R. Laganiere, “Raddet: Range-azimuth-doppler based radar object detection for dynamic road users,” in _2021 18th Conference on Robots and Vision (CRV)_. IEEE, 2021, pp. 95–102.
  * [33] K. Qian, S. Zhu, X. Zhang, and L. E. Li, “Robust multimodal vehicle detection in foggy weather using complementary lidar and radar signals,” in _Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition_ , 2021, pp. 444–453.
  * [34] C. R. Qi, O. Litany, K. He, and L. J. Guibas, “Deep hough voting for 3d object detection in point clouds,” in _Proceedings of the IEEE International Conference on Computer Vision_ , 2019.
  * [35] M. Meyer and G. Kuschk, “Deep learning based 3d object detection for automotive radar and camera,” in _2019 16th European Radar Conference (EuRAD)_. IEEE, 2019, pp. 133–136.
  * [36] ——, “Automotive radar dataset for deep learning based 3d object detection,” in _2019 16th European Radar Conference (EuRAD)_ , 2019, pp. 129–132.
  * [37] E. Bel Kamel, A. Peden, and P. Pajusco, “Rcs modeling and measurements for automotive radar applications in the w band,” in _2017 11th European Conference on Antennas and Propagation (EUCAP)_ , 2017, pp. 2445–2449.
  * [38] S. Lee, S. Kang, S.-C. Kim, and J.-E. Lee, “Radar cross section measurement with 77 ghz automotive fmcw radar,” in _2016 IEEE 27th Annual International Symposium on Personal, Indoor, and Mobile Radio Communications (PIMRC)_. IEEE, 2016, pp. 1–6.