# The Oscura Experiment

Alexis Aguilar-Arevalo  Universidad Nacional Autónoma de México, Ciudad de
México, México  Fabricio Alcalde Bessia  Centro Atomico Bariloche, Rio Negro,
Argentina  Nicolas Avalos  Centro Atomico Bariloche, Rio Negro, Argentina
Daniel Baxter  Fermilab National Accelerator Laboratory, IL, USA  Xavier
Bertou  Centro Atomico Bariloche, Rio Negro, Argentina  Carla Bonifazi
International Center of Advanced Studies and Instituto de Ciencias Físicas,
ECyT-UNSAM and CONICET, Argentina  Ana Botti  Fermilab National Accelerator
Laboratory, IL, USA  Mariano Cababie  Universidad de Buenos Aires, Buenos
Aires, Argentina  Gustavo Cancelo  Fermilab National Accelerator Laboratory,
IL, USA  Brenda Aurea Cervantes-Vergara  Universidad Nacional Autónoma de
México, Ciudad de México, México  Nuria Castello-Mor  Instituto de Física de
Cantabria (IFCA), CSIC–Universidad de Cantabria, Santander, Spain  Alvaro
Chavarria  University of Washington, WA, USA  Claudio R. Chavez  Fermilab
National Accelerator Laboratory, IL, USA  IIIE CONICET and DIEC Universidad
Nacional del Sur, Argentina  Fernando Chierchie  IIIE CONICET and DIEC
Universidad Nacional del Sur, Argentina  Juan Manuel De Egea  Facultad de
Ingeniería, Universidad Nacional de Asunción, Paraguay  Juan Carlos D’Olivo
Universidad Nacional Autónoma de México, Ciudad de México, México  Cyrus E.
Dreyer  Stony Brook University, NY, USA  Center for Computational Quantum
Physics, Flatiron Institute, NY, USA  Alex Drlica-Wagner  Fermilab National
Accelerator Laboratory, IL, USA  University of Chicago, IL, USA  Rouven Essig
Stony Brook University, NY, USA  Juan Estrada  Fermilab National Accelerator
Laboratory, IL, USA  Ezequiel Estrada  Centro Atomico Bariloche, Rio Negro,
Argentina  Erez Etzion  Tel Aviv University, Israel  Guillermo Fernandez-
Moroni  Fermilab National Accelerator Laboratory, IL, USA  Marivi Fernández-
Serra  Stony Brook University, NY, USA  Steve Holland  Lawrence Berkeley
National Laboratory, CA, USA  Agustin Lantero Barreda  Instituto de Física de
Cantabria (IFCA), CSIC–Universidad de Cantabria, Santander, Spain  Andrew
Lathrop  Fermilab National Accelerator Laboratory, IL, USA  José Lipovetzky
Centro Atomico Bariloche, Rio Negro, Argentina  Ben Loer  Pacific Northwest
National Laboratory, WA, USA  Edgar Marrufo Villalpando  University of
Chicago, IL, USA  Fermilab National Accelerator Laboratory, IL, USA  Jorge
Molina  Facultad de Ingeniería, Universidad Nacional de Asunción, Paraguay
Sravan Munagavalasa  University of Chicago, IL, USA  Danielle Norcini
University of Chicago, IL, USA  Santiago Perez  Universidad de Buenos Aires,
Buenos Aires, Argentina  Paolo Privitera  University of Chicago, IL, USA
Dario Rodrigues  Universidad de Buenos Aires, Buenos Aires, Argentina  Richard
Saldanha  Pacific Northwest National Laboratory, WA, USA  Diego Santa Cruz
Facultad de Ingeniería, Universidad Nacional de Asunción, Paraguay  Aman
Singal  Stony Brook University, NY, USA  Nathan Saffold  Fermilab National
Accelerator Laboratory, IL, USA  Leandro Stefanazzi  Fermilab National
Accelerator Laboratory, IL, USA  Miguel Sofo-Haro  Centro Atomico Bariloche,
Rio Negro, Argentina  Javier Tiffenberg  Fermilab National Accelerator
Laboratory, IL, USA  Christian Torres  Facultad de Ingeniería, Universidad
Nacional de Asunción, Paraguay  Sho Uemura  Fermilab National Accelerator
Laboratory, IL, USA  Rocio Vilar  Instituto de Física de Cantabria (IFCA),
CSIC–Universidad de Cantabria, Santander, Spain

## Abstract

The Oscura experiment will lead the search for low-mass dark matter particles
using a very large array of novel silicon Charge Coupled Devices (CCDs) with a
threshold of two electrons and with a total exposure of 30 kg-yr. The R&D
effort, which began in FY20, is currently entering the design phase with the
goal of being ready to start construction in late 2024. Oscura will have
unprecedented sensitivity to sub-GeV dark matter particles that interact with
electrons, probing dark matter-electron scattering for masses down to
$\sim$500 keV and dark matter being absorbed by electrons for masses down to
$\sim$1 eV. The Oscura R&D effort has made some significant progress on the
main technical challenges of the experiment, of which the most significant are
engaging new foundries for the fabrication of the CCD sensors, developing a
cold readout solution, and understanding the experimental backgrounds.

###### Table of Contents

  1. 1 Scientific Background and Goals of the Oscura Experiment
  2. 2 The Oscura 10 kg DM Detector
    1. 2.1 Sensors and Packaging
    2. 2.2 Liquid Nitrogen Pressure Vessel
    3. 2.3 Readout Electronics and DAQ
    4. 2.4 Radiation Shield
  3. 3 Challenges and Early Technical Studies
    1. 3.1 Sensor Development
    2. 3.2 Front End Electronics
    3. 3.3 Radiation Background
      1. 3.3.1 Silicon Activation Studies and Tritium Removal
      2. 3.3.2 Low Radioactivity Flexible Cable Fabrication
      3. 3.3.3 Full Detector Simulations
      4. 3.3.4 The CCD Backside
  4. 4 Oscura Construction
  5. References

## 1\. Scientific Background and Goals of the Oscura Experiment

Identifying the nature of dark matter (DM) is one of the most important
missions of particle physics and astrophysics today, and direct-detection
experiments play an essential role in this endeavor. The search for DM
particles with masses up to a few orders of magnitude below the proton mass
(“sub-GeV DM”) represents an important new experimental frontier that has been
receiving increased attention, e.g. [1, 2, 3, 4, 5].

Traditional direct-detection searches, which look for DM particles scattering
elastically off of nuclei, typically have very little sensitivity to sub-GeV
mass DM. Indeed, the best current bounds below 1 GeV are limited to very large
cross sections and are absent below $\sim$87 MeV [6, 7, 8, 9, 10]. While
improved sensitivity at low DM masses to nuclear recoils from elastic scatters
is possible with the next generation of experiments (see e.g. [5, 11]),
improved sensitivity to DM masses well below the GeV scale is possible by
searching for signals induced by inelastic processes [1], for which a DM
particle is able to deposit much more energy compared to the elastic
scattering off of nuclei. In particular, one of the most promising avenues is
to search for one or a few ionized electrons that are released due to DM
particles interacting with electrons in the detector [1, 5]. Indeed,
sensitivity to, and resolution of, single or few-electron events has been
demonstrated with small solid-state targets (silicon and germanium), using
data from SENSEI [1, 12, 13, 14, 15, 16], SuperCDMS [17, 18, 19], and
EDELWEISS [20], as well with noble-liquid (xenon and argon) targets using data
from XENON10, XENON100, LUX, XENON1T, PandaX, and DarkSide-50 [21, 22, 23, 24,
25, 26, 27, 28, 29].

Figure 1: Demonstration of single-electron charge resolution in a skipper-CCD
(4000 charge readout samples per pixel; bin width of 0.03e-). The measured
pixel charge is shown for pixels with low-light level illumination (left) and
stronger illumination (right). Integer electron peaks can be distinctly
resolved in both regimes. The peak at 0e-has rms noise of 0.068e-while the
peak at 777e-has rms noise of 0.086e-. The two measurements demonstrate the
single-electron sensitivity over a large dynamic range. (Figure and caption
from [13].)

Figure 2: Projected sensitivity to dark-matter-electron scattering of Oscura,
assuming a 30 kg-year exposure (blue). We assume zero background events for
events with two or more electrons, and a fixed single-electron “dark-count”
event rate of $10^{-6}e^{-}/{\rm pix}/{\rm day}$. Projected sensitivities for
SENSEI (DAMIC-M) are shown in cyan (red) [1, 12, 13, 30, 31, 5]. Existing
constraints from a skipper-CCD from SENSEI are shaded in cyan [14, 15, 16].
Shaded gray regions are also constrained by several other experiments (some
shown explicitly) including XENON10/100/1t, DarkSide-50, PandaX-II, CDMS-HVeV,
and DAMIC at SNOLAB [21, 22, 23, 24, 27, 25, 18, 32, 33, 19, 28, 29]. Orange
regions labelled “Key Milestone” are the same as in the BRN report [5]. The
left (right) plot assumes the DM-electron interaction is mediated by a heavy
(light) mediator.

Among the most promising detector technologies for the construction of a large
multi-kg experiment for probing electron recoils from sub-GeV DM are the new
generation of silicon Charged Coupled Devices with an ultralow readout noise,
so-called “skipper-CCDs.” Skipper-CCDs were designed by the Lawrence Berkeley
National Laboratory (LBL) Micro Systems Lab. In 2017, the SENSEI (“Sub-
Electron Noise Skipper-CCD Experimental Instrument”) Collaboration
demonstrated the ability to measure precisely the number of free electrons in
each of the million pixels across the CCD [13], see Fig. 1. SENSEI packaged a
small prototype skipper-CCD sensor and took data at Fermilab on the surface
and underground, setting world-leading constraints on DM-electron interactions
for DM masses in the range of 500 keV to 5 MeV [14, 15, 16], see Fig. 2. Using
new, science-grade skipper-CCDs, pathfinder experiments based on this
technology are planned for the coming years. Specifically, SENSEI-100 plans to
install a $\sim$100 g detector at SNOLAB during 2022, while DAMIC-M plans to
install a $\sim$1 kg detector at Modane during 2023.

The Oscura experiment will have sensitivity to single electrons, and has the
goal of achieving zero background events in the 2–10 electron ionization-
signal region (“dark current”, i.e., thermal fluctuations of electrons from
the valence to the conduction band, presents an irreducible source of single-
electron events). Such an experiment will have unprecedented sensitivity to
sub-GeV DM that interacts with electrons. In particular, this experiment can
probe DM masses in the range of 500 keV to 1 GeV when the DM scatters off
electrons through, a “heavy” or “ultralight” mediator [1, 34, 12, 22, 35], see
Fig. 2. This would probe many well-motivated sub-GeV DM models that have been
highlighted in the recommendations of the Basic Research Needs (BRN) workshop
for Dark Matter New Initiatives [5], and which have been combined into the
orange region labelled “Key Milestone” in Fig. 2 as in the BRN report (see
also e.g. [1, 2, 3, 4, 5, 12] and references therein for additional
discussions of these benchmark models). The same detector is also sensitive to
bosonic DM that is absorbed by electrons, for DM masses in the range of
$\sim$1.1 eV (the silicon band gap) to $\sim$1 keV, see Fig. 3 [36, 37, 38].
The projected sensitivities are approximate, and assume 100% signal
efficiency.

Figure 3:  Projected sensitivity to the absorption of dark-photon DM on
electrons for Oscura, assuming a 30 kg-year exposure (blue). We assume zero
background events for events with two or more electrons, and a fixed single-
electron “dark-count” event rate of $10^{-6}e^{-}/{\rm pix}/{\rm day}$.
Projected sensitivities for SENSEI (DAMIC-M) are shown in cyan (red). Shaded
regions show existing constraints from SENSEI, DAMIC at SNOLAB, XENON10,
XENON100, CDMSlite, and CDMS-HVeV [14, 15, 16, 39, 36, 37, 38, 9, 18, 20, 40];
the solar constraint is in green [40].

The Oscura experiment’s ability to measure precisely the number of electrons
will allow as well unprecedented sensitivity to DM interacting with nuclei,
since such interactions can generate one or more ionized electrons in several
ways: (i) from the interactions of the recoiling nucleus with the surrounding
target material, (ii) from a low-energy photon that is radiated from the
nucleus during the DM scattering event, where the photon subsequently converts
in the material into electrons [41], and (iii) from the Migdal effect, where
the recoiling atom is ionized [42]. The latter effect dominates for low-mass
DM-nuclear scattering (see e.g. [42, 43, 44, 45, 33, 46, 47]), and would
generate multiple electrons, which are easy to measure with the proposed
experiment. We show the expected approximate sensitivity of Oscura to DM-
nucleon couplings from the Migdal effect in Fig. 4.

Figure 4:  Projected sensitivity to dark-matter-nucleus scattering of Oscura,
assuming a 30 kg-year exposure (blue). We assume a two-electron threshold and
zero background events. Projected sensitivities for SENSEI (DAMIC-M) are shown
in cyan (red). The projections have been scaled from [47]. Existing
constraints are shaded gray [48, 49, 6, 50, 27, 33]. The left (right) plot
assumes the DM-nucleus interaction is mediated by a heavy (light) mediator.

Our choice of silicon skipper-CCDs as the target material, over other possible
target materials, is motivated by several important considerations:

  1. (i)

The small silicon band gap allows us to probe DM masses an order of magnitude
lower in mass (for both scattering and absorption) than noble-liquid targets,
which have an ionization energy of $\mathcal{O}$(10 eV).

  2. (ii)

Due to the small silicon band gap, the electron recoil-energy needed to
promote additional electrons from the valence to the conduction band is lower
in silicon ($\sim$3.8 eV) than in many other target materials, so that DM-
electron scattering events will often contain two or more electrons.

  3. (iii)

The small mass of the silicon nucleus ensures that solar neutrinos that
scatter coherently off nuclei are not a limiting background for the proposed
experiment with a 30 kg-year exposure [51].

  4. (iv)

The skipper-CCD technology has already been demonstrated and provides an
unprecedented charge resolution, extremely low leakage currents, exquisite
spatial resolution and three-dimensional reconstruction, and background
identification and rejection capabilities. Indeed, the strongest constraints
on low-mass DM scattering off electrons and absorption by electrons (down to
$\sim$500 keV and $\sim$1 eV, respectively) are currently obtained with
skipper-CCDs [16].

  5. (v)

Rapid progress is being been made in understanding the origin and the
mitigation strategies of single- and few-electron backgrounds [52, 53, 54,
55].

## 2\. The Oscura 10 kg DM Detector

The current strategy for an experiment with the exposure and threshold
requirements discussed in Table 1 is a detector with an active mass of 10 kg
using skipper-CCD sensors. Given the standard dimensions of skipper-CCD pixel
(15 $\mu$m x 15 $\mu$m) and considering the standard thickness of silicon
wafers for processing in commercial foundries (700 $\mu$m), 28 gigapixels are
needed to get to an active mass of 10 kg. We have not yet decided on the
optimal backside treatment for the Oscura sensors. For planning purposes we
will assume that backside treatment (discussed in Sec. 3.3.4) is done at the
same foundry where the detectors are fabricated.

The background requirement of less than 1 event between 2e- and 10e- over a 30
kg-year full exposure translates into a background rate requirement of 0.01
dru.1111 dru (differential rate unit) corresponds to 1 event/kg/day/keV. This
corresponds to a significant improvement over previous CCD experiments, with
DAMIC [56, 57] achieving five dru in 2017, SENSEI-100 [15] designed for five
dru, and DAMIC-M [58] planning for less than one dru. This requirement
mandates strict control of all materials selected for the experiment and an
extensive material assay program, a significant part of the Oscura project
plan. The background requirement also imposes a cosmogenic activation budget
for all parts of the experiment, most significantly for the sensors where we
have established a requirement of less than five days of sea level exposure
equivalent after tritium removal (see Section 3.3.1).

Table 1: Background requirements. | requirement
---|---
total exposure | 30 kg-year
energy threshold | 2e-
background |  $<$1 event in full exposure between 2e- and 10e-

The background requirements in Table 1 also impose an important constraint on
the dark count rate for the skipper-CCD sensors. Considering 28 gigapixels and
a total exposure time of three years, the dark current rate of $\sim 10^{-6}$
e-/pix/day gives ten events with 2e- considering a Poisson distribution. This
number is reduced to a single event with 2e- if the full detector array is
readout in less than two hrs, which can be achieved with 24,000 readout
channels, each running at 166 pixels/second. The operation of skipper-CCDs at
this level of dark current requires cooling down the system to somewhere
between 120K and 140K (the optimal operating point will be determined from the
prototype sensors). The current strategy for the cooling system is to submerge
the full detector array in a Liquid Nitrogen (LN2) bath operated with a vapor
pressure of 450 psi to reach this temperature. The power for each readout
channel is estimated to be 32 mW as measured in the prototype. This
corresponds to less than 1 kW power for the full system. The current plan is
to provide this cooling capacity with closed-cycle cryocoolers [59].

The technical requirements are summarized in Table 2.

Table 2: Technical requirements. system | description | goal
---|---|---
sensor | readout noise | 0.15 e- RMS
sensor | dark current |  $10^{-6}$ e/pix/day
readout | speed | 166 pix/sec
readout | channel count | 24,000
detector array | total mass | 10 kg
detector array | number of pixels | 28 Gpix
background | rate | 0.01 dru
LN2 vessel | operating pressure | 450 psi
cooling | capacity | 1 kW
DAQ | data handling | 1 petabyte/year

###  Sensors and Packaging

The packaging of the Oscura sensors will be based on a Multi-Chip-Module (MCM)
with 16 sensors epoxied to a 150 mm silicon wafer with traces connecting the
CCD to a low radiation background flex as shown in Fig.5. The package is
designed to keep only low-background silicon next to the active volume of the
CCD, and a small fraction of the flex consistent with requirements obtained
from the simulations (see Section 3.3.3). The flex circuit and the sensors are
micro-bonded to the carrier silicon wafer. The Oscura experiment needs 1500
MCMs.

Figure 5: (left) Design of the Oscura Multi Chip Module (MCM) with 16 sensors
mounted on a 150 mm silicon wafer. (right) Oscura Super Module (SM) with 16
MCMs supported and shielded with electroformed copper.

As shown in Fig.5, a Super Module (SM) will be built with 16 MCMs using a
support structure of custom ultrapure electro-deposited copper [60]. The SM
also includes some copper to shield the radiation of the first couple of
centimeters from the sensors. The Oscura experiment needs 100 SM to reach 10
kg active mass.

###  Liquid Nitrogen Pressure Vessel

The 10 kg of sensors can be arranged in 100 SMs with dimensions of 120 cm x 50
cm x 50 cm as shown in Fig. 6. The sensors will operate in an LN2 bath with a
vapor pressure of 450 psi. The simulations studies (see Section 3.3.3)
indicate that a layer of 20 cm of lead is needed between the stainless steel
vessel walls and the SMs, and an inner copper shield is necessary to shield
the SM from the ^210Pb in the lead. The inner lead and copper shields are also
shown in Figure 6. The total diameter of the pressure vessel is approximately
1 m in this configuration, and some optimization could allow reducing this
diameter somewhat. The pressure vessel requires a vacuum jacket to control the
thermal load (the vacuum jacket is not shown in the drawings). The cylindrical
vacuum jacketed pressure vessel will have access on one of the sides and the
detector array with its internal shield assembled outside the vessel and
rolled in place. The mechanical design of the pressure vessel is an important
part of the engineering effort in the next stage of the R&D process.

The electronic signals will be carried outside the pressure vessel by using
pressure interface boards like those currently under test in the prototype LN2
system [61]. The end cap of the cylindrical pressure vessel has not been
designed yet and requires significant engineering effort. This end cap will
include the interface board, the cryocooler coldheads to remove the heat out
of the LN2 bath, and also ports for filling and venting LN2. The majority of
active electronic components will be located between the pressure vessel and
lead shield to shield radioactivity in the electronics.

Figure 6: Sketch of a pressure vessel holding the full detector load and an
inner lead and copper shields. The dimension lines refer to the “stacks” of
supermodules, which in this sketch cover 120$\times$36.5 cm. An external
shield of $\sim$60 cm of HDPE (or equivalent absorption power of water) and an
additional $\sim$10 cm of lead is not shown.

###  Readout Electronics and DAQ

The readout electronics for Oscura are divided into Front-End (FE) and Back-
End (BE). The FE electronics are formed by the correlated double sampling
(CDS) circuit built either with discrete components or with an ASIC as
discussed in Section 3.2.

The DAQ will be a cluster of servers on a high-speed network to handle the
data and distribute it to facilities at the collaborating institutions for
scientific analysis. This will completely be based on off-the-shelf parts and
there is no need for development components beyond the commercially available.

The experiment will have 28 Gpixels running with an exposure time of two hours
for a total of three years of science data. This will produce a total of
13,000 science exposures. Assuming 16 bits per pixel, this results in a total
of 0.75 petabytes of science data. We will design a DAQ system capable of
handling one petabyte of data/year. Data compression schemes are currently
under consideration (zero suppression).

Table 3: Readout electronic parts with the counts needed to instrument the full experiment. Part | Description | technology | count needed
---|---|---|---
CCDs | sensor | skipper-CCDs | 24,000
CDS | analog signal correlated double sampling | ASIC or discrete components | 24,000
Mux1 | x16 analog multiplexing for MCM | discrete components on PCB | 1,500
Mux2 | x16 analog multiplexing for Supermodule | discrete components on PCB | 100
PIBA | carries signal outside the LN2 vessel | high density PCB | 10
Osc-LTA | clock drivers and digitization | PCB | 10

###  Radiation Shield

The external radiation shield for the Oscura experiment has not been designed
yet. We expect to use a neutron and gamma shield outside the pressure vessel.
The details will be designed in the next two years as part of our development
of the Preliminary Design Report for Oscura.

## 3\. Challenges and Early Technical Studies

The technical requirements of the Oscura experiment imposed by the scientific
goals discussed above are summarized in Table 2. The Oscura experiment will be
composed of an array of skipper-CCD detectors with 28 Gpixels operating at
120K to reduce the dark counts. The Oscura detector concept is described in
Section 2. Three major technical challenges were identified by the project
team at the start of the concept design study. The R&D effort during this
stage has been focused on these areas: Sensor development, Front end
electronics, and Radiation Background. In this section, we describe the recent
process on these high priority R&D areas.

###  Sensor Development

Skipper-CCDs used for existing DM experiments have been fabricated in 150 mm
diameter wafers using micro-fabrication tools that are being phased out of
modern foundries. A new fabrication process is being developed with Microchip
Inc. The new sensors were completed during 2021 and their characterization is
ongoing (See Fig.7). We have also identified the opportunity to develop more
than one fabrication facility (MIT-LL foundry) to improve performance and
reduce pressure on the schedule for the experiment construction.

Figure 7: Skipper-CCD sensor design developed for the Oscura experiment to be
fabricated at Microchip (design from S.Holland - LBNL). The sensor has four
skipper-CCD amplifiers, one on each corner. Because of the step-and-repeat
photolithography process used at Microchip, and to avoid the additional
challenge of stitching, we are fabricating small format detectors with
1278x1058 pixels and 4 amplifiers. As for previous skipper-CCDs we are using
15 $\mu$m square pixels. The initial run consisted of 24 wafers, and 47
sensors per wafer. The main objective of this fabrication run is to
demonstrate the sub-electron noise performance on the new skipper-CCDs from
the Microchip foundry. The large quantity of sensors produced during this
initial run will be also used to develop prototype packages for Oscura.

###  Front End Electronics

The 10 kg Oscura experiment will require approximately 24,000 readout channels
and a readout time below two hours for the full detector. The requirement for
two hour readout is derived from the impact of dark current during the readout
time. We are working on the development of a low-cost, scalable readout system
for the 10 kg experiment. The main challenge focuses on the front end
electronics. In order to avoid having a large number of signal lines going
from the sensors (inside the low background cryostat) to the outside world, it
is important to develop cold front end electronics and multiplexing. There are
two main research thrusts towards developing the readout solution for Oscura.
The discrete components solution meets the requirement as discussed in
Ref[62]. The ASIC solution could simplify detector assembly (packaging),
reduce radioactive background, and reduce cost for the experiment. The current
ASIC solutions being considered are the MIDNA[63] and the repurposing of
ASPIC[64].

###  Radiation Background

The Oscura project aims at a radioactive background rate of 10 mdru, i.e. 0.01
counts/kg/keV/day. Achieving this radiation background with an affordable
solution constitutes a major risk for the project. We have three R&D efforts
focusing on reducing the risk of the background from components closest to the
core of the detector, including the cosmic activation of the silicon
detectors. The efforts in understanding the radiation background in Oscura
affect directly all other aspects of the experiment design.

####  3.3.1 Silicon Activation Studies and Tritium Removal

One of the dominant radioactive backgrounds anticipated for Oscura is tritium
(3H). Tritium is a radioactive isotope with a half-life of 12.3 years that
undergoes beta decay with a Q-value of 18.6 keV. The continuous spectrum of
electrons resulting from 3H decays leads to a background that spans the energy
region of interest for low-mass DM interactions. Tritium and other radioactive
isotopes such as 7Be and 22Na are produced in silicon detectors as a result of
cosmic-ray exposure, primarily due to interactions of high-energy cosmic-ray
neutrons with silicon nuclei in the detector substrate. It is expected that
effectively all non-silicon atoms present in the raw silicon material are
driven out during single-crystal growth. Consequently, the background level
from cosmogenic isotopes in the final detector is effectively determined by
the above-ground exposure time during and following detector production, the
cosmic-ray flux, and the isotope production cross sections.

The neutron-induced production cross sections for 3H, 7Be, and to a lesser
extent 22Na, were not experimentally known except for a few measurements at
specific energies leading to large uncertainties in the expected cosmogenic
background. To address this deficiency, several members of the Oscura
collaboration performed a dedicated measurement of the integrated isotope-
production rates from a high-energy neutron beam. This measurement allowed for
a direct extrapolation from the measured beam production rates to the expected
sea-level cosmogenic production rates corresponding to ($124\pm 25$) atoms/(kg
day) for 3H, ($9.4\pm 2.0$) atoms/(kg day) for 7Be, and ($49.6\pm 7.4$)
atoms/(kg day) for 22Na [65]. These numbers are in excellent agreement with
those indirectly obtained from spectral fits to data in DAMIC at SNOLAB [54].

The 3H production rate corresponds to activity at low energies of roughly
0.002 dru per day of sea-level exposure. This places strong restrictions on
the fabrication and transportation of silicon detectors for Oscura. In order
to mitigate the tritium background, we are currently exploring the possibility
of using elevated temperatures to remove implanted tritium from fabricated
silicon devices. It is well-known that at high temperatures hydrogen and
tritium are fairly mobile within the silicon lattice and baking can remove
them from within silicon substrates [66]. However, measurements of the tritium
diffusion coefficient in silicon vary by several orders of magnitude, thought
to be related to variations in the levels of impurities and defects, and the
method by which the tritium was implanted (e.g. as a gas or through particle
interactions) [66, 67, 68, 69].

We have been experimentally measuring the effectiveness of baking to remove
tritium produced by high-energy neutron interactions (the dominant cosmogenic
production mechanism) from high-purity silicon wafers. Silicon wafers
irradiated on a high energy neutron beam at LANSCE [70, 71] are baked at
controlled temperatures to extract the tritium. The tritium is captured,
converted to tritiated water, and then measured using liquid scintillation
counting [72]. Preliminary measurements have shown that cosmogenically
produced tritium can be efficiently removed by baking at $900^{\circ}$C.
Detailed results of this study are being prepared for publication.

In addition, we are investigating if tritium can be removed either after
fabrication of the CCDs or after packaging. Fabricated CCDs cannot be exposed
to very high temperatures and additionally have surface layers (e.g. silicon
nitride) that could impede the diffusion of tritium. Similarly, the CCD
packaging may not survive very high temperatures. We are currently
investigating the maximum temperature that CCDs and CCD packaging can
withstand, and the efficiency of tritium removal at lower temperatures
(300-500∘C). Early results from baking silicon wafers show promising removal
efficiencies at 400∘C and we are putting together a dedicated baking setup for
long-duration baking tests at lower temperatures.

####  3.3.2 Low Radioactivity Flexible Cable Fabrication

The baseline design of the Oscura experiment utilizes flat flexible cables,
made from copper-polyimide laminates, to readout the CCD sensors. Polyimides
are widely used as an insulating substrate in the electronics industry due to
their unique properties of high resistivity, high dielectric strength, and
flexibility. Polyimides are also stable across a wide range of temperatures,
have good thermal conductivity, a thermal expansion coefficient that is close
to copper, and a low outgassing rate.

Commercial laminates of copper and polyimide are typically not very radiopure,
with measured contamination levels at roughly 300 parts-per-trillion (ppt)
^238U and 50 ppt ^232Th. However members of the Oscura collaboration have
previously worked to identify radiopure laminates with contamination levels at
9 and 20 ppt ^238U and ^232Th, respectively [73].

While radiopure copper-polyimide laminates have been identified, the process
of fabricating a cable out of this starting material adds significant
contamination. Measurements of completed flexible cables showed some
impurities at the level of 2000 ppt ^238U and 800 ppt ^232Th. At these levels,
the radioactivity contribution from the flexible cables would dominate the
background budget of the entire experiment. We are currently working with a
commercial manufacturer of flexible cables to identify and mitigate the
sources of contamination during fabrication. Through a careful analysis of
each step in the photolithographic production, we have identified specific
processes and materials that are the dominant contributors to the
contamination and preliminary studies on simplified cables indicate that we
can achieve significant reductions in contamination.

####  3.3.3 Full Detector Simulations

In order to evaluate the background contributions from radioactive
contamination in the materials used to construct the experiment, a model is
being constructed in GEANT4 [74]. Although the energy range of interest ($<$1
keV) has not been well-validated in GEANT4, we have adopted the best practices
developed by the DAMIC, SENSEI, and SuperCDMS collaborations. This includes
use of the EmOption4 high precision low energy electromagnetic physics list,
“Shielding” physics list for hadronic (neutron) interactions, the muelec low
energy silicon list, and tuning low energy atomic deexcitation and secondary
production [75]. Continuing to improve and tune the simulation physics list
will be a significant ongoing R&D effort. The eventual goal of the simulation
effort is to construct a global background budget including all potential
radiological background sources. During the initial design development, we
have focused on the following sources that we expect to be of greatest impact:

##### Cavern background

We consider specifically the measured backgrounds at SNOLAB:
$4000\text{\,}\mathrm{n}\mathrm{e}\mathrm{u}\mathrm{t}\mathrm{r}\mathrm{o}\mathrm{n}\mathrm{s}\mathrm{/}\mathrm{m}^{2}\mathrm{/}\mathrm{d}\mathrm{a}\mathrm{y}$
and
$4.25\text{\,}\mathrm{g}\mathrm{a}\mathrm{m}\mathrm{m}\mathrm{a}\mathrm{s}\mathrm{/}\mathrm{c}\mathrm{m}^{2}\mathrm{/}\mathrm{s}$
[76, 77]. Based on preliminary results, we estimate that a shield of
approximately $60\text{\,}\mathrm{c}\mathrm{m}$ of high-density polyethylene
and $40\text{\,}\mathrm{c}\mathrm{m}$ of lead will reduce the flux to
acceptable levels. Due to the computationally intense nature of these
simulations, this estimate is still in progress.

##### Pressure Vessel

The CCDs will be cooled with liquid nitrogen at
$\sim$$425\text{\,}\mathrm{p}\mathrm{s}\mathrm{i}$ in order to raise the
boiling point to an acceptable working temperature. This requires a large
vacuum-insulated pressure vessel. The baseline design includes a stainless
steel pressure vessel and flanges. Properly sourced stainless steel contains
radioactive isotopes ^40K, ^238U, ^232Th (KUT) on the order of
$1\text{\,}\mathrm{m}\mathrm{B}\mathrm{q}\mathrm{/}\mathrm{k}\mathrm{g}$
levels with significant variation, and ^60Co typically several times higher
than KUT [78]. Some fraction of the lead shielding will be placed inside the
pressure vessel. Currently, we estimate $\sim 16-20$ cm of lead will be
required. Because the cost of the pressure vessel increases rapidly with
radius, this value will be carefully optimized.

We are also considering a titanium vessel in the cost optimization. Titanium
has similar levels of KUT contamination per unit mass as steel, but contains
almost no ^60Co and has roughly four times the strength-to-weight ratio of
steel. So although titanium is much more expensive, it may be possible to use
less internal shielding (current estimates indicate 12 cm may be sufficient),
thus requiring a smaller vessel. We have also considered constructing the
pressure vessel from oxygen-free high thermal conductivity (OFHC) copper, but
at this time it does not appear to be a cost-effective solution.

##### Lead shielding

In addition to KUT, standard lead contains ^210Pb at several hundred
$\mathrm{B}\mathrm{q}\mathrm{/}\mathrm{k}\mathrm{g}$ [79, 80, 81]. The
innermost layers of the lead shield will consist of “ancient” lead with
reduced ^210Pb content. The exact composition will depend on what grades are
available from vendors or for re-use from previous experiments (e.g., DAMIC).
As a rough baseline, we consider a 20 cm lead shield with the outer 10 cm
standard lead, then 5 cm of lead with 1 Bq/kg ^210Pb, then 5 cm of 0.1 Bq/kg
with an 8 cm inner copper layer. Commercial OFHC copper also contains ^210Pb
in the bulk at several tens of mBq/kg [82], so the innermost centimeters of
the copper liner will either have to be very carefully sourced and assayed or
be made of electroformed copper. We are in the process of studying what
shielding is required for KUT contamination in the lead but expect this to be
subdominant.

##### CCD modules

In the current design, 16 MCMs (each containing 16 CCDs) are grouped into
supermodules, with the full 10 kg experiment containing $\sim$100
supermodules. By placing KUT contaminants at the edges of the MCMs defined in
the simulations, we estimate a requirement of
$\lesssim$$100\text{\,}\mathrm{nBq}$ of ^238U and ^232Th and
$\lesssim$$10\text{\,}\mathrm{\SIUnitSymbolMicro Bq}$ ^40K per supermodule. To
meet this requirement, the majority of the mass of the supermodules will be
silicon and electroformed copper. The mass and purity of any epoxies, solder,
wirebond material, etc., will be carefully considered. The most problematic
components will be any non-copper mechanical fasteners (screws, etc) and the
readout cables and cold electronics. With proper screening, silicon bronze and
Polytetrafluoroethylene (PTFE) 222PTFE is a synthetic fluoropolymer of
tetrafluoroethylene: hydrophobic, non-wetting, high density, and resistant to
high temperatures material. appear to be suitable mechanical components.
Production of the electroformed copper could also benefit from a planned
construction of an electroformed copper production facility underground at
SNOLAB.

##### Cold electronics

The target radiopurity limits of $\sim$hundreds of nBq per supermodule and the
target contamination levels in kapton cables in Section 3.3.2 suggest that
only a few centimeters of flex cable can be directly ”visible” to the CCDs
without some intermediate shielding. The current supermodule design achieves
this by clamping the cables between $\sim$$3\text{\,}\mathrm{c}\mathrm{m}$
copper shields. Shortly outside the supermodules, the cables will also be
routed through the internal lead shield. The bulk of the cold electronics will
be placed outside the lead (next to the pressure vessel). Further refinement
of the simulation will be used to specify the available length of cable, and
validate whether any electronic components can be placed inside the lead,
e.g., to reduce the un-amplified signal length or provide a connection point
to simplify assembly and disassembly.

##### Near-infrared (NIR) photon emission, transport, and absorption

High-energy charged particles interacting with the silicon of the CCD and with
the surrounding structures can produce NIR photons from Cherenkov radiation.
In addition, charged particles produce electron-hole pairs in the highly-doped
gettering333Gettering is the process of reducing impurities by localizing them
in predetermined, passive regions of the silicon wafer. layer on the backside
of the CCD that subsequently recombine radiatively and emit near-infrared NIR
photons [53]. These NIR photons have a long attenuation length in silicon and,
once absorbed, produce a single-electron event. Such events are the likely
origin of many of the single-electron events observed in SENSEI data taken at
a shallow underground site [53, 16]. Work is ongoing to simulate in detail the
production and propagation of these NIR photons. We expect the single-electron
event rate to be reduced drastically in a low-background environment, which we
plan to confirm with a detailed simulation. We will also investigate how
different thicknesses of the gettering layer on the CCD backside impact the
single-electron event rate: while a thick layer enhances the radiative
recombination rate, this layer also acts as an efficient absorber of NIR
photons generated by Cherenkov. We discuss the CCD backside further in Sec.
3.3.4.

####  3.3.4 The CCD Backside

The processing of the CCD backside has a non-trivial impact on the observed
low-energy background spectrum and rate. An important development from the
analysis of the latest data from DAMIC at SNOLAB was the discovery of a
$\sim$5 $\mu$m-thick partial charge collection (PCC) region in the backside of
the CCDs [57]. LBNL CCDs feature a 1 $\mu$m-thick in-situ doped polysilicon
(ISDP) backside gettering layer to remove contaminants from the high-
resistivity bulk silicon during fabrication, which also functions as the
backside electrical contact of the CCD. Phosphorous (P) from the highly doped
ISDP diffuses with time into the lightly doped high-resistivity silicon. At
intermediate P concentrations, some of the charge generated by ionization
events recombines before it diffuses into the fully-depleted region and is
drifted across the substrate to the CCD gates. The loss of charge distorts the
spectrum from ionization events on the backside, e.g., surface 210Pb decays,
increasing the amplitude of the background spectrum at low energies, where the
DM signal is expected. Studies of the charge collection efficiency in the
backside of LBNL CCDs performed by Oscura collaborators [83] confirm the
observations at SNOLAB.

As noted previously, the recombination of electron-hole pairs generated in the
gettering layer can generate NIR photons which contributes to the single-
electron events observed in SENSEI data [53, 16]. In addition, a secondary ion
mass spectrometry (SIMS) measurement of the DAMIC wafers shows a concentration
of hydrogen in the gettering layer of $10^{20}$ cm-3, which implies the
presence of radioactive 3H on the CCD backside.

LBNL CCDs have been back-thinned in the past to improve quantum efficiency to
blue light [84]. The CCD wafers were sent for backside polishing before the
final stages of fabrication to remove all polysilicon (including gettering)
and oxide layers. A backside contact was then formed by depositing a thin
($\sim$20 nm) ISDP layer. This process would decrease the thickness of the PCC
region by a factor of $10^{2}$, with a corresponding improvement in the
ionization signal lost to recombination by fast electrons (and associated
radiative backgrounds) and the 3H radioactivity on the backside. However, this
layer also acts as an absorber of some NIR photons from Cherenkov radiation,
so its removal may require additional backside treatment to stop external
photons.

We are exploring several backside treatment options for the Oscura CCDs and
have fabricated and commissioned a calibration system to characterize the
response of CCDs to surface backgrounds at the University of Washington.

## 4\. Oscura Construction

The Oscura construction project has the goal of building the 10 kg detector
according to the details in Section 2. The project is currently in its
conceptual design phase which will be completed in 2024. The full 10 kg Oscura
experiment construction will be completed in four years after the design
phase. Oscura will have a significant detector mass available for scientific
studies before the completion of the four years construction project (three kg
mass available per year). We are currently exploring the possibility of
deploying a fraction of the total mass at an underground facility for early
science and background control of the experiment before the construction is
completed.

## References

## References

  * [1] Essig R, Mardon J, Volansky T (2012) Direct Detection of Sub-GeV Dark Matter. _Phys Rev_ D85:076007. https://doi.org/10.1103/PhysRevD.85.076007. 1108.5383
  * [2] Essig R, et al. (2013) Working Group Report: New Light Weakly Coupled Particles. _Proceedings, 2013 Community Summer Study on the Future of U.S. Particle Physics: Snowmass on the Mississippi (CSS2013): Minneapolis, MN, USA, July 29-August 6, 2013_ , . 1311.0029 URL http://www.slac.stanford.edu/econf/C1307292/docs/IntensityFrontier/NewLight-17.pdf.
  * [3] Alexander J, et al. (2016) Dark Sectors 2016 Workshop: Community Report, . %**** ̵main.bbl ̵Line ̵25 ̵****1608.08632 URL http://lss.fnal.gov/archive/2016/conf/fermilab-conf-16-421.pdf.
  * [4] Battaglieri M, et al. (2017) US Cosmic Visions: New Ideas in Dark Matter 2017: Community Report. _arXiv:170704591 [astro-ph, physics:hep-ex, physics:hep-ph]_ ArXiv: 1707.04591 URL http://arxiv.org/abs/1707.04591.
  * [5] Department of Energy (2019) Basic Research Needs for Dark Matter Small Projects New Initiatives, . URL https://science.energy.gov/~/media/hep/pdf/Reports/Dark˙Matter˙New˙Initiatives˙rpt.pdf.
  * [6] Abdelhameed AH, et al. (CRESST) (2019) First results from the CRESST-III low-mass dark matter program. _Phys Rev D_ 100(10):102002. https://doi.org/10.1103/PhysRevD.100.102002. 1904.00498
  * [7] Angloher G, et al. (CRESST) (2017) Results on MeV-scale dark matter from a gram-scale cryogenic calorimeter operated above ground. _Eur Phys J_ C77(9):637. https://doi.org/10.1140/epjc/s10052-017-5223-9. %**** ̵main.bbl ̵Line ̵50 ̵****1707.06749
  * [8] Petricca F, et al. (CRESST) (2017) First results on low-mass dark matter from the CRESST-III experiment. _15th International Conference on Topics in Astroparticle and Underground Physics (TAUP 2017) Sudbury, Ontario, Canada, July 24-28, 2017_ , . 1711.07692 URL http://inspirehep.net/record/1637341/files/arXiv:1711.07692.pdf.
  * [9] Agnese R, et al. (SuperCDMS) (2016) New Results from the Search for Low-Mass Weakly Interacting Massive Particles with the CDMS Low Ionization Threshold Experiment. _Phys Rev Lett_ 116(7):071301. https://doi.org/10.1103/PhysRevLett.116.071301. 1509.02448
  * [10] Alkhatib I, et al. (2021) Light dark matter search with a high-resolution athermal phonon detector operated above ground. _Physical Review Letters_ 127(6). https://doi.org/10.1103/physrevlett.127.061801. URL http://dx.doi.org/10.1103/PhysRevLett.127.061801
  * [11] D N McKinsey et al [TESSERACT Collaboration] (TESSERACT) The TESSERACT Dark Matter Project, SNOWMASS LOI, Availabe [Online]. URL www.snowmass21.org/docs/files/summaries/CF/SNOWMASS21-CF1˙CF2-IF1˙IF8-120.pdf.
  * [12] Essig R, et al. (2016) Direct Detection of sub-GeV Dark Matter with Semiconductor Targets. _JHEP_ 05:046. https://doi.org/10.1007/JHEP05(2016)046. 1509.01598
  * [13] Tiffenberg J, et al. (2017) Single-Electron and Single-Photon Sensitivity with a Silicon Skipper CCD. _PRL_ 119(13):131802. https://doi.org/10.1103/PhysRevLett.119.131802. 1706.00028
  * [14] Crisler M, et al. (2018) SENSEI: First Direct-Detection Constraints on Sub-GeV Dark Matter from a Surface Run. _PRL_ 121(6):061803. https://doi.org/10.1103/PhysRevLett.121.061803. 1804.00088
  * [15] Abramoff O, et al. (SENSEI) (2019) Sensei: Direct-detection constraints on sub-gev dark matter from a shallow underground run using a prototype skipper ccd. _Phys Rev Lett_ 122:161801. https://doi.org/10.1103/PhysRevLett.122.161801. URL https://link.aps.org/doi/10.1103/PhysRevLett.122.161801
  * [16] L Barak et al [SENSEI Collaboration] (2020) SENSEI: Direct-Detection Results on sub-GeV Dark Matter from a New Skipper-CCD. _Phys Rev Lett_ 125(17):171802. https://doi.org/10.1103/PhysRevLett.125.171802. 2004.11378
  * [17] Romani RK, et al. (2018) Thermal detection of single e-h pairs in a biased silicon crystal detector. _Appl Phys Lett_ 112:043501. https://doi.org/10.1063/1.5010699. 1710.09335
  * [18] Agnese R, et al. (SuperCDMS) (2018) First Dark Matter Constraints from a SuperCDMS Single-Charge Sensitive Detector. _Phys Rev Lett_ 121(5):051301. https://doi.org/10.1103/PhysRevLett.121.051301. [Erratum: Phys.Rev.Lett. 122, 069901 (2019)] 1804.10697
  * [19] Amaral DW, et al. (SuperCDMS) (2020) Constraints on low-mass, relic dark matter candidates from a surface-operated SuperCDMS single-charge sensitive detector. _Phys Rev D_ 102(9):091101. https://doi.org/10.1103/PhysRevD.102.091101. 2005.14067
  * [20] Arnaud Q, et al. (EDELWEISS) (2020) First germanium-based constraints on sub-MeV Dark Matter with the EDELWEISS experiment. _Phys Rev Lett_ 125(14):141301. https://doi.org/10.1103/PhysRevLett.125.141301. 2003.01046
  * [21] Essig R, Manalaysay A, Mardon J, Sorensen P, Volansky T (2012) First Direct Detection Limits on sub-GeV Dark Matter from XENON10. _Phys Rev Lett_ 109:021301. https://doi.org/10.1103/PhysRevLett.109.021301. 1206.2644
  * [22] Essig R, Volansky T, Yu TT (2017) New Constraints and Prospects for sub-GeV Dark Matter Scattering off Electrons in Xenon. _Phys Rev_ D96(4):043017. https://doi.org/10.1103/PhysRevD.96.043017. %**** ̵main.bbl ̵Line ̵150 ̵****1703.00910
  * [23] Angle J, et al. (XENON10) (2011) A search for light dark matter in XENON10 data. _Phys Rev Lett_ 107:051301. https://doi.org/10.1103/PhysRevLett.110.249901,10.1103/PhysRevLett.107.051301. [Erratum: Phys. Rev. Lett.110,249901(2013)] 1104.3088
  * [24] Aprile E, et al. (2016) Low-mass dark matter search using ionization signals in xenon100. _Physical Review D_ 94(9). https://doi.org/10.1103/physrevd.94.092001. URL http://dx.doi.org/10.1103/PhysRevD.94.092001
  * [25] Agnes P, et al. (2018) Constraints on sub-gev dark-matter–electron scattering from the darkside-50 experiment. _Physical Review Letters_ 121(11). https://doi.org/10.1103/physrevlett.121.111303. URL http://dx.doi.org/10.1103/PhysRevLett.121.111303
  * [26] D S Akerib et al [LUX Collaboration] (LUX) (2020) Investigation of background electron emission in the LUX detector. _Phys Rev D_ 102(9):092004. https://doi.org/10.1103/PhysRevD.102.092004. 2004.07791
  * [27] Aprile E, et al. (XENON) (2019) Light Dark Matter Search with Ionization Signals in XENON1T. _Phys Rev Lett_ 123(25):251801. https://doi.org/10.1103/PhysRevLett.123.251801. 1907.11485
  * [28] Cheng C, et al. (PandaX-II) (2021) Search for Light Dark Matter-Electron Scatterings in the PandaX-II Experiment. _Phys Rev Lett_ 126(21):211803. https://doi.org/10.1103/PhysRevLett.126.211803. 2101.07479
  * [29] Aprile E, et al. (XENON) (2021) Emission of Single and Few Electrons in XENON1T and Limits on Light Dark Matter 2112.12116.
  * [30] Settimo M (DAMIC) (2018) The Damic Experiment at Snolab. _Proceedings, 53Rd Rencontres De Moriond on Cosmology: La Thuile, Italy, March 17-24, 2018_ , . 1805.10001.
  * [31] Castelló-Mor N (DAMIC-M) (2020) DAMIC-M Experiment: Thick, Silicon CCDs to search for Light Dark Matter. _Nucl Instrum Meth A_ 958:162933. https://doi.org/10.1016/j.nima.2019.162933. 2001.01476
  * [32] Aguilar-Arevalo A, et al. (DAMIC) (2019) Constraints on Light Dark Matter Particles Interacting with Electrons from DAMIC at SNOLAB. _Phys Rev Lett_ 123(18):181802. https://doi.org/10.1103/PhysRevLett.123.181802. 1907.12628
  * [33] Essig R, Pradler J, Sholapurkar M, Yu TT (2020) Relation between the Migdal Effect and Dark Matter-Electron Scattering in Isolated Atoms and Semiconductors. _Phys Rev Lett_ 124(2):021801. https://doi.org/10.1103/PhysRevLett.124.021801. 1908.10881
  * [34] Graham PW, Kaplan DE, Rajendran S, Walters MT (2012) Semiconductor Probes of Light Dark Matter. _PhysDark Univ_ 1:32–49. https://doi.org/10.1016/j.dark.2012.09.001. %**** ̵main.bbl ̵Line ̵225 ̵****1203.2531
  * [35] Lee SK, Lisanti M, Mishra-Sharma S, Safdi BR (2015) Modulation Effects in Dark Matter-Electron Scattering Experiments. _Phys Rev_ D92(8):083517. https://doi.org/10.1103/PhysRevD.92.083517. 1508.07361
  * [36] An H, Pospelov M, Pradler J, Ritz A (2015) Direct Detection Constraints on Dark Photon Dark Matter. _Phys Lett_ B747:331–338. https://doi.org/10.1016/j.physletb.2015.06.018. 1412.8378
  * [37] Bloch IM, Essig R, Tobioka K, Volansky T, Yu TT (2017) Searching for Dark Absorption with Direct Detection Experiments. _JHEP_ 06:087. https://doi.org/10.1007/JHEP06(2017)087. 1608.02123
  * [38] Hochberg Y, Lin T, Zurek KM (2017) Absorption of light dark matter in semiconductors. _Phys Rev_ D95(2):023013. https://doi.org/10.1103/PhysRevD.95.023013. 1608.01994
  * [39] Aguilar-Arevalo A, et al. (2017) First Direct-Detection Constraints on eV-Scale Hidden-Photon Dark Matter with DAMIC at SNOLAB. _PRL_ 118(14):141803. https://doi.org/10.1103/PhysRevLett.118.141803. 1611.03066
  * [40] An H, Pospelov M, Pradler J (2013) New stellar constraints on dark photons. _Phys Lett_ B725:190–195. https://doi.org/10.1016/j.physletb.2013.07.008. 1302.3884
  * [41] Kouvaris C, Pradler J (2017) Probing sub-GeV Dark Matter with conventional detectors. _Phys Rev Lett_ 118(3):031803. https://doi.org/10.1103/PhysRevLett.118.031803. 1607.01789
  * [42] Ibe M, Nakano W, Shoji Y, Suzuki K (2018) Migdal Effect in Dark Matter Direct Detection Experiments. _JHEP_ 03:194. https://doi.org/10.1007/JHEP03(2018)194. 1707.07258
  * [43] Bell NF, Dent JB, Newstead JL, Sabharwal S, Weiler TJ (2020) Migdal effect and photon bremsstrahlung in effective field theories of dark matter direct detection and coherent elastic neutrino-nucleus scattering. _Physical Review D_ 101(1). https://doi.org/10.1103/physrevd.101.015012. URL http://dx.doi.org/10.1103/PhysRevD.101.015012
  * [44] Dolan MJ, Kahlhoefer F, McCabe C (2018) Directly detecting sub-GeV dark matter with electrons from nuclear scattering. _Phys Rev Lett_ 121(10):101801. https://doi.org/10.1103/PhysRevLett.121.101801. 1711.09906
  * [45] Baxter D, Kahn Y, Krnjaic G (2020) Electron Ionization via Dark Matter-Electron Scattering and the Migdal Effect. _Phys Rev D_ 101(7):076014. https://doi.org/10.1103/PhysRevD.101.076014. 1908.00012
  * [46] Knapen S, Kozaczuk J, Lin T (2021) Migdal effect in semiconductors. _Physical Review Letters_ 127(8). https://doi.org/10.1103/physrevlett.127.081805. URL http://dx.doi.org/10.1103/PhysRevLett.127.081805
  * [47] Knapen S, Kozaczuk J, Lin T (2022) DarkELF: A python package for dark matter scattering in dielectric targets. _Physical Review D_ 105(1). https://doi.org/10.1103/physrevd.105.015014. URL http://dx.doi.org/10.1103/PhysRevD.105.015014
  * [48] Aprile E, et al. (XENON) (2019) Search for Light Dark Matter Interactions Enhanced by the Migdal Effect or Bremsstrahlung in XENON1T. _Phys Rev Lett_ 123(24):241803. https://doi.org/10.1103/PhysRevLett.123.241803. 1907.12771
  * [49] Akerib DS, et al. (LUX) (2019) Results of a Search for Sub-GeV Dark Matter Using 2013 LUX Data. _Phys Rev Lett_ 122(13):131301. https://doi.org/10.1103/PhysRevLett.122.131301. 1811.11241
  * [50] Liu ZZ, et al. (CDEX) (2019) Constraints on Spin-Independent Nucleus Scattering with sub-GeV Weakly Interacting Massive Particle Dark Matter from the CDEX-1B Experiment at the China Jinping Underground Laboratory. _Phys Rev Lett_ 123(16):161301. https://doi.org/10.1103/PhysRevLett.123.161301. 1905.00354
  * [51] Essig R, Sholapurkar M, Yu TT (2018) Solar Neutrinos as a Signal and Background in Direct-Detection Experiments Searching for Sub-GeV Dark Matter With Electron Recoils. _Phys Rev_ D97(9):095029. https://doi.org/10.1103/PhysRevD.97.095029. 1801.10159
  * [52] Aguilar-Arevalo A, et al. (DAMIC) (2020) Results on low-mass weakly interacting massive particles from a 11 kg-day target exposure of DAMIC at SNOLAB. _Phys Rev Lett_ 125:241803. https://doi.org/10.1103/PhysRevLett.125.241803. 2007.15622
  * [53] Du P, Egana-Ugrinovic D, Essig R, Sholapurkar M (2022) Sources of Low-Energy Events in Low-Threshold Dark-Matter and Neutrino Detectors. _Phys Rev X_ 12(1):011009. https://doi.org/10.1103/PhysRevX.12.011009. 2011.13939
  * [54] Aguilar-Arevalo A, et al. (DAMIC) (2021) Characterization of the background spectrum in DAMIC at SNOLAB 2110.13133.
  * [55] Fuss A, Kaznacheeva M, Reindl F, Wagner F (eds) (2022) _EXCESS workshop: Descriptions of rising low-energy spectra_ , . 2202.05097.
  * [56] Aguilar-Arevalo A, et al. (2016) Search for low-mass WIMPs in a 0.6 kg day exposure of the DAMIC experiment at SNOLAB. _PRD_ 94(8):082006. https://doi.org/10.1103/PhysRevD.94.082006
  * [57] Aguilar-Arevalo A, et al. (2020) Results on Low-Mass Weakly Interacting Massive Particles from an 11 kg d Target Exposure of DAMIC at SNOLAB. _Phys Rev Letters_ 125(24):241803. https://doi.org/10.1103/PhysRevLett.125.241803. 2007.15622
  * [58] Settimo M (2020) Search for low-mass dark matter with the DAMIC experiment. _arXiv e-prints_ :arXiv:2003.094972003.09497.
  * [59] (2021) AL600 Cryocoolers, Cryomech, . URL https://www.cryomech.com/products/al600/.
  * [60] Suriano AM, Howard SM, Christofferson CD, Arnquist IJ, Hoppe EW (2018) Developing radiopure copper alloys for high strength low background applications. _Low Radioactivity Techniques 2017 (LRT 2017)_ , _American Institute of Physics Conference Series_ , Vol. 1921 _American Institute of Physics Conference Series_ , Vol. 1921, p 080001. https://doi.org/10.1063/1.5019009
  * [61] (2021) Pressure Interface Board Assembly, Projects Document 7695, . URL https://projects-docdb.fnal.gov/cgi-bin/private/ShowDocument?docid=7695.
  * [62] Sofo Haro M, et al. (2021) Analog pile-up circuit technique using a single capacitor for the readout of Skipper-CCD detectors. _Journal of Instrumentation_ 16(11):P11012. https://doi.org/10.1088/1748-0221/16/11/P11012. 2108.09389
  * [63] England T, et al. (1900) A multi-channel cryogenic low-noise skipper-ccd readout asic https://doi.org/10.2172/1841383. URL https://www.osti.gov/biblio/1841383
  * [64] Antilogus P, et al. (2017) ASPIC and CABAC: two ASICs to readout and pilot CCD. _Journal of Instrumentation_ 12(03):C03017–C03017. https://doi.org/10.1088/1748-0221/12/03/c03017. URL https://doi.org/10.1088%2F1748-0221%2F12%2F03%2Fc03017
  * [65] Saldanha R, et al. (2020) Cosmogenic activation of silicon. _Physical Review D_ 102(10):102006. https://doi.org/10.1103/PhysRevD.102.102006
  * [66] Sopori B, Zhang Y, Ravindra N (2001) Silicon device processing in H-ambients: H-diffusion mechanisms and influence on electronic properties. _Journal of Electronic Materials_ 30(12):1616–1627.
  * [67] Ichimiya T, Furuichi A (1968) On the solubility and diffusion coefficient of tritium in single crystals of silicon. _The International Journal of Applied Radiation and Isotopes_ 19(7):573–578.
  * [68] Qaim S, Wölfle R (1978) Triton emission in the interactions of fast neutrons with nuclei. _Nuclear Physics A_ 295(1):150–162.
  * [69] Saeki M, Yokoyama A, Aratono Y, Tachikawa E (1983) Origins of diffusivity-retardation of tritium in bombarded silicon crystal. _Radiochimica Acta_ 33(2/3):139–143.
  * [70] Lisowski P, Schoenberg K (2006) The Los Alamos Neutron Science Center. _Nuclear Instruments and Methods A_ 562:910–914.
  * [71] Takala B (2006) The ICE House. _Los Alamos Science_ .
  * [72] Warwick P, Kim D, Croudace I, Oh J (2010) Effective desorption of tritium from diverse solid matrices and its application to routine analysis of decommissioning materials. _Analytica chimica acta_ 676(1-2):93–102.
  * [73] Arnquist IJ, Beck C, di Vacri ML, Harouaka K, Saldanha R (2020) Ultra-low radioactivity kapton and copper-kapton laminates. _Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment_ 959:163573.
  * [74] Agostinelli S, et al. (2003) Geant4—a simulation toolkit. _Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment_ 506(3):250–303. https://doi.org/10.1016/S0168-9002(03)01368-8
  * [75] Valentin A, Raine M, Gaillardin M, Paillet P (2012) Geant4 physics processes for microdosimetry simulation: Very low energy electromagnetic models for protons and heavy ions in silicon. _Nuclear Instruments and Methods in Physics Research Section B: Beam Interactions with Materials and Atoms_ 287:124–129. https://doi.org/10.1016/j.nimb.2012.06.007
  * [76] (2006) SNOLAB User’s Handbook Revision 2, . URL http://snolab2008.snolab.ca/snolab˙users˙handbook˙rev02.pdf.
  * [77] PICO Collaboration, et al. (2019) Data-driven modeling of electron recoil nucleation in PICO ${\mathrm{}}C{}{}_{3}{\mathrm{}}F{}{}_{8}$ bubble chambers. _Physical Review D_ 100(8):082006. https://doi.org/10.1103/PhysRevD.100.082006
  * [78] Akerib DS, et al. (2017) Identification of radiopure titanium for the LZ dark matter experiment and future rare event searches. _Astroparticle Physics_ 96:1–10. https://doi.org/10.1016/j.astropartphys.2017.09.002
  * [79] Vojtyla P (1996) Fast computer simulations of background of low-level Ge $\gamma$-spectrometers induced by 210Pb210Bi in shielding lead. _Nucl Instr Meth B_ 117(1–2):189–198. https://doi.org/10.1016/0168-583X(96)00275-3
  * [80] Alessandrello A, et al. (1991) Measurements on radioactivity of ancient roman lead to be used as shield in searches for rare events. _Nuclear Instruments and Methods in Physics Research Section B: Beam Interactions with Materials and Atoms_ 61(1):106–117. https://doi.org/10.1016/0168-583X(91)95569-Y
  * [81] Heusser G (1991) Studies of $\gamma$-ray background with a low level germanium spectrometer. _Nuclear Instruments and Methods in Physics Research Section B: Beam Interactions with Materials and Atoms_ 58(1):79–84. https://doi.org/10.1016/0168-583X(91)95680-C
  * [82] Abe K, et al. (2018) Identification of 210Pb and 210Po in the bulk of copper samples with a low-background alpha particle counter. _Nuclear Instruments and Methods in Physics Research Section A: Accelerators, Spectrometers, Detectors and Associated Equipment_ 884:157–161. https://doi.org/10.1016/j.nima.2017.12.015
  * [83] Fernandez-Moroni G, et al. (2020) Charge collection efficiency in back-illuminated Charge-Coupled Devices. _arXiv e-prints_ :arXiv:2007.042012007.04201.
  * [84] Holland SE, Groom DE, Palaio NP, Stover RJ, Wei M (2003) Fully depleted, back-illuminated charge-coupled devices fabricated on high-resistivity silicon. _IEEE Transactions on Electron Devices_ 50(1):225–238. https://doi.org/10.1109/TED.2002.806476


# Learning secondary tool affordances of human partners using iCub robot’s
egocentric data

Bosong Ding1, Erhan Oztop2, Giacomo Spigler1, Murat Kirtay1 1Bosong Ding is
with the department of Cognitive Science and Artificial Intelligence, Tilburg
University, Tilburg, Netherlands<EMAIL_ADDRESS>Oztop
is with Symbiotic Intelligent Systems Research Center, Institute for Open and
Transdisciplinary, Research Initiatives, Osaka University, Japan. He is also
affiliated with Ozyegin University, Istanbul, Turkey.
<EMAIL_ADDRESS>Spigler is with the department of
Cognitive Science and Artificial Intelligence, Tilburg University, Tilburg,
Netherlands<EMAIL_ADDRESS>Kirtay is with the
department of Cognitive Science and Artificial Intelligence, Tilburg
University, Tilburg, Netherlands<EMAIL_ADDRESS>
###### Abstract

Objects, in particular tools, provide several action possibilities to the
agents that can act on them, which are generally associated with the term of
affordances. A tool is typically designed for a specific purpose, such as
driving a nail in the case of a hammer, which we call as the primary
affordance. A tool can also be used beyond its primary purpose, in which case
we can associate this auxiliary use with the term secondary affordance.
Previous work on affordance perception and learning has been mostly focused on
primary affordances. Here, we address the less explored problem of learning
the secondary tool affordances of human partners. To do this, we use the iCub
robot to observe human partners with three cameras while they perform actions
on twenty objects using four different tools. In our experiments, human
partners utilize tools to perform actions that do not correspond to their
primary affordances. For example, the iCub robot observes a human partner
using a ruler for pushing, pulling, and moving objects instead of measuring
their lengths. In this setting, we constructed a dataset by taking images of
objects before and after each action is executed. We then model learning
secondary affordances by training three neural networks (ResNet-18, ResNet-50,
and ResNet-101) each on three tasks, using raw images showing the ‘initial’
and ‘final’ position of objects as input: (1) predicting the tool used to move
an object, (2) predicting the tool used with an additional categorical input
that encoded the action performed, and (3) joint prediction of both tool used
and action performed. Our results indicate that deep learning architectures
enable the iCub robot to predict secondary tool affordances, thereby paving
the road for human-robot collaborative object manipulation involving complex
affordances.

##  I Introduction

Humans and animals rely on perceiving the possible actions (affordances) a
tool can offer to accomplish a wide array of tasks. These can be as simple as
foraging for basic survival or as complex as creating a design for an
industrial application that includes human-robot collaboration [1], [2], [3].
While tools are generally crafted with a primary function in mind, they
frequently possess the potential for other uses beyond their original intent.
Take a ruler as an instance; it is primarily used to measure the size of an
object, yet it can also serve to pull an object closer. The first use is an
example of the tool’s _primary affordance_ , while the second represents one
of its _secondary affordances_ [1].

The concept of affordance has been explored across multiple fields, such as
user interface design, human-computer interaction, and robotics. However, much
of the research has focused on recognizing and utilizing primary affordances,
especially in the field of robotics [2], [4], [5].

In this study, we addressed one of the less explored areas of affordances in
robotics, focusing on the ability of the iCub robot to learn about secondary
tool functions by observing its human partners through an egocentric
perspective. Specifically, we asked the following research question: _How can
a multisensory-enabled iCub robot learn the secondary affordances of tools
used by human partners?_ To explore this, we designed an interactive
experiment to recognize secondary affordances of tools using image data
acquired on the iCub robot via three cameras before and after operators act on
various objects. It is important to note that in our study, tools are used by
human partners in ways that go beyond their intended function. For example,
_throwability_ is a primary affordance of the boomerang tool, but in our
experiments, a boomerang was repurposed to perform various actions like
pushing and pulling objects.

We assess the ability of Convolutional Neural Networks architectures
(ResNet-18, ResNet-50, and ResNet-101) to predict the secondary affordances of
tools across three tasks. The initial task involves identifying tools from raw
images of the object captured before and after performing an action. The
second task enhances this by including information about the action being
executed. Finally, the third task aims to independently predict the tool and
action based solely on the raw images.

Our results show that the models achieve significant accuracy ($>$ 90%) while
predicting tools and tool-action pairs across three tasks. Based on these
findings, we put forward that the ResNet-based architectures, particularly
ResNet-50, suit tool affordance learning using egocentric data.

Our work offers the following contributions. Firstly, we conduct one of the
first studies using the iCub robot to understand secondary affordances in
human interactions within a real-world environment. Secondly, we deliver
extensive benchmarking results and demonstrate the effectiveness of neural
networks for learning secondary tool affordances.

The structure of this paper is as follows: Section II reviews the literature
on affordance learning in robotics. Section III details the dataset specifics
used in our research. The tasks we investigate are described in Section IV.
Our methods for developing a secondary affordance learning framework are
explained in Section V. Section VI presents results, and Section VII provides
access to the online repository for replicating these results. Lastly, Section
VIII summarizes our conclusions and outlines potential directions for future
work.

##  II Related Work

Tool usage has been one of the active research areas in robotics [6], [7]. Due
to its broad scope, here, we review the studies on tool affordance prediction
and learning studies in robotics. We also note that most studies in robotics
concern primary rather than secondary affordances [2], [8], [9].

Varadarajan and Vince [10] present the extension of affordance computing
initiative (AfNet) to robotics applications, AfRob, to employ vision data to
recognize object affordances, such as containability, rollability, grab
supportability, etc. Although the motivation of the study was framed for
social robots in a domestic environment, the authors did not provide an
application for a real robot. Chu et al. [11] employ a manipulator with RGB-D
sensor to detect and rank objects’ multi-affordances: primary (e.g., grasping
a hammer from handle) and non-primary (e.g., grasping a hammer from head)
affordances of objects. The authors designed a convolutional neural network
(CNN) architecture with an RGB-D data processing pipeline to detect
affordances by labeling each image pixel for different affordance types. Lai
et al. [12] used a deep learning architecture to determine the regions of the
different objects that are functionally the same (i.e., functional
correspondence problem) in performing a task, such as the body of a bottle and
the front part of a shoe can be used a task of pounding. Here, the authors
draw parallels between the secondary affordances of the objects and functional
correspondence. To realize their approach, the authors presented a dataset of
RGB images for different objects with the same affordances (i.e., actions).
The CNN architecture was adopted to detect functionally corresponding points
in images. Do et al. [13] proposed a convolutional network-based primary
affordance and object detection framework. The results show that the
implementation outperforms some state-of-the-art models on affordance
detection tasks using RGB images from off-the-shelf datasets. The authors
provide a robotic application to offer the same approach that can be employed
on a real robot platform to perform a task that requires grasping affordances,
e.g., pouring a bottle.

Based on the studies introduced above, we can conclude that our work offers
the following differences. On the one hand, unlike existing studies that
enable robots to detect object affordances, our study employs the iCub robot
to learn human secondary affordances using egocentric data. On the other hand,
we achieve the results in a noisy real-world setting where the iCub robot is
physically co-located with human partners.

##  III Dataset Acquisition Setup and Specifications

In this section, we describe the dataset acquisition procedure for learning
the secondary affordances of human partners by employing the iCub humanoid
robot [14].

Our experimental setup consists of the upper body of the iCub robot that has
three color cameras (two Dragonfly cameras in each eye and an Intel Realsense
d435i camera located above the eyes). Figure 1(a) shows a photo from one of
the experiments where a right-handed operator uses a ruler as a tool to
perform an action that is associated with its secondary affordance, i.e.,
pulling a wooden cube with a ruler.

(a) Experiment setup

(b) Objects

(c) Tools

Figure 1:  The experimental setup (a), where an operator performs a pull
action on a wooden cube with a ruler as a tool, the objects (b) and tools (c)
that were employed to construct the dataset.

In this setting, we designed the experiment where the iCub robot becomes an
action observer, and four different human partners (i.e., operators) are
assigned to be action performers. To construct the dataset, we used 20 objects
varying in material, color, size, shape, etc.

As shown in Figure 1(a), the operator performs predefined actions (namely,
push, pull, left to right, and right to left) on the object using a tool. The
objects and tools (boomerang, ruler, slingshot, and spatula) used in this
experimental setup are shown in Figure 1(b), 1(c), respectively. We note that
the operators do not perform primary affordance of the tools. For example, see
Figure 1(a), the operator did not measure the size of an object with the ruler
as a tool; instead, the ruler was used for performing predefined actions:
pulling, pushing, moving an object from left to right, and moving an object
from right to left.

(a) Initial pose of the object

(b) Final pose of the object

Figure 2: (a) Initial pose of the object and (b) final pose of the object
after performing an action (left to right) with ruler as a tool.

We captured images using three cameras to record the initial and final poses
of objects before and after actions were performed. Figure 2 illustrates the
initial and final pose of a wooden cube object from one of the cameras located
on the robot. Each object was subject to an action using a specific tool, and
this process was repeated 10 times for every combination of object, action,
and tool. After the experiment, we acquired a total of 3200 samples, each
consisting of 6 color images with a resolution of $640\times 480$ pixels
(i.e., each sample one of all combinations of 20 objects, 4 tools, 4 actions,
10 repetitions, and consist of initial and final pairs of images for each of
the three cameras).

We note that the robot’s role in this setting is to observe human partners’
actions using a set of tools. However, the results presented in this work can
be used as a foundation for employing the iCub robot in a human-robot
collaboration task where the robot actively interacts with the human partner
and performs secondary affordances on the tools.

##  IV Tasks

This paper focuses on two tasks: 1) tool recognition with and without actions
as input and 2) predicting actions and tools that match the change in state of
an object after manipulation by a human partner.

###  IV-A Tool Recognition

In tool recognition task, the model is provided with initial and final images
and information regarding the action used for object manipulation. The
objective is for the model to predict the tools used for the observed action.
Additionally, we conducted experiments using models that received explicit
information about the action performed.

###  IV-B Action and Tool Recognition

This task involves feeding the models with initial and final images. The
objective is to recognize the specific combination of tool and action
responsible for transitioning between the initial and final images,
determining the exact action and tools used in manipulating the object. With
four tools and four actions, the goal is to identify the appropriate
combination of action and tool accurately.

##  V Methods

In this section, we explain the pre-processing of the dataset to create
suitable input-output pairs for the tasks. Next, we discuss the network
architectures employed for the tool affordance learning. We note that
materials are made available for reproducibility of the study on our GitHub
repository, see Section VII.

Figure 3: Schematic representation of the ResNet-50 architecture for tool-
action pair prediction described in Section IV-B. The architecture uses paired
initial and final images processed through shared-weight ResNet layers. The
feature maps generated using two independent branches are merged and fed into
a two-headed classifier to simultaneously predict the tools and actions.

###  V-A Data Pre-processing

We explored the tasks introduced in Section IV using four different tools
(boomerang, ruler, slingshot, and spatula) and four pre-defined actions (push,
pull, left-to-right, right-to-left) across a range of 20 objects. For the tool
prediction tasks, the goal is to predict one out of four possible tools. The
action+tool recognition task is more complex, requiring the correct prediction
of action and tool simultaneously. Each data sample includes ‘initial’ and
‘final’ RGB images, representing the scene before and after manipulation,
respectively, taken from three robot-mounted egocentric cameras (center, left,
right). These images are resized to $128\times 128$ pixels and normalized. In
the second task, we incorporate an extra input as a one-hot vector encoding
the action taken. The dataset is split into training, validation, and test
sets with a ratio of 6:2:2, across action-tool repetitions for each object.

###  V-B Action and Tool Recognition Models

Action and tool recognition is performed by training Convolutional Neural
Networks (CNNs) [15] on the three target tasks due to their proven
effectiveness on image classification problems. To benchmark the results, we
compare three variants of the ResNet architecture [16]. ResNet-18, comprising
11.7 million parameters; ResNet-50, with 25.6 million parameters; and
ResNet-101, with 44.5 million parameters. We implemented these ResNet
architectures because the studies show that they are a particularly effective
neural network architecture whose main feature is ‘skip-connection’ that
improves the flow of gradients across many layers without incurring the
problem of vanishing gradients [17].

###  V-C Network Architectures

We evaluate various ResNet-based network architectures on action and tool
recognition tasks to select the most effective approach. We employed these
networks to determine the differences between the initial and final
configurations in each task. Therefore, we anticipate a significant variance
in performance among models based on how they process these representations.
Specifically, we investigate five distinct methods of handling the data
samples. Here, _C_ and _N_ stand for camera and network, respectively.

  1. 1.

_Stacked-channels (3C-1N)_ : We stack the initial and final images from all
three cameras (i.e., 2x3x3=18 image channels, as
{initial,final}x{left,center,right}x{red,green,blue}), processed by a single
ResNet. Although this is the simplest model we examine in this study, it might
face issues due to the lack of pixel overlap among images from different
cameras and varying initial and final states.

  2. 2.

_Separate (3C-6N)_ : Here, we treat each of the six images (initial and final
from left, center, and right cameras) independently. Each image is processed
by its own ResNet, leading to six distinct networks. The resulting embeddings
are concatenated to form the input to a final common output layer, as
illustrated in Fig. 3.

  3. 3.

_Separate-central (1C-2N)_ : Only the central camera’s initial and final
images are considered, each being processed by a separate ResNet. This gives
rise to two individual neural networks.

  4. 4.

_Shared (3C-3N)_ : In this configuration, each initial and final image is
individually processed through the same network dedicated to each camera,
using shared weights. This means we employ three networks, with each used
twice. The embeddings from all networks are combined as in the previous models
(see Fig. 3).

  5. 5.

_Shared-central (1C-1N)_ : A single ResNet is used for both the initial and
final images from just the central camera, leveraging the same setup as the
‘Shared’ model, but with fewer cameras involved.

The above-listed 5 network architectures can be broadly categorized into three
groups. First, the stacked input model (3C-1N) serves as a baseline to
evaluate the efficacy of different input data processing methods, as in
computer vision, dual input, and stacking are common choices for handling
multiple image inputs. Second, the networks with shared weights, specifically
the 1C-1N and 3C-3N models, are based on the assumption that using shared
weights to process the initial and final images will effectively highlight the
differences. Third, the separate networks approach aims to leverage the full
capabilities of CNNs to detect changes in the input images, with the
individual networks operating independently. For simplicity, we use the 1C-1N
model as a representative example, illustrating its detailed network
architecture in Figure 3. The primary distinction for separate networks is the
non-shared weights. In the case of three cameras, this approach involves
tripling the networks and concatenating a total of six visual embeddings. The
network architecture for the stacked input model is relatively
straightforward, with a single ResNet processing all inputs.

For the last layer of all architectures, we treat tools and actions as
independent labels. Consequently, we employ a dual-head network architecture
for the action-tool recognition tasks. This design allows for separate
processing and classification of tool and action labels.

###  V-D Evaluation

We applied a consistent evaluation approach across all network architectures.
To do this, we trained each network from scratch for 150 epochs using a cross-
entropy loss function. During training, we continually saved the model weights
that achieved the highest validation set accuracy and used them for the final
evaluation on the test set to prevent over-fitting. For hyper-parameter
optimization, we performed a grid search covering learning rates (1e-3, 5e-4,
1e-4), batch sizes (16, 32, 64, 128), and first-block kernel sizes (3x3, 5x5,
7x7) with strides (1x1, 2x2), over the 16-way action-tool prediction task. The
optimal hyper-parameters identified for each model were then used for all
tasks. To reduce variance due to randomness and ensure our results were
reproducible, we averaged the outcomes (i.e., accuracy) across 5 random seeds,
and reported the 95% confidence interval.

(a) Tool recognition with action

(b) Tool recognition without action

(c) ‘Tool’ output head (tool+action prediction network)

(d) ‘Action’ output head (tool+action prediction network)

Figure 4: Normalized Confusion Matrices for ResNet50-based 1C-1N Architecture:
(a) Tool-Only Recognition with Action Reference, (b) Tool-Only Recognition
without Action Reference, (c) Tool Recognition Output Head, and (d) Action
Recognition Output Head. The visualization of these matrices demonstrates the
model’s high accuracy in different scenarios, with the action recognition
component achieving near-perfect accuracy across all actions and tool
recognition showing robust performance.

##  VI Results

This section presents the results of secondary tool affordance learning. We
first introduce statistics of different network architectures and the outcomes
of the best-performing models. We then illustrate confusion matrices for each
task to visually present the accuracy and misclassification rates, offering
insights into the model’s performance across various tasks.

The initial experiment aimed to evaluate the predictive accuracy of the five
models on a task predicting both actions and tools. This task was selected to
effectively represent secondary affordance prediction behavior using the
multisensory data with three cameras from the iCub. Table I presents each
model’s accuracy along with a 95% confidence interval.

TABLE I:  Network Architectures’ Accuracy and 95% confidence interval on Action and Tool Recognition Task  Architectures | ResNet18 | ResNet50 | ResNet101
---|---|---|---
Stacked-channels |  |  |
(3C-1N) | 76.68 ± 1.31 | 80.62 ± 2.70 | 79.00 ± 1.79
Separate |  |  |
(3C-6N) | 81.68 ± 3.35 | 73.90 ± 1.30 | 71.28 ±1.39
Shared |  |  |
(3C-3N) | 81.56 ± 3.54 | 86.09 ± 3.02 | 80.87 ± 1.01
Separate-central |  |  |
(1C-2N) | 78.06 ± 2.57 | 76.31 ± 5.10 | 72.43 ± 1.99
Shared-central |  |  |
(1C-1N) | 86.06 ± 2.05 | 90.78 ± 4.18 | 85.15 ± 5.24

The results in Table I above show that networks using shared-weights (3C-3N
and 1C-1N) to process the initial and final images perform better than those
with stacked-channel inputs (3C-1N) or separate networks (3C-6N, 1C-2N). This
finding indicates that processing images independently or stacking them could
interfere with comparing the initial and final states, highlighting the
benefits of shared-weight networks for this task.

It is interesting to note that models that used all cameras as input (3C-*
models) were found to perform worse than those that only used the central
camera (1C-* models) on either separate networks or shared weights settings.
We hold that this decline may be due to the inferior quality of the iCub’s
left and right cameras compared to the central camera.

Following the evaluation, the 1C-1N network architecture stands out as the
most effective. Therefore, we selected this network for the remaining
experiments. Figure 3 depicts this architecture within the action and tool
recognition task context. When predicting tools alone, action inputs are
represented as one-hot vectors and combined with the visual embeddings from
the ResNets, improving the precision of the secondary affordance predictions.
We then applied the 1C-1N network to all three affordance recognition tasks.
In the “tools” task, we used images of the object’s initial and final poses,
as well as action information. In the “tools no actions” scenario, we relied
solely on the initial and final images for tool prediction. Lastly, in the
“tools+actions” task, we used initial and final images to recognize
combinations of tools and actions. The accuracy of each task is reported in
Table II.

TABLE II: Accuracy and 95% confidence interval of ResNet-based 1C-1N architectures in Tool Recognition Tasks Output | Class | Models | Accuracy (%)
---|---|---|---
Tools | 4 | ResNet18 | 89.00 ± 2.21
|  | ResNet50 | 94.03 ± 1.23
|  | ResNet101 | 93.06 ± 2.38
Tools (no Actions) | 4 | ResNet18 | 88.78 ± 2.30
|  | ResNet50 | 94.21 ± 0.92
|  | ResNet101 | 92.50 ± 2.02
Tools+Actions | 4+4 | ResNet18 | 86.06 ± 2.05
|  | ResNet50 | 90.78 ± 4.18
|  | ResNet101 | 85.15 ± 5.23

Based on the entries in Table II, we found that the ResNet50 architecture
consistently outperformed the other two across all tasks, including the
challenging task of recognizing tools and actions simultaneously.

Finally, we examine the error patterns across the three tasks by analyzing the
normalized confusion matrices for the ResNet50-based 1C-1N architecture, as
shown in Figure 4. The results show no obvious misclassification pattern,
which may be attributable to the difference in action and tool type. However,
it is evident that “action” prediction is easier for the network than “tool”
prediction, as seen in panels (c) and (d).

###  VI-A Supplementary results and ablation study

To further look into the relative difficulty of recognizing actions rather
than tools, we re-trained the CNNs on an action-only prediction task. The
results reveal that this task can be effectively solved with more than 99%
accuracy with all models we tested (see section V). This highlights the
following aspects of our dataset. It is comparatively easier to discriminate
between tools and actions when considered separately than to determine the
joint combination of tools and actions. This indicates that for any given
action, different tools could produce a similar result (i.e., the final pose
of the object). In addition, a single tool is also suitable for multiple
actions.

We also conducted an ablation study in which we performed explicit
classification over all possible combinations of the 4 actions and 4 tools,
equating to 16 categories. However, this approach did not yield performance
comparable to the dual-head networks, where tools and actions were classified
separately. To explain this outcome, we propose that the 16-class
classification may cause redundant network weights in the final layer, unlike
the streamlined, dual-headed network structure.

It is important to note that in our experiments, all tools were used to
perform all actions. Yet, in real-life scenarios, tools are typically not
interchangeable. This means that the likelihood of actions and tools occurring
together might not easily separate into distinct probabilities for each. In
such cases, using a single output layer that predicts all possible action-tool
combinations could be advantageous.

##  VII Reproducibility of The Study

We used a public repository111https://github.com/dingdingding60/HUMANOIDS-2024
for resources –including scripts, trained models, ablation parameters,
figures, and software packages with version numbers– to reproduce the results.

##  VIII Conclusions

In this work, we investigated several variants of ResNet-based deep neural
network architectures and highlighted most successful ones that can serve as a
subsystem for the cognitive architecture of a humanoid robot. In particular,
we have shown that ResNet-based deep learning models enable the iCub robot to
learn secondary tool affordances based on tool use demonstrations of human
partners. These models process egocentric visual data, i.e., images capturing
the initial and final positions of objects when actions are performed with a
tool, to make predictions about the tools. Based on the presented results, we
suggest that interactive robots could learn to understand tool uses, aiding in
collaborative tasks like furniture assembly or kitchen help.

Given that our dataset allows for the use of various tools to accomplish the
same task, we emphasize that this aspect is particularly significant for
learning secondary tool affordances. Overall, our future research will
continue in this direction, further exploring how to integrate multi-sensory
data in a human-robot collaboration task, where the robot actively interacts
with the human partners.

## References

  * [1] L. Ye, W. Cardwell, and L. S. Mark, “Perceiving multiple affordances for objects,” _Ecological Psychology_ , vol. 21, no. 3, pp. 185–217, 2009.
  * [2] L. Jamone, E. Ugur, A. Cangelosi, L. Fadiga, A. Bernardino, J. Piater, and J. Santos-Victor, “Affordances in psychology, neuroscience, and robotics: A survey,” _IEEE Transactions on Cognitive and Developmental Systems_ , vol. 10, no. 1, pp. 4–25, 2016.
  * [3] F. Stramandinoli, A. Roncone, O. Mangin, F. Nori, and B. Scassellati, “An affordance-based action planner for on-line and concurrent human-robot collaborative assembly,” in _2nd ICRA International Workshop on Computational Models of Affordance in Robotics_ , 2019.
  * [4] D. Norman, _The design of everyday things: Revised and expanded edition_. Basic books, 2013.
  * [5] N. Masoudi, G. M. Fadel, C. C. Pagano, and M. V. Elena, “A review of affordances and affordance-based design to address usability,” in _Proceedings of the Design Society: International Conference on Engineering Design_ , vol. 1, no. 1. Cambridge University Press, 2019, pp. 1353–1362.
  * [6] K. P. Tee, J. Li, L. T. P. Chen, K. W. Wan, and G. Ganesh, “Towards emergence of tool use in robots: Automatic tool recognition and use without prior tool learning,” in _2018 IEEE international conference on robotics and automation (ICRA)_. IEEE, 2018, pp. 6439–6446.
  * [7] S. Brown and C. Sammut, “Tool use learning in robots,” in _2011 AAAI Fall Symposium Series_ , 2011.
  * [8] P. Zech, S. Haller, S. R. Lakani, B. Ridge, E. Ugur, and J. Piater, “Computational models of affordance in robotics: a taxonomy and systematic classification,” _Adaptive Behavior_ , vol. 25, no. 5, pp. 235–271, 2017.
  * [9] H. Min, R. Luo, J. Zhu, S. Bi _et al._ , “Affordance research in developmental robotics: A survey,” _IEEE Transactions on Cognitive and Developmental Systems_ , vol. 8, no. 4, pp. 237–255, 2016.
  * [10] K. M. Varadarajan and M. Vincze, “Afrob: The affordance network ontology for robots,” in _2012 IEEE/RSJ international conference on intelligent robots and systems_. IEEE, 2012, pp. 1343–1350.
  * [11] F.-J. Chu, R. Xu, L. Seguin, and P. A. Vela, “Toward affordance detection and ranking on novel objects for real-world robotic manipulation,” _IEEE Robotics and Automation Letters_ , vol. 4, no. 4, pp. 4070–4077, 2019.
  * [12] Z. Lai, S. Purushwalkam, and A. Gupta, “The functional correspondence problem,” in _Proceedings of the IEEE/CVF International Conference on Computer Vision_ , 2021, pp. 15 772–15 781.
  * [13] T.-T. Do, A. Nguyen, and I. Reid, “Affordancenet: An end-to-end deep learning approach for object affordance detection,” in _2018 IEEE international conference on robotics and automation (ICRA)_. IEEE, 2018, pp. 5882–5889.
  * [14] M. Kirtay, U. Albanese, L. Vannucci, G. Schillaci, C. Laschi, and E. Falotico, “The icub multisensor datasets for robot and computer vision applications,” in _Proceedings of the 2020 International Conference on Multimodal Interaction_ , 2020, pp. 685–688.
  * [15] Y. LeCun, Y. Bengio, and G. Hinton, “Deep learning,” _nature_ , vol. 521, no. 7553, pp. 436–444, 2015.
  * [16] K. He, X. Zhang, S. Ren, and J. Sun, “Deep residual learning for image recognition,” in _Proceedings of the IEEE conference on computer vision and pattern recognition_ , 2016, pp. 770–778.
  * [17] H. Li, Z. Xu, G. Taylor, and T. Goldstein, “Visualizing the loss landscape of neural nets,” _CoRR_ , vol. abs/1712.09913, 2017. [Online]. Available: http://arxiv.org/abs/1712.09913