# Comparing SCUBA-2 and ALMA Selections of Faint DSFGs in Abell 2744

S. J. McKay Department of Physics, University of Wisconsin–Madison, 1150
University Avenue, Madison, WI 53706, USA  A. J. Barger Department of
Astronomy, University of Wisconsin–Madison, 475 N. Charter Street, Madison, WI
53706, USA  Department of Physics and Astronomy, University of Hawaii, 2505
Correa Road, Honolulu, HI 96822, USA  Institute for Astronomy, University of
Hawaii, 2680 Woodlawn Drive, Honolulu, HI 96822, USA  L. L. Cowie Institute
for Astronomy, University of Hawaii, 2680 Woodlawn Drive, Honolulu, HI 96822,
USA

###### Abstract

We make a comparison of deep SCUBA-2 450 $\mu$m and 850 $\mu$m imaging on the
massive lensing cluster field Abell 2744 with Atacama Large
Millimeter/submillimeter Array (ALMA) 1.2 mm data. Our primary goal is to
assess how effective the wider-field SCUBA-2 sample, in combination with red
JWST priors, is for finding faint dusty star-forming galaxies (DSFGs) compared
to the much more expensive mosaicked ALMA observations. We cross-match our
previously reported direct ($>5\sigma$) SCUBA-2 sample and red JWST NIRCam
prior-selected ($>3\sigma$) SCUBA-2 sample to direct ALMA sources from the
DUALZ survey. We find that roughly 95% are confirmed by ALMA. The red priors
also allow us to probe deeper in the ALMA image. Next, by measuring the 450
$\mu$m and 850 $\mu$m properties of the full ALMA sample, we show that 46/69
of the ALMA sources are detected at 850 $\mu$m and 24/69 are detected at 450
$\mu$m in the SCUBA-2 images, with a total detection fraction of nearly 75%.
All of the robust ($>5\sigma$) ALMA sources that are not detected in at least
one SCUBA-2 band lie at 1.2 mm fluxes $\lesssim$ 0.6 mJy and are undetected
primarily due to the higher SCUBA-2 flux limits. We also find that the SCUBA-2
detection fraction drops slightly beyond $z=3$, which we attribute to the
increasing 1.2 mm to 850 $\mu$m and 1.2 mm to 450 $\mu$m flux ratios combined
with the ALMA selection. The results emphasize the power of combining SCUBA-2
data with JWST colors to map the faint DSFG population.

††facilities: ALMA, JCMT††software: astropy (Astropy Collaboration et al.,
2022)

##  1 Introduction

The demonstration by the COBE satellite that the cosmic infrared background
(CIB) is comparable to the cosmic optical background showed that a significant
fraction of the UV/optical starlight in the Universe is absorbed and
reradiated by dust (Puget et al., 1996; Fixsen et al., 1998; Hauser et al.,
1998; Dole et al., 2006). Resolving the CIB into individual sources has been
one of the fundamental goals of modern astronomy for over two decades, which
led to the submillimeter detection of extremely luminous, dusty star-forming
galaxies (DSFGs; or submillimeter galaxies, SMGs) with star formation rates
(SFRs) up to $\sim$1000 M⊙/yr (Smail et al., 1997; Barger et al., 1998; Hughes
et al., 1998; Eales et al., 1999).

Due to the relatively low surface density of DSFGs and the strong negative
K-correction at submillimeter and millimeter wavelengths, wide-field single-
dish submillimeter/millimeter surveys have been most effective at obtaining
large samples of these galaxies. Currently, the most efficient instruments for
this are SCUBA-2 (Holland et al., 2013) at 450 $\mu$m and 850 $\mu$m on the
James Clerk Maxwell Telescope (JCMT), NIKA2 (Monfardini et al., 2014; Calvo et
al., 2016) at 1.2 mm and 2 mm on the IRAM 30 m telescope, and soon TolTEC
(Wilson et al., 2020) from 1.1 mm to 2 mm on the Large Millimeter Telescope
(LMT).

However, single-dish submillimeter surveys are typically limited to picking
out bright SMGs ($f_{850\mu{\rm m}}>2$ mJy), because the low angular
resolution (e.g., 14′′ for SCUBA-2 at 850 $\mu$m) causes the blending of faint
unresolved sources to dominate the noise below a certain flux level (the
confusion limit; e.g., $\sim$1.6 mJy for SCUBA-2 at 850 $\mu$m; Cowie et al.
2017). The bright SMG population constitutes $\sim 20$–30% of the CIB. An
effective method for pushing to deeper survey depths is to leverage the
gravitational lensing of massive galaxy clusters (e.g., Smail et al., 1997;
Cowie et al., 2002), which offers the additional benefit of increased spatial
resolution. Such studies show that the majority of the CIB is made up of faint
SMGs ($f_{850\mu{\rm m}}\lesssim 1$ mJy) (e.g., Knudsen et al., 2008; Chen et
al., 2013; Hsu et al., 2016).

In the last decade, substantial efforts have been made to study the faint end
of the DSFG population. Interferometers, such as the Atacama Large
Millimeter/submillimeter Array (ALMA), are capable of probing to deeper flux
limits than SCUBA-2, but the small field-of-view means that direct searches
using mosaicked observations (e.g., Hatsukade et al., 2016; Aravena et al.,
2016; Walter et al., 2016; Dunlop et al., 2017; Umehata et al., 2017;
González-López et al., 2017; Muñoz Arancibia et al., 2018; Hatsukade et al.,
2018; González-López et al., 2020; Gómez-Guijarro et al., 2022; Casey et al.,
2021; Zavala et al., 2021; Muñoz Arancibia et al., 2023; Fujimoto et al.,
2023a, b) are very costly.

Using JWST data from multiple NIRCam programs (JWST-ERS-1324, JWST-GO-2561,
JWST-DDT-2756) on the Abell 2744 (A2744) cluster field (Paris et al., 2023;
Weaver et al., 2023), along with ALMA data from the ALMA Lensing Cluster
Survey (ALCS; 1.2 mm) (Fujimoto et al., 2023a) and the ALMA Frontier Fields
Survey (AFFS; 1.1 mm) (González-López et al., 2017; Muñoz Arancibia et al.,
2023) and SCUBA-2 data from Cowie et al. (2022), Barger & Cowie (2023)
developed a JWST NIRCam red galaxy selection that identified all known
($>4.5\sigma$) ALMA sources (9) and nearly all known ($>5\sigma$) SCUBA-2 850
$\mu$m sources (17 out of 19). They then used the red galaxy priors to probe
deeper into the SCUBA-2 data ($>3\sigma$), finding sources down to faint
intrinsic 850 $\mu$m fluxes of $\sim$0.1 mJy with the help of gravitational
lensing.

Fujimoto et al. (2023b) subsequently presented the DUALZ ALMA 1.2 mm survey,
which maps most of the JWST NIRCam region of the A2744 UNCOVER program
(Bezanson et al., 2022). They released a catalog of 69 ALMA sources down to a
minimum S/N of 4.2. However, they did not cross-reference their sources with
the SCUBA-2 data to see how many of their sources had already been detected.
In addition to SCUBA-2 covering much larger areas than ALMA, its extended
wavelength coverage is important for studying the physical properties of
DSFGs, since 450 $\mu$m and $850~{}\mu$m lie closer to the far-infrared (FIR)
peak of the dust emission and thus can help constrain dust temperatures and
redshifts (e.g., Casey et al., 2013; Zavala et al., 2018; Lim et al., 2020;
Barger et al., 2022).

In this paper, we revisit the overlap between the SCUBA-2 and ALMA samples in
A2744, incorporating the new DUALZ data. In Section 2, we describe the data
that we use in our analysis. In Section 3, we compare the ALMA sample to the
direct and red galaxy prior-based SCUBA-2 samples, and we also recover fainter
ALMA sources using the priors. In Section 4, we use the 450 $\mu$m and 850
$\mu$m data to learn more about the properties of the ALMA sources. In Section
5, we contrast the properties of sources detected in the SCUBA-2 maps against
those that are not, and we assess how effective the red galaxy priors are for
recovering SCUBA-2 and ALMA sources. Finally, in Section 6, we summarize our
results.

We assume a flat concordance $\Lambda$CDM cosmology with $\Omega_{M}=0.3$,
$\Omega_{\Lambda}=0.7$, and $H_{0}=70.0$ km s-1 Mpc-1.

Figure 1:  JWST NIRCam RGB image (R: F444W, G: F150W, B: F115W) from Paris et
al. (2023) with the outline of the approximate ALMA DUALZ coverage ($\sim$24
arcmin2) from Fujimoto et al. (2023b) overlaid in yellow. Red squares mark the
positions of the SCUBA-2 sources with red NIRCam counterparts from Barger &
Cowie (2023) (either direct $>5\sigma$ 850 $\mu$m detections or selected using
the red galaxy priors; 49 sources). Note that one of the direct detections has
a different red NIRCam counterpart confirmed by ALMA than was originally found
by Barger & Cowie (2023). We show the DUALZ sources in green if $>5\sigma$ (59
sources) and in magenta if $<5\sigma$ (10 sources). The SCUBA-2 imaging covers
the entire NIRCam region and extends significantly beyond it.  Figure 2: ALMA
1.2 mm flux density vs. SCUBA-2 850 $\mu$m flux density for sources in either
the DUALZ or SCUBA-2 catalogs. The $>5\sigma$ SCUBA-2 sources are shown as
squares. The $>3\sigma$ SCUBA-2 sources obtained with red NIRCam priors are
shown as circles. The sources that are only in the DUALZ catalog are shown as
leftward-facing arrows. The color of the points denotes the S/N of the ALMA
detection using the righthand scale (black if not in the DUALZ catalog—note
that there are three overlapping SCUBA-2 sources at $\sim$1.1 mJy on the
figure).

##  2 Data

The massive lensing cluster field A2744 ($z=0.308$) is one of the six Hubble
Frontier Fields (Lotz et al., 2017) and has been the target of hundreds of
hours of multiwavelength observations from the UV to the radio. We have been
observing A2744 with SCUBA-2 for a number of years and have generated deep 450
$\mu$m and 850 $\mu$m images covering the entire central region of the cluster
(Cowie et al., 2022). The minimum 1$\sigma$ rms of the current 850 $\mu$m
image is 0.26 mJy, while the minimum 1$\sigma$ rms of the current 450 $\mu$m
image is 2.8 mJy, slightly deeper than those presented in Cowie et al. (2022).
The reduction of the SCUBA-2 maps and the source detection follow that
described in Chen et al. (2013) and Cowie et al. (2017).

A2744 has been observed in several JWST surveys, including deep NIRCam imaging
from the UNCOVER (Bezanson et al., 2022), GLASS-JWST (Treu et al., 2022), and
DDT #2756 programs (PI: W. Chen; Roberts-Borsani et al. 2023). In addition to
lensing models released by five independent teams prior to JWST, Furtak et al.
(2023) published a lensing model for the cluster region using the UNCOVER JWST
observations.

The ALMA data for this field come primarily from the DUALZ program (Fujimoto
et al., 2023b), which mapped the UNCOVER NIRCam area with ALMA Band 6
observations (effective wavelength of 1.2 mm) down to a minimum rms noise of
32.7 $\mu$Jy. The data from the 1.2 mm ALCS (Fujimoto et al., 2023a) and 1.1
mm AFFS (González-López et al., 2017; Muñoz Arancibia et al., 2023) surveys
were combined into the Deep maps. The Wide maps use only the DUALZ
observations. From the Deep and Wide maps (both with and without $uv$-tapering
applied), Fujimoto et al. (2023b) produced a source catalog of 69 ALMA sources
down to a minimum S/N of 4.2 (their detection threshold varied depending on
which map was used for the detection). In Figure 1, we show their 69 sources
and the coverage of the DUALZ survey overlaid on the JWST NIRCam image of
Paris et al. (2023).

##  3 ALMA properties of SCUBA-2 sources

Restricting to the DUALZ ALMA area, we first compare the 17 $>5\sigma$ SCUBA-2
sources with the DUALZ catalog. Barger & Cowie (2023) found red NIRCam
counterparts (defined as $f_{\rm F444W}$/$f_{\rm F150W}$ $>$ 3.5 and $f_{\rm
F444W}$$>1$ $\mu$Jy) within 4′′ of 15 of these SCUBA-2 sources (note that in
one case, two SCUBA-2 sources share one counterpart). The ALMA data confirm
all but one of these; for the remaining source, the ALMA detection matches to
a different but similarly red galaxy at a slightly larger separation from the
SCUBA-2 centroid. The two SCUBA-2 sources without a red NIRCam counterpart do
not appear to have ALMA detections, though for one, there is an ALMA source at
5.3′′ from the SCUBA-2 position. In summary, 15/17 (88%) of the $>5\sigma$
SCUBA-2 detections have ALMA 1.2 mm counterparts.

With the ALMA data, we can check whether multiple DSFGs may be contributing to
the 850 $\mu$m flux. We find that only two of the 15 $>5\sigma$ SCUBA-2
sources (SCUBA7 and SCUBA8) have multiple ALMA counterparts within $4\farcs 5$
of the SCUBA-2 850 $\mu$m position, giving a multiplicity of 13% (2/15).

Barger & Cowie (2023) also used the red NIRCam galaxies in the field as priors
to identify fainter sources than could be securely extracted with the SCUBA-2
data alone. They compiled a sample of 44 priors with associated $>3\sigma$
SCUBA-2 fluxes (their Table 3), 39 of which fall within the DUALZ area. Twelve
of these correspond to 13 of the 15 direct $>5\sigma$ SCUBA-2 sources with red
NIRCam counterparts in the area, since two SCUBA-2 sources have the same red
counterpart. The other two sources (SCUBA9 and SCUBA12) are not included in
this sample due to differences in the SCUBA-2 deblending when using priors. We
show the positions of all the red NIRCam counterparts to the SCUBA-2 sources
in Figure 1 with red squares.

Thirty-one of the 39 sources match to the DUALZ catalog, while eight do not.
Combining these 31 with SCUBA9 and SCUBA12, the SCUBA-2 sources account for
33/69 (48%) of the DUALZ ALMA sample. If we consider only the 59 $>5\sigma$
DUALZ sources, then the SCUBA-2 sources account for 32/59 (54%).

In Figure 2, we show the 1.2 mm and 850 $\mu$m fluxes for any ALMA or SCUBA-2
source in the DUALZ area, distinguishing between those with direct $>5\sigma$
SCUBA-2 detections (squares) and those based on the red galaxy priors
(circles). We show the ALMA sources not matched to a SCUBA-2 source as
leftward-facing arrows, and the $>5\sigma$ SCUBA-2 detections not in the DUALZ
catalog in black along the bottom. For the sources detected in both ALMA and
SCUBA-2, the median 850 $\mu$m to 1.2 mm flux ratio is 2.6, with a 16–84th
percentile range of 1.8 to 5.4. From Figure 2, we can also see that most of
the sources detected by DUALZ but not by SCUBA-2 are faint and/or lower
signal-to-noise (S/N) ALMA sources (righthand color scale).

Next, we investigate whether any of the eight $>3\sigma$ red prior-selected
SCUBA-2 sources not in the DUALZ catalog are detected in the ALMA image at a
lower significance. To test this, we measure the ALMA 1.2 mm fluxes at the
positions of these eight SCUBA-2 $>3\sigma$ sources by finding the largest
local peak within $1\farcs 0$ of the red NIRCam prior, using the natural
weighted DUALZ Wide map with primary beam correction applied. The local noise
is estimated by measuring the dispersion in random pixels within 12′′ of the
source, after masking out any $>4\sigma$ sources nearby.

We find that of these eight sources, six are detected at $>2.5\sigma$ (three
at $>3\sigma$). However, we note that one of the $>3\sigma$ sources falls near
the edge where the noise is higher, so the error measured by our method may be
an underestimate of the actual uncertainty for this source. In Figure 3, we
show the 1.2 mm and 850 $\mu$m fluxes for all the SCUBA-2 sources in the DUALZ
area, including the eight sources for which we measure the ALMA fluxes. We
show the SCUBA-2 sources that match to direct DUALZ detections as blue
squares, and mark the median 850 $\mu$m to 1.2 mm flux ratio of 2.6 on the
plot (dashed line). The six red prior-selected sources for which we measure
$>2.5\sigma$ 1.2 mm fluxes are shown as red circles, while the two sources at
lower significance are shown as black triangles. The deepest 2$\sigma$ and
3$\sigma$ limits of the DUALZ Wide map are also shown (shaded region), though
the local 2$\sigma$ and 3$\sigma$ limits will generally be higher than this at
any given point in the maps. This figure illustrates that the red NIRCam
priors enable us to probe to fainter ALMA fluxes than would be possible with a
direct search.

In summary, we find that within the DUALZ area, 37/39 of the red prior-
selected SCUBA-2 sources from Barger & Cowie (2023) are detected at
$>2.5\sigma$ in the ALMA image (34/39 if restricted to $>3\sigma$), confirming
the robustness of this selection.

Figure 3: ALMA 1.2 mm vs. SCUBA-2 850 $\mu$m flux densities for the combined
$>5\sigma$ and $>3\sigma$ SCUBA-2 sources (Barger & Cowie 2023 Tables 2 and
3). Direct detections in the DUALZ catalog are shown as blue squares, while
sources for which we measure the ALMA flux densities are shown as red circles
if $>2.5\sigma$ and as black triangles if $<2.5\sigma$ in the DUALZ Wide map.
Also shown are the minimum 3$\sigma$ and 2$\sigma$ limits (shaded regions; the
actual limits may be higher than these at any given point). We show the median
850 $\mu$m to 1.2 mm ratio of 2.6 for the sources that are detected in the
DUALZ catalog (dashed line).

##  4 SCUBA-2 450 and 850 micron properties of ALMA sources

Having shown that a high fraction of the SCUBA-2 sources detected either
directly or with the red galaxy priors are confirmed with ALMA, we next invert
the analysis and study the SCUBA-2 450 $\mu$m and 850 $\mu$m properties of the
69 ALMA sources in the DUALZ catalog. All of these sources fall on the deep
SCUBA-2 450 $\mu$m and 850 $\mu$m maps from Cowie et al. (2022).

We extract the 450 $\mu$m and 850 $\mu$m fluxes and errors for the ALMA priors
using an iterative procedure to avoid contamination of fainter sources by
brighter ones: We first measure the SCUBA-2 flux at the position of each ALMA
source; then, working down the catalog from brighter to fainter SCUBA-2 flux,
we measure the peak flux and the error within 4′′ from the ALMA position and
subtract from the residual image the SCUBA-2 PSF scaled to the peak flux. We
then repeat for the next brightest SCUBA-2 flux until all sources have been
measured. Although this process is not strictly necessary for the 450 $\mu$m
maps due to the higher angular resolution, we perform the iteration on both
SCUBA-2 maps anyway.

The resulting distribution of SCUBA-2 450 $\mu$m and 850 $\mu$m fluxes for the
ALMA sample is shown in Figure 4. The mean of the respective distributions is
also shown: The mean 850 $\mu$m flux is 1.79 mJy, which is roughly six times
the central rms noise of 0.28 mJy, while the mean 450 $\mu$m flux is 8.5 mJy,
or three times the central rms of 2.8 mJy.

Figure 4: Distribution of 450 $\mu$m and 850 $\mu$m flux densities measured in
the SCUBA-2 images at the positions of the 69 DUALZ sources. The darker colors
denote the $>5\sigma$ ALMA sources. We show the mean 450 $\mu$m and 850 $\mu$m
fluxes for the full DUALZ sample (dotted lines). The minimum rms noise at 850
$\mu$m is 0.26 mJy and at 450 $\mu$m is 2.8 mJy (Cowie et al., 2022). Figure
5: ALMA 1.2 mm flux densities vs. SCUBA-2 850 $\mu$m (left) and 450 $\mu$m
(right) flux densities measured at the ALMA positions, restricting to DUALZ
sources with $>5\sigma$ ALMA detections. On the left, we distinguish between
$>2.5\sigma$ detections at 850 $\mu$m (red squares) and sources detected at
$>2.5\sigma$ at 450 $\mu$m but not 850 $\mu$m (black filled triangles); while
on the right, we show $>2.5\sigma$ detections at 450 $\mu$m (black squares)
and sources detected at $>2.5\sigma$ at 850 $\mu$m but not 450 $\mu$m (red
filled triangles). In both panels, we show sources not detected by SCUBA-2 in
either band as blue open triangles, and we show sources with negative measured
850 $\mu$m or 450 $\mu$m SCUBA-2 fluxes at a nominal flux of 0.11 mJy. The
shaded region denotes the 1.2 mm flux regime (above a cutoff threshold of
$\sim$0.6 mJy) in which the ALMA sources are all detected in the SCUBA-2 data.

We find that 37 of the ALMA sources (54%) are detected at $>3\sigma$ at 850
$\mu$m, with an additional six detected at $>2.5\sigma$. All of the direct
$>5\sigma$ SCUBA-2 sources previously matched to DUALZ sources are recovered
by this method, as are all but three of the $>3\sigma$ SCUBA-2 sources based
on the red galaxy priors that were previously matched to the DUALZ sources.
Since the $>3\sigma$ SCUBA-2 sources were extracted using the same method as
Barger & Cowie (2023), the fact that three of them are at lower significance
here is likely because the prior sample is different (ALMA versus JWST) and
thus our extraction is deblending the sources differently. Combining the 43
ALMA sources detected at $>2.5\sigma$ in the 850 $\mu$m map with the three
that were previously identified as SCUBA-2 counterparts but were not selected
by this method, we estimate an upper limit on the fraction of ALMA sources
detected at 850 $\mu$m of 46/69 (67%). By searching for peaks within 4′′ of
5000 random positions in the SCUBA-2 map, we estimate that the false positive
rate for this method is $\sim$8% at 2.5$\sigma$; in other words, we expect at
most four of our $>2.5\sigma$ detections could be false positives.

Of the 69 ALMA sources, 24 are detected at $>2.5\sigma$ at 450 $\mu$m.
Eighteen of these are also detected at 850 $\mu$m, with a median
$f_{450\mu{\rm m}}$/$f_{850\mu{\rm m}}$ ratio of 3.75. By measuring the fluxes
and errors at random positions, we estimate that the false positive rate is
3%. Thus, we expect that at most one of our 24 detections could be a false
positive. Of the six sources detected at 450 $\mu$m but not detected at 850
$\mu$m, one source corresponds to one of the three previously-identified red
priors that was deblended differently in the 850 $\mu$m extraction, as
mentioned above. This leaves five unique sources detected at 450 $\mu$m that
are not counted in the 850 $\mu$m total. If we combine the 46 sources detected
at 850 $\mu$m with these five sources, then 51/69 (74%) of the ALMA sources
have SCUBA-2 $>2.5\sigma$ detections in at least one band.

We list our measured SCUBA-2 450 $\mu$m and 850 $\mu$m fluxes, errors, and S/N
ratios for all the DUALZ sources in Table 1.

Figure 6: For sources in the DUALZ catalog with ALMA S/N $>$ 5: (left) 1.2 mm
to 850 $\mu$m flux ratio vs. redshift and (right) 1.2 mm to 450 $\mu$m flux
ratio vs. redshift. In each panel, for sources with $>2.5\sigma$ detections in
the relevant SCUBA-2 bands, photzs are shown as black circles and speczs as
red diamonds. The 2$\sigma$ lower limits (upward-pointing arrows; speczs are
red circles and photzs are gray circles) are shown for non-detections at 850
$\mu$m (left panel) or 450 $\mu$m (right panel). In each panel, we plot the
expected flux ratios across redshift for modified blackbodies with varying
emissivity indices, $\beta$, and dust temperatures, $T_{d}$ (dotted and dashed
lines—see legend).

##  5 Discussion

###  5.1 Properties of ALMA Sources Detected in SCUBA-2 versus Non-detections

The analysis of the previous section has shown that nearly 75% of the ALMA
sources in DUALZ are also detected in SCUBA-2 at either 850 $\mu$m, 450
$\mu$m, or both. Here we compare the flux and redshift properties of these
sources to study any correlations with SCUBA-2 detection rate.

We have already seen from Figure 2 that most of the DUALZ sources not in the
SCUBA-2 catalogs of Barger & Cowie (2023) are those with lower ALMA S/N ratios
and/or 1.2 mm fluxes. In Figure 5, we make this more quantitative by comparing
the ALMA fluxes to the SCUBA-2 850 $\mu$m and 450 $\mu$m fluxes directly
measured at each ALMA position, limiting the sample to the 59 DUALZ sources
with ALMA S/N $>$ 5.

We first see that down to a 1.2 mm flux limit of $\sim$0.6 mJy (shaded
region), we detect all the $>5\sigma$ ALMA sources in at least one SCUBA-2
band at $>2.5\sigma$, but below this limit, we start to lose SCUBA-2
detections. Only one of the ALMA sources (#23 in Table 1) above the 0.6 mJy
limit at 1.2 mm is detected only at 450 $\mu$m and not at 850 $\mu$m; this
source is a low-redshift ($z_{\rm spec}$ = 1.498 from Muñoz Arancibia et al.
2023) galaxy that is one of the brightest detections at 450 $\mu$m (22.1 mJy).
We can also see that the loss of SCUBA-2 detections below the 0.6 mJy cut at
1.2 mm is primarily an issue of SCUBA-2 image depth: The strong correlation
between 1.2 mm and 850 $\mu$m flux implies that fainter ALMA sources will
start to fall below the SCUBA-2 850 $\mu$m 2.5$\sigma$ detection limit
($\sim$0.8 mJy in the current images).

We next use the available redshifts for these sources to investigate how the
SCUBA-2-detected fraction (i.e., the fraction of ALMA sources with
$>2.5\sigma$ 450 $\mu$m and/or 850 $\mu$m detections) varies with redshift.
Fujimoto et al. (2023b) published photometric redshifts (photzs) estimated
with prospector (Johnson et al., 2021) and eazy (Brammer et al., 2008) and
spectroscopic redshifts (speczs) from NIRSpec prism observations for the DUALZ
catalog (both the speczs and photzs are discussed in more detail in Wang et
al. 2023). For the 27 sources with speczs, they find excellent agreement with
the photz estimates.

In Figure 6, we show the 1.2 mm to 850 $\mu$m flux ratios (left) and 1.2 mm to
450 $\mu$m flux ratios (right) versus redshift for the $>5\sigma$ DUALZ
sources with redshifts (two $>5\sigma$ ALMA sources have neither speczs nor
photzs and are not shown). We show 2$\sigma$ lower limits (upward-pointing
arrows) in cases where the SCUBA-2 fluxes have less than $2.5\sigma$
significance. In each panel, we also show the expected tracks for simple
modified blackbody models of the form $f_{\nu}\propto\nu^{\beta}B(\nu,T_{d})$
with varying emissivity indices, $\beta$, and dust temperatures, $T_{d}$ (see
figure legend; e.g., McKay et al. 2023). In the right panel, we see good
alignment between the measured 1.2 mm to 450 $\mu$m flux ratios and the
models; while in the left panel, the measured 1.2 mm to 850 $\mu$m flux ratios
show a larger degree of scatter. This is likely because the 850 $\mu$m and 1.2
mm bands lie close to each other and are therefore more susceptible to
uncertainties in the flux ratios.

There is a slight evolution with redshift in the SCUBA-2-detected fraction of
ALMA sources. Specifically, the SCUBA-2-detected fraction for the $>5\sigma$
ALMA sources with $1<z<3$ is 81%, while for $3<z<5$ it is 60%. From the
modified blackbody tracks in both panels of Figure 6, we can see that the
expected 1.2 mm to 850 $\mu$m and 1.2 mm to 450 $\mu$m flux ratios increase
from $z=0$ to $z=5$, which suggests that some of these galaxies may be
undetected simply because of the deeper ALMA flux limits combined with the
higher mm to submm flux ratios as we move to higher redshift.

Other authors have compared submm/mm flux ratios to published speczs and
photzs (Casey et al., 2013; Lim et al., 2020; Barger et al., 2022; Cowie et
al., 2023) or used them to make rough redshift estimates for sources whose
optical/near-infrared (NIR) counterparts are faint or not identified (e.g.,
Wang et al., 2019). These estimates rely on the fact that the submm bands lie
closer to the peak of the dust emission in the FIR and therefore have flux
ratios that change substantially as the spectral energy distribution (SED) is
redshifted.

For the DUALZ sources with ALMA S/N $>$ 5, we find that the 1.2 mm to 450
$\mu$m flux ratio (right panel of Figure 6) shows a steeper dependence on
redshift than the 1.2 mm to 850 $\mu$m ratio, reflecting the larger separation
between the observed wavelengths. The best-fit line is $\log(f_{1.2{\rm
mm}}/f_{450\mu{\rm m}})=0.16z-1.50$, with some scatter. The 1.2 mm to 850
$\mu$m flux ratio has a slightly shallower best-fit line of $\log(f_{1.2{\rm
mm}}/f_{450\mu{\rm m}})=0.10z-0.70$. Although other studies (e.g., Barger et
al., 2022) have found a good redshift correlation with the 450 $\mu$m to 850
$\mu$m ratio, there are not enough sources with both 450 $\mu$m and 850 $\mu$m
detections to constrain this relationship for the DUALZ sample. Deepening the
450 $\mu$m imaging on the field (which is substantially shallower than the
data used in Barger et al. 2022) would help constrain these relationships and
could help to further refine the redshifts and dust properties of these
galaxies.

We note that nearly all of the DUALZ sources are at $z\lesssim 4$, with just
two sources between $4.5<z<5$ and only one source at $z>5$. Fujimoto et al.
(2023b) discuss the latter object (#66 in Table 1; this object is not shown in
Figure 6 since it has an ALMA S/N of 4.9), which has $z_{\rm
phot}=9.89^{+1.22}_{-9.20}$, suggesting that it may be an example of a JWST-
dark galaxy. We find that this source is not detected in either the 450 $\mu$m
or 850 $\mu$m data. This is not unexpected if the source is real and located
at $z\sim 10$, since a simple modifed blackbody fit to the ALMA point would
suggest the 850 $\mu$m flux should be around 0.20 mJy (with some dependence on
the assumed $T_{d}$ and $\beta$), just below the noise level. Nevertheless,
the 450 $\mu$m and 850 $\mu$m upper limits could provide additional
constraints on the photz fitting.

###  5.2 Using Red JWST Priors to Identify Counterparts to Faint SCUBA-2
Galaxies

The expanded ALMA data in the A2744 field provided by the DUALZ survey allows
us to test how well the red NIRCam selection of Barger & Cowie (2023) picks
out the ALMA counterparts to submillimeter-bright DSFGs. As described in
Section 3, 31/39 of the red priors with $>3\sigma$ SCUBA-2 fluxes are also in
the DUALZ catalog of Fujimoto et al. (2023b), with another six detected at
$>2.5\sigma$ in the DUALZ map. This corresponds to a total fraction of red
galaxies with $>3\sigma$ SCUBA-2 fluxes confirmed by ALMA of $\sim$95%.

This result emphasizes the utility of the red color selection as a method for
probing to fainter submillimeter fluxes, and shows that one can obtain large
samples of faint DSFGs in fields with both deep SCUBA-2 and JWST coverage.
Furthermore, this red selection provides an efficient way to identify sources
for high-resolution ALMA follow-up imaging or for millimeter spectral scans at
a much lower investment of observing time than large mosaicked surveys (e.g.,
Cowie et al., 2022).

We note that a red NIRCam prior selection used to select the SCUBA-2
$>3\sigma$ sources does introduce a potential selection bias against faint
submillimeter/millimeter sources with less red NIR counterparts, including
some of the other ALMA sources in the Fujimoto et al. (2023b) catalog. In an
upcoming paper (McKay et al., 2023, in preparation), we will further discuss
the selection effects of using red counterparts to pick out DSFGs and study
the properties of red sources detected in the submillimeter/millimeter versus
non-detections. In addition, we note that upcoming ALMA Band 7 follow-up
observations (ALMA project #2023.1.00468.S; PI: F. Bauer) of over 80 SCUBA-2
850 $\mu$m-selected DSFGs across three lensing cluster fields with JWST data
(including A2744) will help to pin down the fraction of SCUBA-2 sources that
can be recovered with red priors. Furthermore, these observations will greatly
increase the number of faint DSFGs with ALMA detections while constraining the
cosmic variance between fields. Finally, the combination of the ALMA and
SCUBA-2 data with JWST NIRCam imaging will make it possible to measure stellar
masses and star formation rates from panchromatic SED fits.

##  6 Summary

We used complementary ALMA and SCUBA-2 observations to compare submillimeter
and millimeter galaxy selections in the A2744 cluster field. We showed that
15/17 of the SCUBA-2 $>5\sigma$ sources are detected in the DUALZ ALMA 1.2 mm
mosaic of Fujimoto et al. (2023b), with a low multiplicity of 13%. We also
showed that about 95% of the red galaxy priors identified as $>3\sigma$
SCUBA-2 sources in Barger & Cowie (2023) are detected by ALMA. Six of these
are not detected outright in the DUALZ catalog but are detected at lower
significance ($>2.5\sigma$) in the ALMA image. From the high ALMA detection
fraction, we concluded that red prior selection combined with SCUBA-2 data is
a straightforward and efficient way to select faint DSFGs.

We also analyzed the SCUBA-2 450 $\mu$m and 850$\mu$m fluxes of the DUALZ
sample and found that 74% of the ALMA sources are detected in at least one
submillimeter band: 46/69 are detected at $>2.5\sigma$ at 850 $\mu$m, with an
additional five detected at $>2.5\sigma$ in the 450 $\mu$m data but not at 850
$\mu$m. All of the $>5\sigma$ ALMA sources that are not detected in SCUBA-2
have 1.2 mm fluxes $\lesssim$ 0.6 mJy. Thus, we concluded that the lack of
detections in SCUBA-2 is primarily an issue of depth.

Using the photzs and speczs for the DUALZ sample, we compared the variation in
the measured submillimeter/millimeter fluxes and the SCUBA-2 detection
fraction with redshift. We found that the SCUBA-2 detection fraction of ALMA
sources dropped slightly beyond $z=3$, possibly due to higher millimeter to
submillimeter flux ratios combined with the deeper ALMA flux limits.
Furthermore, we showed that the 1.2 mm to 450 $\mu$m flux ratio is well
correlated with redshift for this sample. Deeper 450 $\mu$m imaging, in
particular, could help constrain the redshifts and dust properties of the full
ALMA sample.

We gratefully acknowledge support for this research from the William F. Vilas
Estate (S.J.M.), a WARF Named Professorship from the University of Wisconsin-
Madison Office of the Vice Chancellor for Research and Graduate Education with
funding from the Wisconsin Alumni Research Foundation (A.J.B.), and NASA grant
80NSSC22K0483 (L.L.C.). The National Radio Astronomy Observatory is a facility
of the National Science Foundation operated under cooperative agreement by
Associated Universities, Inc. This paper makes use of the following ALMA data:
ADS/JAO.ALMA#2013.1.00999.S.
ADS/JAO.ALMA#2015.1.01425.S,
ADS/JAO.ALMA#2017.1.01219.S,
ADS/JAO.ALMA#2018.1.00035.L,
and ADS/JAO.ALMA#2022.1.00073.S.
ALMA is a partnership of ESO (representing its member states), NSF (USA), and
NINS (Japan), together with NRC (Canada), MOST and ASIAA (Taiwan), and KASI
(Republic of Korea), in cooperation with the Republic of Chile. The Joint ALMA
Observatory is operated by ESO, AUI/NRAO, and NAOJ. The James Clerk Maxwell
Telescope is operated by the East Asian Observatory on behalf of The National
Astronomical Observatory of Japan, Academia Sinica Institute of Astronomy and
Astrophysics, the Korea Astronomy and Space Science Institute, the National
Astronomical Observatories of China and the Chinese Academy of Sciences (grant
No. XDB09000000), with additional funding support from the Science and
Technology Facilities Council of the United Kingdom and participating
universities in the United Kingdom and Canada. We wish to recognize and
acknowledge the very significant cultural role and reverence that the summit
of Maunakea has always held within the indigenous Hawaiian community. We are
most fortunate to have the opportunity to conduct observations from this
mountain.

## References

  * Aravena et al. (2016) Aravena, M., Decarli, R., Walter, F., et al. 2016, ApJ, 833, 68
  * Astropy Collaboration et al. (2022) Astropy Collaboration, Price-Whelan, A. M., Lim, P. L., et al. 2022, ApJ, 935, 167
  * Barger & Cowie (2023) Barger, A. J., & Cowie, L. L. 2023, arXiv e-prints, arXiv:2303.07419
  * Barger et al. (2022) Barger, A. J., Cowie, L. L., Blair, A. H., & Jones, L. H. 2022, ApJ, 934, 56
  * Barger et al. (1998) Barger, A. J., Cowie, L. L., Sanders, D. B., et al. 1998, Nature, 394, 248
  * Bezanson et al. (2022) Bezanson, R., Labbe, I., Whitaker, K. E., et al. 2022, arXiv e-prints, arXiv:2212.04026
  * Brammer et al. (2008) Brammer, G. B., van Dokkum, P. G., & Coppi, P. 2008, ApJ, 686, 1503
  * Calvo et al. (2016) Calvo, M., Benoît, A., Catalano, A., et al. 2016, Journal of Low Temperature Physics, 184, 816
  * Casey et al. (2013) Casey, C. M., Chen, C.-C., Cowie, L. L., et al. 2013, MNRAS, 436, 1919
  * Casey et al. (2021) Casey, C. M., Zavala, J. A., Manning, S. M., et al. 2021, ApJ, 923, 215
  * Chen et al. (2013) Chen, C.-C., Cowie, L. L., Barger, A. J., et al. 2013, ApJ, 776, 131
  * Cowie et al. (2023) Cowie, L. L., Barger, A. J., & Bauer, F. E. 2023, ApJ, 952, 28
  * Cowie et al. (2022) Cowie, L. L., Barger, A. J., Bauer, F. E., et al. 2022, ApJ, 939, 5
  * Cowie et al. (2017) Cowie, L. L., Barger, A. J., Hsu, L. Y., et al. 2017, ApJ, 837, 139
  * Cowie et al. (2002) Cowie, L. L., Barger, A. J., & Kneib, J. P. 2002, AJ, 123, 2197
  * Dole et al. (2006) Dole, H., Lagache, G., Puget, J. L., et al. 2006, A&A, 451, 417
  * Dunlop et al. (2017) Dunlop, J. S., McLure, R. J., Biggs, A. D., et al. 2017, MNRAS, 466, 861
  * Eales et al. (1999) Eales, S., Lilly, S., Gear, W., et al. 1999, ApJ, 515, 518
  * Fixsen et al. (1998) Fixsen, D. J., Dwek, E., Mather, J. C., Bennett, C. L., & Shafer, R. A. 1998, ApJ, 508, 123
  * Fujimoto et al. (2023a) Fujimoto, S., Kohno, K., Ouchi, M., et al. 2023a, arXiv e-prints, arXiv:2303.01658
  * Fujimoto et al. (2023b) Fujimoto, S., Bezanson, R., Labbe, I., et al. 2023b, arXiv e-prints, arXiv:2309.07834
  * Furtak et al. (2023) Furtak, L. J., Zitrin, A., Weaver, J. R., et al. 2023, MNRAS, 523, 4568
  * Gómez-Guijarro et al. (2022) Gómez-Guijarro, C., Elbaz, D., Xiao, M., et al. 2022, A&A, 658, A43
  * González-López et al. (2017) González-López, J., Bauer, F. E., Romero-Cañizales, C., et al. 2017, A&A, 597, A41
  * González-López et al. (2020) González-López, J., Novak, M., Decarli, R., et al. 2020, ApJ, 897, 91
  * Hatsukade et al. (2016) Hatsukade, B., Kohno, K., Umehata, H., et al. 2016, PASJ, 68, 36
  * Hatsukade et al. (2018) Hatsukade, B., Kohno, K., Yamaguchi, Y., et al. 2018, PASJ, 70, 105
  * Hauser et al. (1998) Hauser, M. G., Arendt, R. G., Kelsall, T., et al. 1998, ApJ, 508, 25
  * Holland et al. (2013) Holland, W. S., Bintley, D., Chapin, E. L., et al. 2013, MNRAS, 430, 2513
  * Hsu et al. (2016) Hsu, L.-Y., Cowie, L. L., Chen, C.-C., Barger, A. J., & Wang, W.-H. 2016, ApJ, 829, 25
  * Hughes et al. (1998) Hughes, D. H., Serjeant, S., Dunlop, J., et al. 1998, Nature, 394, 241
  * Johnson et al. (2021) Johnson, B. D., Leja, J., Conroy, C., & Speagle, J. S. 2021, ApJS, 254, 22
  * Knudsen et al. (2008) Knudsen, K. K., van der Werf, P. P., & Kneib, J. P. 2008, MNRAS, 384, 1611
  * Lim et al. (2020) Lim, C.-F., Wang, W.-H., Smail, I., et al. 2020, ApJ, 889, 80
  * Lotz et al. (2017) Lotz, J. M., Koekemoer, A., Coe, D., et al. 2017, ApJ, 837, 97
  * McKay et al. (2023, in preparation) McKay, S. J., Barger, A. J., & Cowie, L. L. 2023, in preparation
  * McKay et al. (2023) McKay, S. J., Barger, A. J., Cowie, L. L., Bauer, F. E., & Rosenthal, M. J. N. 2023, ApJ, 951, 48
  * Monfardini et al. (2014) Monfardini, A., Adam, R., Adane, A., et al. 2014, Journal of Low Temperature Physics, 176, 787
  * Muñoz Arancibia et al. (2018) Muñoz Arancibia, A. M., González-López, J., Ibar, E., et al. 2018, A&A, 620, A125
  * Muñoz Arancibia et al. (2023) —. 2023, A&A, 675, A85
  * Paris et al. (2023) Paris, D., Merlin, E., Fontana, A., et al. 2023, ApJ, 952, 20
  * Puget et al. (1996) Puget, J. L., Abergel, A., Bernard, J. P., et al. 1996, A&A, 308, L5
  * Roberts-Borsani et al. (2023) Roberts-Borsani, G., Treu, T., Chen, W., et al. 2023, Nature, 618, 480
  * Smail et al. (1997) Smail, I., Ivison, R. J., & Blain, A. W. 1997, ApJ, 490, L5
  * Treu et al. (2022) Treu, T., Roberts-Borsani, G., Bradac, M., et al. 2022, ApJ, 935, 110
  * Umehata et al. (2017) Umehata, H., Tamura, Y., Kohno, K., et al. 2017, ApJ, 835, 98
  * Walter et al. (2016) Walter, F., Decarli, R., Aravena, M., et al. 2016, ApJ, 833, 67
  * Wang et al. (2023) Wang, B., Leja, J., Labbé, I., et al. 2023, arXiv e-prints, arXiv:2310.01276
  * Wang et al. (2019) Wang, T., Schreiber, C., Elbaz, D., et al. 2019, Nature, 572, 211
  * Weaver et al. (2023) Weaver, J. R., Cutler, S. E., Pan, R., et al. 2023, arXiv e-prints, arXiv:2301.02671
  * Wilson et al. (2020) Wilson, G. W., Abi-Saad, S., Ade, P., et al. 2020, in Society of Photo-Optical Instrumentation Engineers (SPIE) Conference Series, Vol. 11453, Millimeter, Submillimeter, and Far-Infrared Detectors and Instrumentation for Astronomy X, ed. J. Zmuidzinas & J.-R. Gao, 1145302
  * Zavala et al. (2018) Zavala, J. A., Aretxaga, I., Dunlop, J. S., et al. 2018, MNRAS, 475, 5585
  * Zavala et al. (2021) Zavala, J. A., Casey, C. M., Manning, S. M., et al. 2021, ApJ, 909, 165

Table 1:  SCUBA-2 450 $\mu$m and 850 $\mu$m Properties of DUALZ Catalog

| ALMA | SCUBA-2 | SCUBA-2 | ALMA
---|---|---|---|---
No. | R.A. | Decl. | $f_{450\,\mu{\rm m}}$ | Error | S/N | $f_{850\,\mu{\rm m}}$ | Error | S/N | $f_{\rm 1.2\,mm}$ | Error | S/N
| J2000.0 | J2000.0 | (mJy) |  | (mJy) |  | (mJy) |
1 | 3.6036340 | -30.429562 | 3.65 | 3.82 | 1.0 | 0.89 | 0.35 | 2.5 | 0.27 | 0.05 | 5.3
2 | 3.5755829 | -30.424378 | 11.1 | 3.39 | 3.3 | 6.62 | 0.30 | 22.2 | 3.02 | 0.12 | 41.4
3 | 3.5863515 | -30.425044 | 7.52 | 3.46 | 2.2 | 2.33 | 0.31 | 7.6 | 0.80 | 0.05 | 15.0
4 | 3.6056155 | -30.418061 | 10.0 | 3.49 | 2.9 | 0.49 | 0.31 | 1.6 | 0.28 | 0.05 | 5.3
5 | 3.5760482 | -30.413200 | 17.5 | 3.00 | 5.8 | 6.96 | 0.27 | 25.9 | 3.04 | 0.12 | 45.2
6 | 3.6085748 | -30.414591 | 11.4 | 3.46 | 3.3 | 0.96 | 0.31 | 3.1 | 0.55 | 0.05 | 10.5
7 | 3.5650518 | -30.412200 | 5.11 | 3.07 | 1.7 | -0.14 | 0.27 | -0.5 | 0.28 | 0.05 | 6.1
8 | 3.5690146 | -30.402798 | 5.36 | 2.93 | 1.8 | 0.68 | 0.26 | 2.6 | 0.29 | 0.05 | 6.3
9 | 3.5723726 | -30.395954 | 4.63 | 2.93 | 1.6 | 0.56 | 0.26 | 2.1 | 0.40 | 0.05 | 7.5
10 | 3.6276995 | -30.394278 | 13.5 | 3.92 | 3.4 | 2.13 | 0.37 | 5.8 | 1.06 | 0.08 | 21.5
11 | 3.6005232 | -30.396158 | 3.34 | 2.97 | 1.1 | 0.89 | 0.28 | 3.2 | 0.18 | 0.03 | 5.2
12 | 3.6175369 | -30.395013 | 5.63 | 3.45 | 1.6 | 0.59 | 0.32 | 1.8 | 0.28 | 0.06 | 6.0
13 | 3.5667961 | -30.394888 | 7.16 | 3.00 | 2.4 | 1.80 | 0.27 | 6.8 | 0.70 | 0.05 | 15.3
14 | 3.6326635 | -30.393642 | 7.47 | 4.08 | 1.8 | 0.67 | 0.38 | 1.8 | 0.61 | 0.06 | 14.0
15 | 3.6214798 | -30.393109 | 5.36 | 3.57 | 1.5 | 1.44 | 0.34 | 4.3 | 0.56 | 0.06 | 13.2
16 | 3.5809797 | -30.390749 | 3.08 | 2.87 | 1.1 | 1.68 | 0.27 | 6.3 | 0.29 | 0.05 | 5.5
17 | 3.5474111 | -30.388289 | 17.1 | 3.64 | 4.7 | 5.16 | 0.31 | 16.8 | 2.82 | 0.07 | 55.2
18 | 3.5327226 | -30.387320 | 6.20 | 4.38 | 1.4 | 0.08 | 0.36 | 0.2 | 0.37 | 0.06 | 6.6
19 | 3.5719637 | -30.383018 | 12.1 | 3.05 | 4.0 | 4.40 | 0.28 | 15.7 | 0.68 | 0.05 | 13.5
20 | 3.5190588 | -30.386326 | 9.36 | 5.10 | 1.8 | 3.99 | 0.41 | 9.6 | 1.50 | 0.20 | 8.9
21 | 3.5825067 | -30.385468 | 5.85 | 2.92 | 2.0 | 2.58 | 0.28 | 9.3 | 1.49 | 0.06 | 42.0
22 | 3.5850010 | -30.381794 | 11.1 | 2.98 | 3.7 | 3.82 | 0.29 | 13.2 | 1.71 | 0.12 | 24.5
23 | 3.5732507 | -30.383497 | 22.1 | 3.01 | 7.3 | 0.05 | 0.28 | 0.2 | 0.91 | 0.06 | 24.6
24 | 3.6357675 | -30.382387 | 1.82 | 3.98 | 0.5 | 0.36 | 0.43 | 0.8 | 0.60 | 0.06 | 11.3
25 | 3.5126766 | -30.380966 | 10.4 | 5.27 | 2.0 | 2.06 | 0.42 | 4.9 | 0.71 | 0.15 | 6.9
26 | 3.5920643 | -30.380487 | 5.85 | 3.02 | 1.9 | 0.34 | 0.30 | 1.1 | 0.53 | 0.06 | 11.5
27 | 3.5812925 | -30.380250 | 0.60 | 3.04 | 0.2 | 0.47 | 0.29 | 1.6 | 0.33 | 0.05 | 9.1
28 | 3.5797072 | -30.378413 | 7.94 | 3.04 | 2.6 | 3.08 | 0.30 | 10.4 | 0.94 | 0.07 | 21.5
29 | 3.5599876 | -30.377804 | 3.40 | 3.35 | 1.0 | 0.58 | 0.30 | 1.9 | 0.29 | 0.06 | 5.9
30 | 3.5579207 | -30.377236 | 8.23 | 3.43 | 2.4 | 2.52 | 0.31 | 8.1 | 0.38 | 0.06 | 6.7
31 | 3.5823806 | -30.377170 | 8.65 | 3.08 | 2.8 | 1.14 | 0.30 | 3.8 | 0.30 | 0.06 | 5.8
32 | 3.5995954 | -30.374707 | 4.95 | 3.22 | 1.5 | 1.11 | 0.33 | 3.4 | 0.38 | 0.06 | 7.4
33 | 3.5103148 | -30.375438 | 6.91 | 5.18 | 1.3 | 1.57 | 0.42 | 3.7 | 0.36 | 0.07 | 6.1
34 | 3.5543533 | -30.371955 | 13.2 | 3.60 | 3.7 | 2.13 | 0.33 | 6.4 | 0.64 | 0.06 | 13.1
35 | 3.6172782 | -30.368795 | 9.80 | 3.58 | 2.7 | 4.78 | 0.38 | 12.5 | 1.97 | 0.10 | 35.6
36 | 3.5237288 | -30.371469 | 10.3 | 4.53 | 2.3 | 1.85 | 0.38 | 4.8 | 0.33 | 0.06 | 8.7
37 | 3.5638035 | -30.367614 | 9.99 | 3.54 | 2.8 | 1.70 | 0.34 | 4.9 | 0.26 | 0.05 | 5.4
38 | 3.5375675 | -30.365649 | 14.0 | 4.06 | 3.4 | 0.76 | 0.37 | 2.0 | 0.34 | 0.06 | 6.9
39 | 3.5311608 | -30.361294 | 7.09 | 4.14 | 1.7 | 3.35 | 0.38 | 8.8 | 0.50 | 0.06 | 9.2
40 | 3.6006013 | -30.362714 | 1.04 | 3.70 | 0.3 | 1.28 | 0.39 | 3.3 | 0.75 | 0.06 | 16.0
41 | 3.5391400 | -30.360300 | 17.4 | 3.97 | 4.4 | 2.59 | 0.38 | 6.8 | 0.60 | 0.06 | 10.6
42 | 3.5362850 | -30.360378 | 32.1 | 4.02 | 8.0 | 7.69 | 0.38 | 20.2 | 2.68 | 0.09 | 50.4
43 | 3.5211455 | -30.360680 | 9.95 | 4.40 | 2.3 | 2.06 | 0.40 | 5.2 | 0.55 | 0.06 | 11.1
44 | 3.5990300 | -30.359757 | 10.6 | 3.78 | 2.8 | 3.55 | 0.40 | 8.9 | 0.68 | 0.06 | 12.2
45 | 3.5938354 | -30.356617 | 23.0 | 3.98 | 5.8 | 4.70 | 0.42 | 11.2 | 1.54 | 0.12 | 24.7
46 | 3.5237434 | -30.359221 | -0.51 | 4.33 | -0.1 | 0.65 | 0.39 | 1.7 | 0.29 | 0.06 | 5.9
47 | 3.5926433 | -30.356140 | 2.93 | 4.05 | 0.7 | 1.08 | 0.43 | 2.5 | 0.44 | 0.05 | 8.3
48 | 3.5583154 | -30.354969 | 4.86 | 3.92 | 1.2 | 2.13 | 0.42 | 5.1 | 1.22 | 0.09 | 23.8
49 | 3.5463709 | -30.353476 | 9.03 | 3.92 | 2.3 | 1.06 | 0.40 | 2.6 | 0.28 | 0.05 | 6.8
50 | 3.5491494 | -30.352245 | 15.8 | 3.93 | 4.0 | 5.71 | 0.41 | 13.9 | 1.87 | 0.07 | 39.3
51 | 3.5305552 | -30.352317 | 9.51 | 4.10 | 2.3 | 1.49 | 0.40 | 3.8 | 0.32 | 0.06 | 7.6
52 | 3.5612866 | -30.326963 | 8.22 | 4.27 | 1.9 | 0.93 | 0.46 | 2.0 | 0.22 | 0.04 | 5.4
53 | 3.5555550 | -30.335627 | 8.67 | 4.15 | 2.1 | 0.34 | 0.45 | 0.8 | 0.26 | 0.05 | 6.3
54 | 3.5528434 | -30.344173 | 1.05 | 4.07 | 0.3 | 0.94 | 0.44 | 2.1 | 0.44 | 0.05 | 8.8
55 | 3.5641531 | -30.344466 | 12.5 | 4.21 | 3.0 | 2.31 | 0.47 | 4.9 | 0.97 | 0.09 | 17.1
56 | 3.5611003 | -30.330043 | 5.33 | 4.22 | 1.3 | 1.80 | 0.46 | 3.9 | 0.42 | 0.05 | 8.8
57 | 3.5566688 | -30.384299 | 10.3 | 3.31 | 3.1 | 0.66 | 0.29 | 2.3 | 0.26 | 0.06 | 4.5
58 | 3.6384449 | -30.381213 | 6.38 | 4.07 | 1.6 | 1.51 | 0.45 | 3.4 | 0.31 | 0.07 | 4.6
59 | 3.5582541 | -30.374437 | 5.58 | 3.47 | 1.6 | 0.7 | 0.32 | 2.2 | 0.26 | 0.06 | 4.7
60 | 3.5923715 | -30.368396 | 4.90 | 3.42 | 1.4 | 0.91 | 0.35 | 2.6 | 0.26 | 0.06 | 4.6
61 | 3.5580404 | -30.331104 | 10.6 | 4.20 | 2.5 | 0.16 | 0.46 | 0.3 | 0.26 | 0.05 | 4.9
62 | 3.5456886 | -30.337678 | 8.42 | 4.1 | 2.1 | 0.66 | 0.44 | 1.5 | 0.24 | 0.05 | 4.5
63 | 3.5671705 | -30.341419 | 5.84 | 4.32 | 1.4 | 0.07 | 0.48 | 0.1 | 0.24 | 0.05 | 4.4
64 | 3.5618008 | -30.330958 | 15.1 | 4.22 | 3.6 | -0.06 | 0.46 | -0.1 | 0.29 | 0.05 | 5.4
65 | 3.5685704 | -30.329642 | 3.05 | 4.32 | 0.7 | -0.13 | 0.46 | -0.3 | 1.14 | 0.24 | 4.7
66 | 3.5938125 | -30.408263 | 1.16 | 3.02 | 0.4 | 0.47 | 0.27 | 1.7 | 0.15 | 0.05 | 4.9
67 | 3.6017503 | -30.407851 | 5.60 | 3.17 | 1.8 | 0.29 | 0.29 | 1.0 | 0.16 | 0.04 | 5.2
68 | 3.5932584 | -30.384375 | 2.74 | 2.95 | 0.9 | 0.49 | 0.29 | 1.7 | 0.17 | 0.05 | 5.0
69 | 3.5902747 | -30.400439 | 6.64 | 2.89 | 2.3 | 0.76 | 0.27 | 2.9 | 0.18 | 0.04 | 4.2

Note. — Fluxes have not been corrected for magnification.


Westner

# Cycling on the Freeway: The Perilous State of Open Source Neuroscience
Software

Britta U. Westner  Department of Cognitive Neuroscience, Donders Institute for
Brain, Cognition and Behaviour, Radboud University Medical Center, Nijmegen,
The Netherlands  Donders Centre for Cognition, Donders Institute for Brain,
Cognition and Behaviour, Radboud University, Nijmegen, The Netherlands  Daniel
R. McCloy  Institute for Learning and Brain Sciences, University of
Washington, Seattle, WA, United States  Eric Larson  Institute for Learning
and Brain Sciences, University of Washington, Seattle, WA, United States
Alexandre Gramfort  Université Paris-Saclay, Inria, CEA, Palaiseau, France
Daniel S. Katz  NCSA & CS & ECE & iSchool, University of Illinois Urbana-
Champaign, Illinois, United States  Arfon M. Smith  GitHub Inc.  invited co-
signees

###### Abstract

Most scientists need software to perform their research (Barker et al., 2020;
Carver et al., 2022; Hettrick, 2014; Hettrick et al., 2014; Switters and
Osimo, 2019), and neuroscientists are no exception. Whether we work with
reaction times, electrophysiological signals, or magnetic resonance imaging
data, we rely on software to acquire, analyze, and statistically evaluate the
raw data we obtain — or to generate such data if we work with simulations. In
recent years there has been a shift toward relying on free, open-source
scientific software (FOSSS) for neuroscience data analysis (Poldrack et al.,
2019), in line with the broader open science movement in academia (McKiernan
et al., 2016) and wider industry trends (Eghbal, 2016). Importantly, FOSSS is
typically developed by working scientists (not professional software
developers) which sets up a precarious situation given the nature of the
typical academic workplace (wherein academics, especially in their early
careers, are on short and fixed term contracts). In this paper, we will argue
that the existing ecosystem of neuroscientific open source software is
brittle, and discuss why and how the neuroscience community needs to come
together to ensure a healthy growth of our software landscape to the benefit
of all.

###### keywords:

open science | open source | neuroscience | electrophysiology

britta.westner[at]donders.ru.nl

### The neuroscience open source system is moving towards a crisis.

The development and especially maintenance of open source scientific software
is labor-intensive, especially when adopting best practices of open science:
readability, resilience, and re-use (Connolly et al., 2023). Critically, the
survival of any open source software is guaranteed not only by the total
number of contributors, but by the number of developers who have a
sufficiently complete overview of the source code and the tools and processes
in place for maintenance of the project. The _Truck Factor_ (or _Bus Factor_)
quantifies the number of such maintainers for a given project, and can be seen
as an indicator of the risk of incomplete knowledge- and capability-sharing
among the project’s team members (Avelino et al., 2016). Table 1 shows the
Truck Factors of several widely used analysis software projects in
electrophysiological neuroscience. It is evident that projects typically have
a Truck Factor of one to three, meaning that only one to three people per
project have sufficient knowledge to keep the project alive (note that the
Truck Factor does not take into account whether those contributors with the
relevant knowledge are still active or have funding/support to do the
maintenance work). This reveals the fragility of the academic open source
system: _all users_ of a software package rely on the work of _one to three
people_ to keep everyone’s data analyses from failing. It also highlights the
enormous pressure the maintainers of open source software are under. Given the
importance of scientific software to the practice of modern science, one would
hope developing FOSSS to be a well-supported and incentivized role in
academia. This hope is not borne out: the academic incentive structure for
software development and maintenance is woefully inadequate (Carver et al.,
2022; Davenport et al., 2020; Merow et al., 2023; Smith et al., 2018) and can
even be perceived as hostile (Millman and Pérez, 2018; Pérez, 2011). As
Davenport et al. (2020) state: “In spite of the vital role research software
plays, it largely remains undervalued, with time spent in training or
development seen as detracting from the ‘real research’.” The current
incentive model often summarized as “publish-or-perish” has a significant
negative impact on the software ecosystem’s health in several ways, which we
will describe below.

### Rewarding publications in promotion disfavors software development.

Many scientists who develop software publish a paper to introduce their
project to the community, and direct users to cite that paper if they use the
software for their analysis (Katz and Chue Hong, 2018; Smith et al., 2016).
However, software papers are usually published to mark the first public
release or sometimes a later major release of a software package. Given that
publication counts are such an important (and often the primary) metric for
academic tenure and promotion, once the paper announcing the software is
published, there is little incentive from the employer to continue developing
the software. This may be changing as academic institutions rely more on
composite citation metrics rather than simple publication counts — by
incentivizing that the paper be widely cited — but the reward of new projects
and attendant new publications can easily outweigh the reward of garnering
more software-paper citations, especially given that software-papers are
viewed as less valuable than research papers (Davenport et al., 2020). At the
same time, people who contribute to development or maintenance of FOSSS at a
later stage do not get rewarded by citations of the initial paper (Davenport
et al., 2020), thus increasing the chance that the software will become
unmaintained. We worry that the lack of benefit for project “latecomers” also
often incentivizes starting new projects (instead of contributing to existing
ones) — a new project means a new publication! — which in turn risks further
increasing the number of unmaintained software packages in the field. To spell
it out: open source work done by academics (in time they might otherwise use
to do research) sustains _other_ academics writing _their_ papers (Merow et
al., 2023) — which puts the software-developing academics behind in the
publish-or-perish culture of academia.

### Lack of stable long-term funding for scientific software stifles its
growth.

Funding is hard to obtain for the development of new software — and even
harder for the maintenance of existing software (Davenport et al., 2020). That
means that many contributors and maintainers either do not have secure long-
term academic positions or are not paid primarily for their software work.
This is slowly changing: programs like POSE from the US National Science
Foundation, open science supplements to grants from the US National Institutes
of Health, the UK Research and Innovation office, the national plan for open
science (“plan national pour la science ouverte”) of the French government,
and software-focused grants from organizations like CZI, the Sloan Foundation,
and the Simons Foundation are all helping to support the development of FOSSS.
But the (mostly) short-term grant-based nature of these new support mechanisms
means that once the grant is over the primary incentive to continue
maintaining the software is largely gone, a void quickly filled by the ever-
present publish-or-perish incentive lurking in the background.

### Existing barriers skew the developer demographic to the detriment of
science.

Research shows that diversity has positive effects on project outcomes
generally (Earley and Mosakowski, 2000; Hoogendoorn et al., 2013; Jackson and
Joshi, 2004; Roberson, 2019) and for open source projects in particular
(Daniel et al., 2013; Vasilescu et al., 2015). However, structural factors
such as misogyny and racism hinder diversity of the developer pool: only
1.1-5.4% of open source developers are perceptible as or identify as women
(Eghbal, 2016; Geiger, 2022; Ghosh et al., 2002; Nafus, 2012) and less than
17% are perceptible as Non-White (Nadri et al., 2021). The cost of
contributing is higher for minorities (Whitaker and Guest, 2020): not only do
they often feel a pressure to have to be perfect (Singh and Bongiovanni,
2021), underrepresented groups have to face stereotyping, discrimination, and
harassment in the open source software world (Frluckaj et al., 2022; Nadri et
al., 2021; Nafus, 2012; Singh and Bongiovanni, 2021; Vasilescu et al., 2015).

We are unaware of any systematic studies examining diversity in neuroscience
software, but our collective experience in the field suggests that it is not
substantially different from the broader open-source software community. If
true, this is unsurprising: academics need a certain level of privilege to be
able to contribute to open source in the first place. Typical barriers to
participation in FOSSS projects are the lack of permission and support from a
supervisor, the lack of “free” time outside normal work hours due to family or
financial demands on that time, and lack of confidence due to inadequate
training, role models, and guidance. These barriers all affect marginalized
groups more strongly, and are compounded by the lack of representation and
role models from underrepresented demographics and by the myth of meritocracy
(Nafus, 2012). Moreover, the “informalization” of open source (Nafus, 2012)
compounds the problem by eschewing traditional application and advancement
processes and legal workplace protections in the spaces and interactions where
FOSSS work is carried out, and consequently such spaces are often dominated by
“old boys” networks where again underrepresented identities face an uphill
battle. Moreover, the publish-or-perish culture and lack of stable funding for
software development in academia also contribute to the substantial lack of
qualified labor in FOSSS communities (cf. Table 1). This further impacts the
diversity of FOSSS communities with respect to career stage (an imperfect
proxy for age) — many academic open-source contributors are pursuing or have
just finished their PhD and the devaluing of software work makes it harder for
them to achieve tenured positions, while more senior academics who contributed
early in their career are often not able to prioritize open source work
anymore.

In summary, workplace incentives in academia disfavor participation in open
source communities; the lack of a level playing field further discourages
participation for certain classes of academics; and changing the incentives
through extramural funding has so far been a temporary, partial fix on too
small a scale to be transformative. But given the shift toward relying on
FOSSS in neuroscience, it seems clear that the incentive structure must
change. In the following, we will discuss ways in which such change can be
realized.

Software | Truck Factor | Hash
---|---|---
Brainstorm | 1 | 053b2ea6bb46ca633b931d9467f292bd7eae1361
EEGLAB | 1 | cc87c2e08716d397da76821f0fd651bbb9c4357c
FieldTrip | 1 | f41a2f7aa84d2fbebbf17369e155855b4358c3b8
MNE-Python | 3 | 9627f43bb2c03233827046bc06acdab0968d6610
SPM | 3 | fba4f8e139c3f975b61e2581828d151b26dbe68a
Astropy | 6 | 53188c39a23c33b72df5850ec59e31886f84e29d
Table 1: The reported Truck Factors were estimated using the heuristic-based
approach reported in Avelino et al. (2016). This algorithm estimated the Truck
Factor using a degree-of-authorship metric (Fritz et al., 2010): “authors” of
files are those developers who are able to maintain a file moving forward.
Developers who have (joint) authorship of at least 50% of files count towards
the Truck factor. For each package, the Truck Factor analysis was performed 29
August 2023; the commit hash for the state of the codebase when we performed
the analysis is shown. We compare neuroscience packages to Astropy (The
Astropy Collaboration et al., 2013), a well-supported and widely used software
package in astronomy and astrophysics.

### Professionalize academic software development.

One obvious cornerstone for a more sustainable and professional FOSSS
ecosystem is funding, especially funding that supports maintenance of
existing, widely-used projects (Merow et al., 2023). That funding should
support scientists who engage in software development as part of their normal
academic appointment duties. It has further been suggested that all levels of
software development in academia (from widely-used packages down to single-
user analysis scripts) can be carried out or supported by experienced research
software engineers (Connolly et al., 2023; Merow et al., 2023). As others have
noted, there are at least two obstacles to academic software work being
carried out by professional software engineers: first, academia struggles to
attract technically skilled professionals given the (much) higher compensation
available in industry (Connolly et al., 2023; Gewin, 2022; Seidl et al.,
2016), and second, there is a perception among professional engineers that
career advancement prospects in academia are limited (Carver et al., 2022;
Connolly et al., 2023; Merow et al., 2023). However, even if institutional
support for research software engineering were to increase dramatically, we
see a further obstacle to the sustainability of FOSSS: academic software
development needs the participation of scientists that are active in the field
and have the domain knowledge necessary to understand relevant use cases, best
practices, and common analysis pitfalls. This latter point also underscores
the importance of having more senior academics involved in software
development, which (as discussed above) is disincentivized and consequently
rare. Thus we believe that a robust solution must involve institutions
increasing their support of _working scientists_ who develop software (perhaps
alongside professional research software engineers), and recognizing and
rewarding software contributions as an integral part of the practice of modern
science.

### Adopting new software citation practices can increase the valuation of
FOSSS.

Increasing the valuation of open source work for academic career advancement
is crucial. One suggested shortcut to this would be the exploitation of the
existing publish-or-perish culture by introducing “update publications” after
software development milestones (Merow et al., 2023). However, in our view
this risks perpetually re-creating the same set of problems, just on a shorter
time-scale. Encouraging researchers to cite the software they use — and to
cite the _software itself_ (i.e., the specific version used), not the
canonical paper that introduced the software — seems like a better adaptation
within the existing culture: it does not require the developers to write a new
paper, and all contributors to a specific version of the software get credit.
Indeed, “cite the software, not the paper” is considered best practice already
(Katz et al., 2020; Smith et al., 2016), and we hope that this can become
standard practice in neuroscience; publishers, journals, editors, and
reviewers must play a role here in insisting that authors attribute their
software usage in line with those best practices.

### Start viewing research software work as an important contribution to
science.

We urge funding bodies and promotion committees to value substantial open
source software work in their evaluation guidelines as an important
contribution to science of equal value as one or (ideally) multiple papers.
The development of widely used analysis software should be acknowledged and
rewarded as a contribution to science rather than viewed as merely the
development of one’s coding skills. Individual investigators can play an
important role here too: if your lab relies on FOSSS for data analysis,
consider allocating grant funds to support a scientific software developer,
making FOSSS contribution part of the job description for your next postdoc or
research scientist hire, or planning that your graduate students will need
either a longer duration of support or fewer research output expectations (or
both) in order to develop the necessary competencies _to both use and
contribute to_ the tools they rely on. Importantly, it should not be the
developer’s responsibility to prove the value of their contributions; the
burden should lie with promotion and tenure committees to become familiar with
how to evaluate the scope and import of software contributions (fortunately,
the Research Software Alliance and Research Data Alliance are working on
policy recommendations on this topic,111https://www.rd-
alliance.org/groups/policies-research-organisations-research-software-pro4rs
and OSPO++ is working to create, support and improve open-source program
offices across academia222https://ospoplusplus.org/). Failing that, promotion
and tenure committees should at minimum allow, request, and encourage
scientists who develop software to contextualize the scope and impact of their
software work (Hafer and Kirkpatrick, 2009). We however want to caution
against the development of new and too simplistic metrics for the measure of
open source work in academia, as measuring software impact is difficult and
takes time (Afiaz et al., 2023). Furthermore, Goodhart’s law postulates that
every measure which becomes a target becomes a bad measure (Afiaz et al.,
2023; Goodhart, 1984), as has happened to the _h-index_ (e.g., Bartneck and
Kokkelmans, 2011; Purvis, 2006; Seeber et al., 2019; Zhivotovsky and
Krutovsky, 2008).

### Better training can facilitate a healthy FOSSS ecosystem in the long-term.

Lastly, we advocate for programs and departments to incorporate better
training of students and PhD candidates in software development and
maintenance (Carver et al., 2022; Millman and Pérez, 2018; Guest and Forbes,
2023): this would not only increase the software engineering skills of
academics generally (Connolly et al., 2023), but would also send a strong
signal about the importance of well-written, reusable code to junior
researchers. While training programs such as e.g., Software
Carpentry333https://software-carpentry.org/,
CodeRefinery444https://coderefinery.org/, INTERSECT555https://intersect-
training.github.io, Neurohackademy666https://neurohackademy.org or Neuromatch
Academy777https://academy.neuromatch.io give excellent training courses and
summer schools, we also call on teaching coordinators of universities to
reflect an increasing need for programming literacy in their neuroscience
programs. Together with the recognition for writing open source software, this
would be an important step towards a healthy software ecosystem in
neuroscience.

### A higher valuation of FOSSS will increase the inclusivity of open source
spaces.

We predict that all action points discussed above will have a positive impact
on the inclusivity of open source in academia. Changing the incentive
structure such that open source work does not have to be a privilege anymore
but gets seen as what it is: a valid and critical contribution to science,
will facilitate the participation of underrepresented groups. Beyond this, we
strongly advocate that open source projects in neuroscience make sure to be
welcoming to everyone and to prevent any harassment, e.g. by stating and
adhering to Community Guidelines.

### Conclusion

Summarizing the key points of this paper, we hope to raise awareness within
the neuroscientific community about its dependence on a rather brittle
structure. Your open source software ecosystem needs your help! Immediate
action can be taken by citing current software versions instead of the seminal
software-describing paper, by making space for your trainees to engage in
FOSSS communities, and rewarding them (or at least not penalizing them) when
they do. Beyond this, the incentive structure in academia and the policies
that support it (including those created by research performing organizations,
funders, publishers, etc.) urgently need to be re-thought (Hostler, 2023;
Jensen and Katz, 2023; Merow et al., 2023; Millman and Pérez, 2018; Munafò et
al., 2017; Neylon et al., 2012) — not only for the sake of the academic open
source ecosystem, but for the good of the neuroscience community as a whole.

###### Acknowledgements.

This project has been made possible in part by grant number 2021-237679 from
the Chan Zuckerberg Initiative DAF, an advised fund of Silicon Valley
Community Foundation.

## References

## References

  * Afiaz et al. (2023) Afiaz A, Ivanov AA, Chamberlin J, Hanauer D, Savonen CL, Morgan M, Reich M, Getka A, Holmes A, Pati S, Boutros PC, Bakas S, Caporaso JG, Fiol GD, Haas B, Schloss PD, Eddy JA, Albrecht J, Fedorov A, Hoffman AM, Bradshaw RL, Leek JT, Wright C (2023)  Evaluation of Software Impact Designed for Biomedical Research: Are We Measuring What’s Meaningful?  arXiv .
  * Avelino et al. (2016) Avelino G, Passos L, Hora A, Valente MT (2016)  A Novel Approach for Estimating Truck Factors  In 2016 IEEE 24th International Conference on Program Comprehension (ICPC), pp. 1–10.
  * Barker et al. (2020) Barker M, Katz DS, Gonzalez-Beltran A (2020)  Evidence for the importance of research software.
  * Bartneck and Kokkelmans (2011) Bartneck C, Kokkelmans S (2011)  Detecting h-index manipulation through self-citation analysis.  Scientometrics 87:85–98.
  * Carver et al. (2022) Carver JC, Weber N, Ram K, Gesing S, Katz DS (2022)  A survey of the state of the practice for research software in the United States.  PeerJ Computer Science 8:e963.
  * Connolly et al. (2023) Connolly A, Hellerstein J, Alterman N, Beck D, Fatland R, Lazowska E, Mandava V, Stone S (2023)  Software Engineering Practices in Academia: Promoting the 3Rs—Readability, Resilience, and Reuse.  Harvard Data Science Review 5\.
  * Daniel et al. (2013) Daniel S, Agarwal R, Stewart KJ (2013)  The Effects of Diversity in Global, Distributed Collectives: A Study of Open Source Project Success.  Information Systems Research 24:312–333.
  * Davenport et al. (2020) Davenport JH, Grant J, Jones CM (2020)  Data Without Software Are Just Numbers.  Data Science Journal 19:3.
  * Earley and Mosakowski (2000) Earley CP, Mosakowski E (2000)  Creating Hybrid Team Cultures: An Empirical Test of Transnational Team Functioning.  Academy of Management Journal 43:26–49.
  * Eghbal (2016) Eghbal N (2016)  The Unseen Labor Behind Our Digital Infrastructure  Technical report.
  * Fritz et al. (2010) Fritz T, Ou J, Murphy GC, Murphy-Hill E (2010)  A degree-of-knowledge model to capture source code familiarity  In Proceedings of the 32nd ACM/IEEE International Conference on Software Engineering - Volume 1, pp. 385–394, Cape Town South Africa. ACM.
  * Frluckaj et al. (2022) Frluckaj H, Dabbish L, Widder DG, Qiu HS, Herbsleb JD (2022)  Gender and Participation in Open Source Software Development.  Proceedings of the ACM on Human-Computer Interaction 6:1–31.
  * Geiger (2022) Geiger RS (2022)  Summary Analysis of the 2017 GitHub Open Source Survey .
  * Gewin (2022) Gewin V (2022)  Has the ’great resignation’ hit academia?  Nature pp. 211–213.
  * Ghosh et al. (2002) Ghosh RA, Glott R, Krieger B, Robles G (2002)  FLOSS Final report.  Technical report.
  * Goodhart (1984) Goodhart CA (1984)  Problems of Monetary Management: The UK Experience Springer.
  * Guest and Forbes (2023) Guest O, Forbes SH (2023)  Teaching coding inclusively: If this, then what?  Preprint, SocArXiv.
  * Hafer and Kirkpatrick (2009) Hafer L, Kirkpatrick AE (2009)  Assessing open source software as a scholarly contribution.  Communications of the ACM 52:126–129.
  * Hettrick (2014) Hettrick S (2014)  It’s impossible to conduct research without software, say 7 out of 10 UK researchers.  Software Sustainability Institute. Retrieved 20.06.2023 20:2016.
  * Hettrick et al. (2014) Hettrick S, Antonioletti M, Carr L, Hong NC, Crouch S, Roure DCD, Emsley I, Goble C, Hay A, Inupakutika D, Jackson M, Nenadic A, Parkinson T, Parsons MI, Pawlik A, Peru G, Proeme A, Robinson J, Sufi S (2014)  Uk research software survey 2014.
  * Hoogendoorn et al. (2013) Hoogendoorn S, Oosterbeek H, Van Praag M (2013)  The Impact of Gender Diversity on the Performance of Business Teams: Evidence from a Field Experiment.  Management Science 59:1514–1528.
  * Hostler (2023) Hostler TJ (2023)  The Invisible Workload of Open Research.  Journal of Trial and Error .
  * Jackson and Joshi (2004) Jackson SE, Joshi A (2004)  Diversity in social context: A multi-attribute, multilevel analysis of team diversity and sales performance.  Journal of Organizational Behavior 25:675–702.
  * Jensen and Katz (2023) Jensen EA, Katz DS (2023)  Charting the course: Policy and planning for sustainable research software.  Technical report.
  * Katz and Chue Hong (2018) Katz DS, Chue Hong NP (2018)  Software Citation in Theory and Practice  In Davenport JH, Kauers M, Labahn G, Urban J, editors, Mathematical Software – ICMS 2018, Vol. 10931, pp. 289–296. Springer International Publishing, Cham.
  * Katz et al. (2020) Katz DS, Hong NPC, Clark T, Muench A, Stall S, Bouquin D, Cannon M, Edmunds S, Faez T, Feeney P, Fenner M, Friedman M, Grenier G, Harrison M, Heber J, Leary A, MacCallum C, Murray H, Pastrana E, Perry K, Schuster D, Stockhause M, Yeston J (2020)  Recognizing the value of software: A software citation guide.  F1000Research 9\.
  * McKiernan et al. (2016) McKiernan EC, Bourne PE, Brown CT, Buck S, Kenall A, Lin J, McDougall D, Nosek BA, Ram K, Soderberg CK, Spies JR, Thaney K, Updegrove A, Woo KH, Yarkoni T (2016)  How open science helps researchers succeed.  eLife 5:e16800.
  * Merow et al. (2023) Merow C, Boyle B, Enquist BJ, Feng X, Kass JM, Maitner BS, McGill B, Owens H, Park DS, Paz A, Pinilla-Buitrago GE, Urban MC, Varela S, Wilson AM (2023)  Better incentives are needed to reward academic software development.  Nature Ecology & Evolution 7:626–627.
  * Millman and Pérez (2018) Millman K, Pérez F (2018)  Developing Open-Source Scientific Practice *  In Stodden V, Leisch F, Peng RD, editors, Implementing Reproducible Research, pp. 149–183. Chapman and Hall/CRC, 1 edition.
  * Munafò et al. (2017) Munafò MR, Nosek BA, Bishop DVM, Button KS, Chambers CD, Percie Du Sert N, Simonsohn U, Wagenmakers EJ, Ware JJ, Ioannidis JPA (2017)  A manifesto for reproducible science.  Nature Human Behaviour 1:0021.
  * Nadri et al. (2021) Nadri R, Rodriguezperez G, Nagappan M (2021)  On the Relationship Between the Developer’s Perceptible Race and Ethnicity and the Evaluation of Contributions in OSS.  IEEE Transactions on Software Engineering pp. 1–1.
  * Nafus (2012) Nafus D (2012)  ‘Patches don’t have gender’: What is not open in open source software.  New Media & Society 14:669–683.
  * Neylon et al. (2012) Neylon C, Aerts J, Brown CT, Coles SJ, Hatton L, Lemire D, Millman KJ, Murray-Rust P, Perez F, Saunders N, Shah N, Smith A, Varoquaux G, Willighagen E (2012)  Changing computational research. The challenges ahead.  Source Code for Biology and Medicine 7:2, 1751–0473–7–2.
  * Pérez (2011) Pérez F (2011)  Ten years of (interactive) scientific Python.
  * Poldrack et al. (2019) Poldrack RA, Gorgolewski KJ, Varoquaux G (2019)  Computational and Informatic Advances for Reproducible Data Analysis in Neuroimaging.  Annual Review of Biomedical Data Science 2:119–138.
  * Purvis (2006) Purvis A (2006)  The h index: Playing the numbers game.  Trends in Ecology & Evolution 21:422.
  * Roberson (2019) Roberson QM (2019)  Diversity in the Workplace: A Review, Synthesis, and Future Research Agenda.  Annual Review of Organizational Psychology and Organizational Behavior 6:69–88.
  * Seeber et al. (2019) Seeber M, Cattaneo M, Meoli M, Malighetti P (2019)  Self-citations as strategic response to the use of metrics for career decisions.  Research Policy 48:478–491.
  * Seidl et al. (2016) Seidl A, Wrzaczek S, El Ouardighi F, Feichtinger G (2016)  Optimal Career Strategies and Brain Drain in Academia.  Journal of Optimization Theory and Applications 168:268–295.
  * Singh and Bongiovanni (2021) Singh V, Bongiovanni B (2021)  Motivated and Capable but No Space for Error: Women’s Experiences of Contributing to Open Source Software.  The International Journal of Information, Diversity, & Inclusion (IJIDI) 5\.
  * Smith et al. (2016) Smith AM, Katz DS, Niemeyer KE, FORCE11 Software Citation Working Group (2016)  Software citation principles.  PeerJ Computer Science 2:e86.
  * Smith et al. (2018) Smith AM, Niemeyer KE, Katz DS, Barba LA, Githinji G, Gymrek M, Huff KD, Madan CR, Cabunoc Mayes A, Moerman KM, Prins P, Ram K, Rokem A, Teal TK, Valls Guimera R, Vanderplas JT (2018)  Journal of Open Source Software (JOSS): Design and first-year review.  PeerJ Computer Science 4:e147.
  * Switters and Osimo (2019) Switters J, Osimo D (2019)  Recognising the importance of software in research: Research Software Engineers (RSEs), a UK example.  Technical report, European Commission. Directorate General for Research and Innovation., LU.
  * The Astropy Collaboration et al. (2013) The Astropy Collaboration, Robitaille TP, Tollerud EJ, Greenfield P, Droettboom M, Bray E, Aldcroft T, Davis M, Ginsburg A, Price-Whelan AM, Kerzendorf WE, Conley A, Crighton N, Barbary K, Muna D, Ferguson H, Grollier F, Parikh MM, Nair PH, Günther HM, Deil C, Woillez J, Conseil S, Kramer R, Turner JEH, Singer L, Fox R, Weaver BA, Zabalza V, Edwards ZI, Azalee Bostroem K, Burke DJ, Casey AR, Crawford SM, Dencheva N, Ely J, Jenness T, Labrie K, Lim PL, Pierfederici F, Pontzen A, Ptak A, Refsdal B, Servillat M, Streicher O (2013)  Astropy: A community Python package for astronomy.  Astronomy & Astrophysics 558:A33.
  * Vasilescu et al. (2015) Vasilescu B, Filkov V, Serebrenik A (2015)  Perceptions of Diversity on Git Hub: A User Survey  In 2015 IEEE/ACM 8th International Workshop on Cooperative and Human Aspects of Software Engineering, pp. 50–56, Florence, Italy. IEEE.
  * Vasilescu et al. (2015) Vasilescu B, Posnett D, Ray B, van den Brand MG, Serebrenik A, Devanbu P, Filkov V (2015)  Gender and Tenure Diversity in GitHub Teams  In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 3789–3798, Seoul Republic of Korea. ACM.
  * Whitaker and Guest (2020) Whitaker K, Guest O (2020)  #bropenscience is broken science .
  * Zhivotovsky and Krutovsky (2008) Zhivotovsky LA, Krutovsky KV (2008)  Self-citation can inflate h-index.  Scientometrics 77:373–375.