The 21cm brightnesstemperature field of neutral hydrogen during the Epoch of Reionization and Cosmic Dawn is a rich source of cosmological and astrophysical information primarily due to its significant nonGaussian features. However the complex nonlinear nature of the underlying physical processes makes analytical modelling of this signal challenging. Consequently studies often resort to seminumerical simulations. Traditional analysis methods which rely on a limited set of summary statistics may not adequately capture the nonGaussian content of the data as the most informative statistics are not predetermined. This paper explores the application of machine learning ML to surpass the limitations of summary statistics by leveraging the inherent nonGaussian characteristics of the 21cm signal. We demonstrate that a welltrained neural network can independently reconstruct the hydrogen density spintemperature and neutralfraction fields with crosscoherence values exceeding 0.95 for kmodes below 0.5 Mpc h1 based on a representative simulation at a redshift of z approx 15. To achieve this the neural network utilises the nonGaussian information in brightness temperature images over many scales. We discuss how these reconstructed fields which vary in their sensitivity to model parameters can be employed for parameter inference offering more direct insights into underlying cosmological and astrophysical processes only using limited summary statistics of the brightness temperature field such as its power spectrum.
['Astrophysics - Cosmology and Nongalactic Astrophysics']
Inferring the density spintemperature and neutralfraction fields of HI from its 21cm brightness temperature field using machine learning
{'Inferring the density, spin-temperature and neutral-fraction fields of HI from its 21-cm brightness temperature field using machine learning': 'Bohdan Bidenko, 1 , 2 Léon V. E. Koopmans, 1 ★ P. Daniel Meerburg 2  \n- 1 Kapteyn Astronomical Institute, University of Groningen, PO Box 800, NL-9700 AV Groningen, the Netherlands\n- 2 Van Swinderen Institute for Particle Physics and Gravity, University of Groningen, Nĳenborgh 4, 9747 AG Groningen, The Netherlands  \nAccepted XXX. Received YYY; in original form ZZZ', 'ABSTRACT': 'The 21-cm brightness-temperature field of neutral hydrogen during the Epoch of Reionization and Cosmic Dawn is a rich source of cosmological and astrophysical information, primarily due to its significant non-Gaussian features. However, the complex, nonlinear nature of the underlying physical processes makes analytical modelling of this signal challenging. Consequently, studies often resort to semi-numerical simulations. Traditional analysis methods, which rely on a limited set of summary statistics, may not adequately capture the non-Gaussian content of the data, as the most informative statistics are not predetermined. This paper explores the application of machine learning (ML) to surpass the limitations of summary statistics by leveraging the inherent non-Gaussian characteristics of the 21-cm signal. We demonstrate that a well-trained neural network can independently reconstruct the hydrogen density, spin-temperature, and neutral-fraction fields with cross-coherence values exceeding 0.95 for 𝑘 -modes below 0 . 5 Mpc h -1 , based on a representative simulation at a redshift of 𝑧 ≈ 15. To achieve this, the neural network utilises the non-Gaussian information in brightness temperature images over many scales. We discuss how these reconstructed fields, which vary in their sensitivity to model parameters, can be employed for parameter inference, offering more direct insights into underlying cosmological and astrophysical processes only using limited summary statistics of the brightness temperature field, such as its power spectrum.  \nKey words: dark ages, reionization, first stars - methods: data analysis - software: machine learning', '1 INTRODUCTION': "Throughout much of the history of the evolution of the universe in its first billion years, its baryonic component consisted mostly of neutral hydrogen. Analysis of physical processes during this period is possible with the help of the 21-cm emission line caused by the spin-flip transition of the electron-pair making up the hydrogen atom.  \nThe unique physical information contained in this signal has led to the development of an entire field known as 21-cm cosmology (e.g., Barkana & Loeb 2001; Furlanetto et al. 2006; Pritchard & Loeb 2012; Mesinger 2019). Forecasts for experiments such as the GMRT(Paciga et al. 2013), LOFAR (Patil et al. 2017; Mertens et al. 2020; Ghara et al. 2020), NenuFAR (Mertens et al. 2021; Munshi et al. 2024), LWA (Eastwood et al. 2019), MWA (Barry et al. 2019; Li et al. 2019; Trott et al. 2020), PAPER (Kolopanis et al. 2019), HERA (DeBoer et al. 2017; Abdurashidova et al. 2022), and SKA (Koopmans et al. 2015; Mellema et al. 2015) aim to significantly improve constraints on the models describing the astrophysics of the first ionizing sources as well as cosmological parameters (Liu & Shaw 2020).  \nThe traditional approach to studying the 21-cm signal has relied on measuring or constraining fluctuations of the 21-cm brightness temperature, which can be qualified by taking moments of the distribution, of which the most well-studied example is the power spectrum  \n- ★ E-mail: koopmans@astro.rug.nl  \n(see e.g., Morales & Hewitt 2004). Current attempts to measure the 21-cm power spectrum (see references above) mostly aim to detect a signal from the period when the universe was completely neutral to mostly ionized. This window is referred to as Cosmic Dawn (CD) and the Epoch of Reionization (EoR) and roughly spans 30 ≤ 𝑧 ≤ 5. Below 𝑧 = 5, the 21-cm signal can still be observed inside of halos/galaxies as a biased tracer of structure on large scales, which is the target for current experiments such as CHIME (Amiri et al. 2022), Tianlai (Chen 2012), HIRAX (Crichton et al. 2022) and MEERKAT (Cunnington et al. 2022), who recently claimed detection of the 21cm signal power spectrum at low redshift.  \nWhile most effort has been put into detecting the power spectrum (or cross-power with large scale structure in the case of low𝑧 21-cm signal, see e.g. (Pen et al. 2009; Chang et al. 2010; Masui et al. 2013; Anderson et al. 2018; Wolz et al. 2022)), the distribution of 21-cm fluctuations is non-Gaussian and its power spectrum does not capture all information present in the field. As such, relying on the power spectrum alone is sub-optimal in constraining the physics that sources the signal. In this paper, we focus on the late stages of the CD or early stages of the EoR, when the universe is still mostly neutral, but patches of ionized regions start to appear. Several studies have explored the non-Gaussian nature of the signal in this epoch, both using higher-order moments, such as the bi-spectrum (Saxena et al. 2020; Hutter et al. 2020; Watkinson et al. 2021, 2022), as well as the morphology (Chen et al. 2019; Gazagnes et al. 2021; Giri & Mellema 2021; Kapahtia et al. 2021). The bi-spectrum is the most sensitive  \nwhen non-Gaussianity is weak, which is not the case during the CD and EoR.  \nWhat complicates the application of summary statistics is the lack of a closed analytical formulation of the fluctuations during this epoch (see, for example, McQuinn & D'Aloisio (2018); Qin et al. (2022) for recent attempts). This stands in contrast to the post-EoR epoch and the 21-cm signal from the Dark Ages. Thus, both the modelling and analysis of the signals during the CD and EoR epochs heavily rely on simulations. Consequently, the non-linear nature of the processes governing astrophysics and cosmological evolution, as well as the resulting non-Gaussian signal, are challenging to qualify analytically. Developing a standardised estimator that optimally captures this information is likely impossible. Fortunately, with the advancement of machine learning (ML), it has become possible to infer complex relationships between different datasets, enabling us to qualify data beyond its summary statistics.  \nApplying ML techniques to cosmological and astrophysical problems is a rapidly developing field. For example, in the context of 21-cm data, deep-learning-based methodologies were shown to be successful in identifying ionised regions in the presence of challenging astrophysical foreground and instrumental noise contamination (Gagnon-Hartman et al. 2021; Bianco et al. 2021, 2024), in applying sample generation and parameter inference (Schmit & Pritchard 2017; Zhao et al. 2022; List & Lewis 2020) and reconstructing the density fields (Villanueva-Domingo & Villaescusa-Navarro 2021).  \nOne can think of the 21-cm field during the CD and EoR as a product of cosmological and astrophysical evolution (Madau & Rees 2004). The key ingredients of the 21-cm brightness signal are the spin temperature 𝑇 𝑆 , the ionization fraction 𝑥 HI and baryonic matter density fluctuations 𝛿 𝑏 . Optimal use of the data for astrophysical and cosmological inference would benefit from separating astrophysical quantities 𝑇 𝑆 and 𝑥 HI fromthecosmological signal 𝛿 𝑏 , and vice versa. The same reasoning applies when considering the astrophysics hidden in the signal. It is known that at the level of the power spectrum, parameters that are associated with cosmology and astrophysics can have strong degeneracies, which can only be partially broken when considering measurements of the power spectrum at several redshifts (see e.g., Greig & Mesinger 2017). Given the non-Gaussian nature of the 21-cm signal during the CD and EoR, it is only natural to explore whether non-Gaussian information is capable of reducing these degeneracies and providing additional information. This is the main motivation behind this paper.  \nIn particular, inspired by applications of machine learning aimed at density-field reconstruction (Flöss & Meerburg 2024), we develop a three-dimensional field-to-field reconstruction network that is capable of reconstructing the three fields 𝑇 𝑆 , 𝑥 HI and 𝛿 𝑏 directly from the brightness fluctuations 𝛿𝑇 𝑏 , i.e. the observed field. The network utilizes the full non-Gaussian information in the observed field to reconstruct the other fields. We show that the information in the power spectra of these fields is complementary to the power spectrum of 𝛿𝑇 𝑏 , and if combined, they could lead to improved parameter constraints. In this paper, we consider a simple set-up focused on a single redshift, but considering a range of reionization histories simulated using the semi-numerical public code 21cmFAST (Mesinger et al. 2011; Greig & Mesinger 2015; Park et al. 2019). Performing parameter forecasts and exploring applications of the reconstructed fields will be presented in future publications. As such, this paper serves as a proof-of-principle.  \nOur paper is organized as follows. In section 2, we briefly describe the difference between the observable 21-cm signal and the underlying physical fields and motivate using a simple two-parameter Fisher forecast why reconstructing the field could benefit parameter infer-  \nence. In section 3, we describe the simulations used for training and testing of the neural network and provide details on its architecture and the training procedure. In section 4, we present the results of the reconstruction and qualify the performance of the network by measuring the cross-spectra. Finally, we discuss limitations and discuss future directions in section 5.", '2 THE 21-CM SIGNAL OF NEUTRAL HYDROGEN': "In this section, we summarise the brightness temperature field's dependence on the neutral hydrogen fraction, its spin temperature, and the baryonic overdensity field. Through a straightforward Fisher information analysis, we demonstrate that the power spectra of these fields provide additional information beyond that contained in the power spectrum of the brightness temperature field alone.", '2.1 The Brightness Temperature Field': "The observed 21-cm brightness temperature is defined by the difference between the spin temperature 𝑇 𝑆 and the Cosmic Microwave Background (CMB) temperature 𝑇 CMB , modulated by the Sobolov optical depth along the line of sight. It can be approximated as follows (Madau & Rees 2004; Furlanetto et al. 2006):  \n𝛿𝑇 𝑏 = 𝑇 𝑆 -𝑇 CMB 1 + 𝑧 ( 1 -e -𝜏 𝜈 ) ≈ 27 𝑥 HI ( 1 + 𝛿 𝑏 ) GLYPH<18> Ω 𝑏 ℎ 2 0 . 023 GLYPH<19> GLYPH<18> 0 . 15 Ω 𝑚 ℎ 2 1 + 𝑧 10 GLYPH<19> 1 / 2 × GLYPH<18> 𝑇 𝑆 -𝑇 CMB 𝑇 𝑆 GLYPH<19> GLYPH<20> 𝜕 𝑟 𝑣 𝑟 ( 1 + 𝑧 ) 𝐻 ( 𝑧 ) GLYPH<21> mK , (1)  \nHere, 𝑥 HI is the neutral-hydrogen fraction, 𝛿 𝑏 is the fractional overdensity of baryonic matter, and 𝜕 𝑟 𝑣 𝑟 is the velocity gradient along the line of sight. The spin temperature measures the number of hydrogen atoms in the two hyperfine spin-states (parallel or antiparallel). During the initial period after the CMB has formed, the CMB and the gas temperature are tightly coupled, and a balance of collisional and radiative transitions is set, 𝑇 𝑆 = 𝑇 CMB = 𝑇 𝑔 . As the universe expands, the CMB photons decouple from the gas while collisions couple the spin temperature to the gas temperature, which cools adiabatically. This is the moment ( 𝑧 ∼ 200) when the 21-cm brightness can be seen in absorption against the CMB. As the gas dilutes further, collisions become inefficient, and the spin temperature approaches the CMB temperature ( 𝑧 ∼ 50). When the first stars form, Lyman𝛼 photons again couple the spin temperature to the gas, leading to a peak in the brightness temperature (still in absorption) at 𝑧 ∼ 20, until X-rays from the first stars heat the gas during the EoR. The brightness temperature can then be observed in either absorption ( 𝑇 𝑆 < 𝑇 CMB ) or emission ( 𝑇 𝑆 > 𝑇 CMB ). During most of the EoR, 𝑇 𝑆 > 𝑇 CMB , and hence the brightness temperature is positive. This is the regime we will study in this paper. We refer to Pritchard & Loeb (2012) and Mesinger (2019) for a more detailed review of the above processes.  \nThe primary focus of nearly all studies has been the summary statistics of the 𝛿𝑇 𝑏 field. When observing the 21-cm signal, the 𝛿𝑇 𝑏 field is the only direct observable. As shown by Eq. (1), however, the brightness temperature is related to three other fields 𝑇 𝑆 , 𝑥 HI and 𝛿 𝑏 . Note that in principle, it also couples to fluctuations in the CMB temperature, but these are typically negligible since Δ 𝑇 CMB / 𝑇 CMB ∼ O( 10 -5 ) . The evolution of the fluctuations in these  \nFigure 1. Fisher forecast for 𝜁 , 𝛼 X and 𝑇 vir derived from the power spectrum of the brightness temperature field (blue), the spin temperature field (red), and the ionization field (green). We used cosmic variance limited power spectra in the range 0 . 04 h / Mpc ≤ k ≤ 0 . 6 h / Mpc. The power spectra were obtained from simulations at 𝑧 = 15 obtained with the semi-numerical code 21cmFAST , fixing all other parameters. The derivatives were computed using finite difference derivatives. This simple Fisher analysis shows how different fields respond to reionization parameters and whether the information in these fields is independent. Combining their information would break degeneracies and improve parameter constraints.  \n<!-- image -->  \nfields is determined by complex cosmological and astrophysical processes. The baryonic matter density perturbations, 𝛿 𝑏 , grow through gravitational interactions but, on small scales, are expected to be affected by astrophysical feedback effects. The neutral fraction of hydrogen (and its fluctuations), 𝑥 HI , is determined by photoionization by early stars and quasars. The spin temperature field, 𝑇 𝑆 , is defined by collisional coupling in dense regions, coupling with CMB radiation in lower kinetic temperature regions, X-ray heating, and Ly𝛼 pumping mechanism (for details see e.g. Barkana & Loeb 2001; Furlanetto et al. 2006; Pritchard & Loeb 2012; Mesinger 2019).  \nModelling these effects analytically is challenging (see e.g. McQuinn & D'Aloisio 2018; Qin et al. 2022) and we rely on numerical simulations to make predictions about the brightness temperature field, which relies on 𝑇 𝑆 , 𝛿 𝑏 and 𝑥 HI . Full numerical codes that include ray-tracing are computationally expensive, and to render the number of cosmological boxes necessary for parameter inference, or in this case, for the training of a neural network, is infeasible at the moment. Hence, for that purpose, we use the semi-numerical code 21cmFAST (Mesinger et al. 2011; Park et al. 2019). It starts with cosmological initial conditions rendered using 2LPT using standard cosmological parameters. The code models the formation and evolution of dark-matter halos using the Press-Schechter formalism, incorporates star formation and feedback processes within these halos, and calculates the propagation of ionizing radiation from stars and galaxies. Key parameters are the ionizing efficiency 𝜁 , representing the number of ionizing photons emitted per baryon in collapsed structures and 𝑇 vir , which sets the conditions under which gas can col-  \nwithin dark matter halos and 𝛼 X which quantifies how efficiently X-ray photons can heat the surrounding intergalactic medium (IGM) compared to ionizing photons produced by stars and galaxies.", '2.2 Fisher Information': 'In this subsection, we show the information these fields contain on three astrophysical parameters to motivate our effort to reconstruct the fields determining the observed brightness temperature.  \nIn Figure 1, we show the one- and two-dimensional marginal constraints determined by performing a simple Fisher forecast (Fisher 1935) derived directly from 21cmFAST simulations using the power spectra of 𝛿𝑇 𝑏 (blue contours), 𝑇 𝑆 (red) and 𝑥 HI (green). Because we focus on astrophysical parameters rather than containing cosmology (as done by Flöss & Meerburg 2024), we can ignore the baryonic density field. The results are obtained for a noiseless cosmic-variancelimited experiment, and the power spectra are measured at redshift 𝑧 = 15 using 256 3 resolution boxes with a side scale of 1024 Mpc/h, ensuring 𝑇 CMB / 𝑇 𝑆 is not so small such that the effect of 𝑇 𝑆 is negligible.  \nWenotice that the X-ray spectral index parameter, 𝛼 X , has stronger constraints from the 𝑇 𝑆 power spectrum than from 𝛿𝑇 𝑏 due to stronger 𝑇 𝑆 coupling with the X-ray background (Field 1959; Mesinger et al. 2011). The parameter that defines the ionizing efficiency of UV radiation from early galaxies 𝜁 is constrained stronger by the 𝑥 HI field. The minimal virial temperature parameter log 10 ( 𝑇 min vir ) has similar constraints from 𝑇 𝑆 and 𝛿𝑇 𝑏 . However, due to different types of degeneracies with the 𝛼 X parameter, the constraints could be optimised by combining information from these fields (after properly accounting for their covariance). This simple parameter Fisher forecast shows that if one were to have access to these additional fields, a simple summary statistic, such as their power spectra, could already improve astrophysical parameter constraints.  \nHowever, since we do not have direct access to these fields, we explore the potential of a deep learning-based algorithm for the reconstruction of the 𝑇 𝑆 , 𝑥 HI and 𝛿 𝑏 using only the observed 𝛿𝑇 𝑏 field. Note that the information to reconstruct these fields must be derived from the non-Gaussian nature of the brightness temperature. In principle, all information is present in this one observable field, but this information is contained in multiple higher-order moments. If we were able to capture and model that information correctly, the resulting constraints would be more optimal. However, given the challenge of both modelling and measuring these higher-order moments, a direct reconstruction would allow for a relatively straightforward analysis using the power spectra after reconstruction.', '3 METHODOLOGY': 'In this section, we shortly describe the simulations of the brightnesstemperature field and the neural network used to infer the neutralfraction, baryonic-density and spin-temperature fields from the brightness-temperature field.', '3.1 Simulations': "We aim to recover underlying physical fields using supervised machine learning based on the information present in the observable. As a first attempt, we will use simulations without including any observational and instrumental effects. Therefore, the simulations used in this work do not contain any noise, and the information  \nFigure 2. Architecture of the neural network used for reconstruction. Image credit: Flöss & Meerburg (2024)  \n<!-- image -->  \nabout the physical processes is limited only by the quality of the simulations, their size, and the binning effect. We leave that to a future analysis. We simulate the four fields using 21cmFAST . For the training and validation set, we generate simulations with different astrophysical parameters: 𝜁 ∈ { 28 , 32 } , 𝛼 x ∈ { 0 . 935 , 1 . 065 } , and log 10 ( 𝑇 min vir ) ∈ { 4 . 67 , 4 . 69 , 4 . 71 , 4 . 73 } . We chose parameters whose constraints could benefit from the field reconstruction as shown in Figure 1. The chosen range of parameters lies within 1 𝜎 of the parameter constraints forecast for upcoming experiments (e.g. Saxena et al. 2023). The test data is generated with parameter values between the test grid points: 𝜁 = 30, 𝛼 x = 1, and log 10 ( 𝑇 min vir ) = 4 . 7, which corresponds to the 'FAINT GALAXIES' model of Greig & Mesinger (2017). With this set of simulations, we determine to what extent our network can interpolate to models that are not part of the training set. A more elaborate study of reconstruction quality as a function of parameter space is a topic of ongoing investigation. The range considered here should again serve as a proof-of-principle.  \nWe chose to study the reconstruction in a regime close to the maximumaverage brightness temperature. Therefore, we simulate all fields at a redshift 𝑧 = 15. Each simulation data cube has a resolution of 256 3 voxels and a physical side size of 1024 Mpc/h. Besides 𝛼 X , 𝑇 vir and 𝜁 , all other astrophysical and cosmological parameters in our simulations were set to the default values of 21cmFAST . All simulations used for training, validation and testing are generated with different randomised initial realisations (seeds).", '3.2 U-net architecture': 'For the reconstruction of the physical fields, we use the neural networkwiththesamearchitecture as was proposed in Flöss & Meerburg (2024), which was inspired by the work of Gagnon-Hartman et al. (2021). The reconstruction algorithm starts with the construction of  \nsix additional fields 1 for each input 𝛿𝑇 𝑏 field:  \n𝒖 ( 𝒙 ) = ∫ 𝑑 3 𝒌 -𝑖 𝒌 𝑘 2 𝛿𝑇 𝑏 ( 𝒌 ) 𝑒 𝑖 𝒌 · 𝒙 (2)  \n𝜕 𝑖 ( 𝒖 ) 𝑗 ( 𝒙 ) = ∫ 𝑑 3 𝒌 𝑘 𝑖 𝑘 𝑗 𝑘 2 𝛿𝑇 𝑏 ( 𝒌 ) 𝑒 𝑖 𝒌 · 𝒙 , (3)  \nwhere 𝛿𝑇 𝑏 ( 𝒌 ) is a Fourier transform of the temperature brightness field. It was shown by Flöss & Meerburg (2024) that adding the gradient fields improve reconstruction.  \nThe neural network architecture utilised in our study consists of an initial convolutional layer with a 3 × 3 × 3 kernel size, followed by a context block consisting of two additional convolutional layers with the same kernel size. A residual connection is then applied, combining the output of the initial convolution with that of the context block. This process is repeated, with the initial convolutional layer now employing a stride of two, resulting in the down-sampling of feature maps by a factor of two in each spatial dimension. This down-sampling procedure is iterated three more times until the feature maps reach a size of 16 × 16 × 16 cells. Subsequently, the feature maps are upsampled by a factor of two, and the outputs of the residual connections from the down-sampling phase are concatenated, creating skip connections. These concatenated feature maps are then fed into a localisation block, comprising a 3 × 3 × 3 ordinary convolutional layer followed by a 1 × 1 × 1 convolutional layer, with this process repeated until the input size reaches 256 × 256 × 256. Finally, one more convolutional layer of size 1 × 1 × 1 is applied to generate a single feature map representing the reconstructed field. Throughout the architecture, except for the final convolutional layer, each layer consists of convolution, instance normalisation, and leaky ReLU activation functions.  \nmK  \nFigure 3. Slices of the four fields projected along the third dimension of the data cube. First column: the input brightness temperature field. Second column: corresponding target field. Third column: reconstructed field. Fourth column: the difference between the reconstructed and true field. The first row shows the reconstruction of the spin temperature field 𝑇 𝑆 , the second row shows the reconstruction of the ionization fraction 𝑥 HI, and the third row shows the reconstruction of the baryonic matter fractional overdensity 𝛿 𝑏 . Residuals are very small, indicating high-quality reconstruction for all three target fields.  \n<!-- image -->', '3.3 Training': "The U-net is trained independently for each type of the reconstructed physical field due to memory limitations. Thus, we train three independent neural networks in this work. For each network, we use 240 simulations of 𝛿𝑇 𝑏 field data cubes as input dataset and corresponding 𝛿 𝑏 , 𝑇 𝑠 , and 𝑥 HI as output datasets. The training set contains 15 field realisations for each point in parameter space. These simulations are used to optimise neural network parameters using a mean square error loss function:  \nL MSE = 1 𝑁 sim 1 256 3 𝑁 sim ∑︁ 𝑖 = 1 GLYPH<0> 𝐹 𝑖, true -𝐹 𝑖, rec GLYPH<1> 2 , (4)  \nwhere 𝐹 𝑖, true and 𝐹 𝑖, rec are the true target and reconstructed field, respectively, and the difference is calculated between individual voxel values. The loss function is normalised by the number of voxels and simulations in the set 𝑁 sim . The validation dataset consists of 48 simulations (three per parameter-space point), and the test dataset consists of 90 simulations with one fixed parameter set as discussed in section 3.1.  \nFor the spin temperature field, 𝑇 𝑆 , we cap the field's maximum value at the 99.5th percentile. This limit is essential to prevent instability in the learning process that arises when the U-net reconstructs very large localised peaks and affects only a few voxels. This adjustment has a negligible impact on the power spectrum and other statistical properties of the fields. Finally, for training purposes, we  \nnormalize fields using the standard deviation of the training set fields and subtract the mean value of the training fields. The true values of the reconstructed fields are recovered through inverse re-scaling and applying a corresponding shift.", '4 RESULTS': 'In this section, we present a representative reconstruction of the neutral-fraction, baryonic-density, and spin-temperature fields from a random brightness-temperature field, which was given as input to the trained U-net. We visually inspect these, analyse their power spectra, and assess their cross-coherence with the true underlying fields. Again, we emphasise that this is proof of concept, and a full exploration of a wider range of redshifts and model parameters is left for future analyses.', '4.1 Visual inspection': "To qualitatively assess the reconstructed fields, we first perform a visual comparison. Representative examples of the network reconstruction are shown in Figure 3. In the first column, we show a slide of the input 𝛿𝑇 𝑏 field along one of the data cube's axes. We note that these are the same for each row but shown for ease of comparison with the other fields. The second column shows the true target fields 𝑇 𝑆 , 𝑥 HI , and 𝛿 𝑏 in the first, second and third rows, respectively.  \nThe third column shows the results of the reconstruction, obtained with a U-net for each field. The difference between the true and the reconstructed field is shown in the fourth column.  \nRemarkably, there are no strong visual imperfections in residuals across all the simulations used in training, validation and tests. Residuals do not feature apparent structure or inhomogeneities. The 𝑥 HI residuals show some minor localised peaks. These peaks are associated with a small misalignment of the positions of ionised regions between the reconstructed and the true field. However, due to the small-scale nature of these features, they do not introduce significant errors on the scales that forthcoming experiments are expected to probe with reasonable signal-to-noise (i.e., 𝑘 ≤ 0 . 5 h/Mpc).", '4.2 Power spectra of the reconstructed and true fields': 'To quantitatively assess the reconstructed fields, we compute the power spectrum of the reconstructed field and compare it to the power spectrum of the target field. The power spectrum of the binned three-dimensional fields is calculated using the following estimator:  \n𝑃 ( 𝑘 𝑖 ) = ⟨| 𝐹 ( 𝒌 )| 2 ⟩ = 1 𝑁 𝑘 𝑖 ∑︁ 𝒌 ∈ 𝑘 𝑖 ± Δ 𝑘 | 𝐹 ( 𝒌 )| 2 , (5)  \nwhere 𝐹 ( 𝒌 ) is a Fourier transform of the field 𝐹 . The square magnitude of the Fourier transform is calculated as the spherical layer average in the wavenumber space defined by the radius 𝑘 𝑖 , with a width Δ 𝑘 equal to the fundamental Fourier mode of the data cube. The number of Fourier modes in the spherical layer 𝑁 𝑘 𝑖 is taken into account as a normalization factor.  \nWe present the power spectra in Figure 4, averaged over the test dataset. From left to right, the panel shows the statistical properties of the 𝑇 𝑆 field, the 𝑥 HI field, and 𝛿 𝑏 field, respectively. The power spectra of the reconstructed fields (shown in red lines) in all three cases deviate from true target power spectra (blue dashed lines) by less than 10% across the entire range of scales. We also show the residual power spectra (black lines). The residual power is ≤ 7% of the field power spectrum for 𝑘 ≤ 0 . 5 h / Mpc. The 𝑥 HI residual power spectrum increases on small scales (high 𝑘 values) as expected from the localized peaks in the residual. Despite this behaviour of the residual, the deviation of the reconstructed power spectrum is ≤ 2 . 5% in the case of 𝑥 HI field (black dashed line in the bottom panel of Figure 4). Similarly, the reconstructed power spectra of 𝑇 𝑆 and 𝛿 𝑏 deviate ≤ 7%. Overall, the reconstructions are remarkably good.', '4.3 Cross-coherence between the reconstructed and true fields': "While the previous section showed that the U-net reconstruction method successfully recovers the power spectra of the target fields, a more ambitious goal is to recover the target fields on a pixel-to-pixel level. This is possible because the network relies on a pixel-by-pixel comparison of the loss function used in training (Eq. (4)). To further quantify the quality of the reconstructed fields, we compute the cross-coherence between the target (i.e. true) and the reconstructed field. Similar to the power spectra, the coherence is defined as the normalised cross-power spectrum, as follows:  \n𝐶 𝑖, 𝑗 ( 𝑘 ) = ⟨ 𝐹 𝑖 ( 𝒌 ) 𝐹 ∗ 𝑗 ( 𝒌 )⟩ 2 ⟨| 𝐹 𝑖 ( 𝒌 )| 2 ⟩⟨| 𝐹 𝑗 ( 𝒌 )| 2 ⟩ = 𝑃 2 𝑖, 𝑗 ( 𝑘 ) 𝑃 𝑖 ( 𝑘 ) 𝑃 𝑗 ( 𝑘 ) . (6)  \nHere 𝑃 𝑖, 𝑗 is cross-power spectrum of field 𝑖 and field 𝑗 , 𝑃 𝑖 and 𝑃 𝑗 are auto-power-spectra of fields 𝑖 and 𝑗 , obtained using Eq. (5), respectively. Coherence values of unity correspond to identical target  \nand reconstructed fields, and values of zero correspond to completely uncorrelated target and reconstructed fields.  \nWe show the coherence between reconstructed and target fields (black lines), as well as the coherence of the target with the input field 𝑇 𝑏 (blue), in Fig. 5. The reconstructed fields in the test dataset have a mean coherence of ≥ 0 . 92 for scales 𝑘 ≤ 0 . 5h / Mpc with the target field. The coherence increases on larger physical scales, which is in agreement with the trend in the power spectrum residual. It is also indicative that non-Gaussian information is being used to reconstruct these scales. The rapid decrease in cross-coherence on smaller scales ( 𝑘 ≥ 0 . 15h / Mpc) between the reconstructed field and target field shows that they are not simply scaled versions of each other.  \nTo test that a single point in the reconstructed field depends on a wide range of scales and surrounding points, we examined the network's reconstruction capabilities by applying cuts in Fourier space, thereby removing certain scales from the input field similar to Gagnon-Hartman et al. (2021). As anticipated, the quality of the reconstruction deteriorates, showing sensitivity to the specific scales being excluded. When large scales (small 𝑘 values) are removed, the networks can still reconstruct modes absent in the observed fields, indicating the importance of small, non-Gaussian scales in the reconstruction process. Conversely, removing small scales hampers the reconstruction, as it eliminates critical non-Gaussian information. A comprehensive analysis of the specific scales contributing to the reconstruction will be addressed in future work.", '5 DISCUSSION AND CONCLUSIONS': 'We have presented a deep learning-based (U-net) method that successfully reconstructs all three physical fields (i.e., baryonic density, neutral fraction and spin-temperature) that largely determine the observed 21-cm brightness temperature field of neutral hydrogen during the Epoch of Reionization and Cosmic Dawn. The motivation to build such a network stems from the fact that these fields have different responses to both astrophysical and cosmological parameters, making it possible to independently harvest the information in these fields to improve parameter constraints. The network utilises the non-Gaussian information present in the brightness temperature field. This non-Gaussian information non-trivially couples modes at different scales and the networks use this coupling to reconstruct the target fields. This is most evident from the fact that the observed field 𝑇 𝑏 has poor cross-correlation with the target fields on small scales, while after reconstruction, we find very high cross-correlation on those same scales. Since this is not simply an overall re-scaling of the field, the information is recovered from non-linear relations, i.e. from non-Gaussian information present in the brightness temperature data cube. Concretely, we find the following:  \n- · The power spectra of the reconstructed spin temperature 𝑇 𝑆 , neutral hydrogen fraction 𝑥 HI and baryonic matter density perturbation 𝛿 𝑏 field - simulated with the semi-numerical code 21cmFAST -deviate ≤ 7% at 𝑘 = 0 . 5 h/Mpc from the power spectra of the true target field, and even less on larger scales.\n- · The cross-coherence of the reconstructed fields with the true target fields is found to be 𝐶 ≥ 0 . 92 on scales 𝑘 < 0 . 5h / Mpc, much higher than the cross-coherence with the brightness-temperature field. This confirms that the reconstruction extends beyond the similarity of statistical properties but features a high degree of pixel-topixel accuracy of the reconstructed field.  \nThe primary conclusion of our proof-of-principle analysis is that  \nFigure 4. Top panel: Power spectrum of reconstructed field (red), true target field (blue dashed), and their residual (black). Left: spin temperature field; central: neutral hydrogen fraction; right: baryonic density fluctuations. Bottom panels: fractional deviation of the reconstructed field power spectrum from the target true field power spectrum.  \n<!-- image -->  \nFigure 5. Cross-correlation (coherence) between the target field from the test dataset brightness temperature fields (blue) and the reconstructed field (black). The left panel is the spin temperature field; the central panel is the hydrogen neutral fraction; the right panel is the density field fluctuations. Curves represent the mean value for the entire test dataset. On large scales, the brightness temperature is coherent with all other fields. On small scales, the brightness temperature becomes more decoherent. After reconstructing the target fields, we obtain significant coherence up to 𝑘 ∼ 0 . 5 Mpc/h.  \n<!-- image -->  \nthe redshifted 21-cm brightness temperature field retains most of the information regarding the three underlying fields: baryonic matter density, neutral fraction, and spin temperature. These fields can be accurately reconstructed using a neural network that effectively disentangles them from a single input field.  \nHowever, several additional steps are necessary to fully evaluate the practical applications of these methods. First, it is crucial to assess whether the inclusion of realistic noise, instrumental effects, and foregrounds impacts the reconstruction quality. Second, performing a Fisher analysis on the reconstructed fields is essential to determine their utility for parameter inference, as demonstrated by Flöss & Meerburg (2024). Third, our current setup focuses on a relatively high redshift of 𝑧 = 15, where the ionized regions are small 2 . Some preliminary tests, however, show that the reconstruction quality diminishes when large ionized patches are present in the observed field. To mitigate this, one possible approach is to complement the 21-cm brightness measurements with other line tracers, such as OIII or CO , which exhibit significant fluxes in ionised regions. Integrating  \nthese tracers into the reconstruction process would be interesting to examine.  \nIn addition to using the reconstructed fields for parameter constraints, another promising application is their use in crosscorrelation studies. For instance, inhomogeneous (patchy) reionization induces secondary fluctuations in the CMB. It is feasible to reconstruct the associated patchy 𝜏 map, which serves as a proxy for the ionization power spectrum integrated along the line of sight. According to Eq. (1), the 21-cm brightness temperature is sensitive to 𝑥 HI . Cross-correlations between the brightness temperature and the reconstructed patchy 𝜏 signal have been explored in the literature (Meerburg et al. 2013; Roy et al. 2020). However, the line-of-sight mode is typically lost when foregrounds are either avoided or removed, complicating the direct measurement of this cross-correlation. Based on our analysis, it is likely that the reconstructed 𝑥 HI field, even with large-scale cuts to account for foreground removal or avoidance, may be less affected by this issue. Moreover, the reconstructed 𝑥 HI field is expected to have a high coherence with the patchy 𝜏 signal, given that both measure the same underlying field, albeit one integrated along the line of sight.  \nWe are currently investigating this and other potentially valuable applications, which we aim to report in future publications.', 'ACKNOWLEDGEMENTS': 'We link to thank Thomas Flöss, and Anchal Saxena for useful discussions and initial collaboration on the project. Furthermore, we like to thank the Center for Information Technology of the University of Groningen for their support and for providing access to the Habrok high-performance computing cluster. BB is supported by the Fundamentals of the Universe research program at the University of Groningen. LVEK acknowledges the financial support from the European Research Council (ERC) under the European Union\'s Horizon 2020 research and innovation programme (Grant agreement No. 884760, "CoDEX").', 'DATA AVAILABILITY': 'Accompanying code is available at https://github.com/ bidenkobd/reconstruction/ . The data underlying this article will be shared on reasonable request to the corresponding author.', 'APPENDIX A: CROSS-SEED TEST': "When training our network, we use randomised seeds (phases) of the initial density field. This ensures that the network does not 'learn' a particular realisation of the fields but instead must learn the correlation structure. To verify that the randomisation of the seeds is sufficient and that the network does not pick up any recurring structure, we computed the correlation coefficients (coherence) of the true fields ( 𝑇 𝑆 , 𝑥 HI , and 𝛿 𝑏 ) generated from different initial seeds and compared these to the correlation coefficients of the reconstructed fields, derived from applying the network to the simulated 𝛿𝑇 𝑏 sourced with randomised seeds. The results are shown in Fig. A1.  \nOverall, the correlation coefficient is ≪ 1, as expected, given that these fields have different phases. Any correlation structure should  \nFigure A1. Coherence between simulated fields with different initial conditions (black) and corresponding reconstructed fields (red)  \n<!-- image -->  \nbe due to the chance correlation of the two random fields. The reconstructed fields also exhibit similar behaviour, where the chance correlations almost perfectly match those of the true fields. Only for the ionization field is there a small decrease in correlation after reconstruction on scales 𝑘 > 0 . 3 ℎ Mpc -1 .  \nThe cause of this increase is not entirely clear. We have explored several hypotheses, but none adequately explains this behaviour. The effect is extremely small and may be simply a result of the coherence between the true field and the reconstructed fields dropping towards smaller scales. However, the absence of this effect in the other two reconstructed fields is somewhat puzzling. Further investigation is necessary to rule out any issues with the network fully. Because the effect is minor and has a negligible impact on the quality of the reconstruction, we will defer this investigation to future work. construction, we will leave this for future work.  \nThis paper has been typeset from a T E X/L A T E X file prepared by the author."}