Deep sound-field denoiser: optically-measured sound-field denoising
  using deep neural network
This paper proposes a deep sound-field denoiser, a deep neural network (DNN)
based denoising of optically measured sound-field images. Sound-field imaging
using optical methods has gained considerable attention due to its ability to
achieve high-spatial-resolution imaging of acoustic phenomena that conventional
acoustic sensors cannot accomplish. However, the optically measured sound-field
images are often heavily contaminated by noise because of the low sensitivity
of optical interferometric measurements to airborne sound. Here, we propose a
DNN-based sound-field denoising method. Time-varying sound-field image
sequences are decomposed into harmonic complex-amplitude images by using a
time-directional Fourier transform. The complex images are converted into
two-channel images consisting of real and imaginary parts and denoised by a
nonlinear-activation-free network. The network is trained on a sound-field
dataset obtained from numerical acoustic simulations with randomized
parameters. We compared the method with conventional ones, such as image
filters, a spatiotemporal filter, and other DNN architectures, on numerical and
experimental data. The experimental data were measured by parallel
phase-shifting interferometry and holographic speckle interferometry. The
proposed deep sound-field denoiser significantly outperformed the conventional
methods on both the numerical and experimental data. Code is available on
GitHub: https://github.com/nttcslab/deep-sound-field-denoiser.
Kenji Ishikawa, Daiki Takeuchi, Noboru Harada, Takehiro Moriya
# Deep sound-field denoiser: optically-measured sound-field denoising using deep neural network

###### Abstract

This paper proposes a deep sound-field denoiser, a deep neural network (DNN) based denoising of optically measured sound-field images. Sound-field imaging using optical methods has gained considerable attention due to its ability to achieve high-spatial-resolution imaging of acoustic phenomena that conventional acoustic sensors cannot accomplish. However, the optically measured sound-field images are often heavily contaminated by noise because of the low sensitivity of optical interferometric measurements to airborne sound. Here, we propose a DNN-based sound-field denoising method. Time-varying sound-field image sequences are decomposed into harmonic complex-amplitude images by using a time-directional Fourier transform. The complex images are converted into two-channel images consisting of real and imaginary parts and denoised by a nonlinear-activation-free network. The network is trained on a sound-field dataset obtained from numerical acoustic simulations with randomized parameters. We compared the method with conventional ones, such as image filters and a spatiotemporal filter, on numerical and experimental data. The experimental data were measured by parallel phase-shifting interferometry and holographic speckle interferometry. The proposed deep sound-field denoiser significantly outperformed the conventional methods on both the numerical and experimental data.

## 1 Introduction

Optical imaging has recently been used for high-spatial-resolution imaging of acoustic phenomena in airborne sound fields that conventional acoustic sensors cannot accomplish [1]. The acousto-optic effect [2], in which the refractive index of a medium is changed by sound, allows sound to be measured from the optical phase variation. Various optical methods have been used, for example, laser Doppler vibrometry (LDV) [2, 3], parallel phase-shifting interferometry (PPSI) [1], and digital holography [4, 5, 6, 7]. The applications include investigation of acoustic phenomena [8, 9, 10] and sound-field back-projection [2, 11, 12, 13]. Owing to their significant advantages, optical technologies are considered promising as a next-generation acoustic sensing modality.

The sound field measured by a high-speed camera or scanning laser beam can be represented as an image sequence. Each pixel value is proportional to the line integral of the sound pressure along the corresponding optical path with superimposed noise. Because the phase fluctuation of light caused by audible sound is tiny owing to its physical origin, noise reduction of sound-field images is a fundamental concern. Whereas sound-field image denoising is typically conducted using filters, as discussed in section 2.1, no machine-learning-based sound-field image denoising method has been proposed so far.

In this paper, we propose a denoising method for sound-field images that is based a deep neural network (DNN) (Fig. 1). A sound-field image sequence is Fourier transformed along the time direction at each pixel, by which is obtained complex-amplitude sound-field images corresponding to the frequency bins of the discrete Fourier transform (FT). Then, each complex
amplitude image is converted into a two-channel image consisting of real and imaginary parts and denoised by using a trained DNN. To train the network, we generated training datasets by performing acoustic simulations with white and/or speckle noises. Randomizing the simulation parameters ensured variety in the training data. Numerical experiments confirmed that the proposed DNN-based method performs better than conventional methods. We also applied the method to data measured by PPSI and holographic speckle interferometry (HSI). It outperformed conventional methods on these experimental data without priori knowledge of the sound field.

## 2 Related works

### Sound-field denoising

The physical properties of sound are commonly utilized for designing noise-reduction filters. These filters can be categorized into time-domain processing, spatial-domain processing, and spatiotemporal-frequency-domain processing.

Time-domain processing is typically the first choice. Because sound pressure varies over time, a high-pass filter with a very low cutoff frequency can eliminate static optical phase components and low-frequency fluctuations caused by air fluctuation and seismic vibration. Taking the difference between successive frames of an image sequence, which is a simple high pass filter, has been used as an easy denoising method for sound-field image sequences [8]. When the frequencies of a measured sound field are known, the noise-reduction performance can be improved by designing an appropriate temporal filter [14].

Spatial-domain processing can be applied independently of the time-domain processing. A spatial filter is applied to the sound-field image at each frame. Since sound is a spatially smooth variation and steep edges are usually absent, typical image processing filters, such as Gaussian and median filters, are effective [15].

Figure 1: Overview of the deep sound-field denoiser. (a) Training process. A sound-field dataset is generated in a 2D acoustic simulation with randomized parameters. Each data is a complex-amplitude sound-field image of a harmonic frequency \(\omega\). A nonlinear activation-free network (NAFNet) is trained using the clean and noisy pairs of the simulated sound fields. (b) Inference process. The time-sequential sound-field images are transformed into complex amplitude images, where each image is denoised by the trained network.
Spatiotemporal-frequency-domain processing utilizes the fact that sound satisfies the equation: \(k=\omega/c\), where \(k\) is the acoustic wavenumber, \(\omega\) is the acoustic angular frequency, and \(c\) is the speed of sound. If we consider a two-dimensional space, this equation forms a cone in \(k-\omega\) space [15]. Since all of the spatiotemporal components in the recorded images that do not exist on the cone are noise, they can be eliminated by filtering [15, 16].

The methods used so far are all classical filters. We developed a DNN-based denoising of sound fields and confirmed that it outperforms these conventional methods.

### Natural image denoising by DNNs

DNNs have been extensively applied to image-denoising tasks and have outperformed classical methods. Convolutional neural networks (CNN) [17, 18, 19, 20, 21, 22] and transformers [23, 24, 25, 26] have widely been used. Among the numerous DNNs, a nonlinear activation free network (NAFnet) [22] has a simple and efficient structure and has achieved peak signal-to-noise ratio (PSNR) of 40.30 dB on a smartphone image denoising dataset [27]. We chose this architecture for our sound-field denoiser.

### DNNs for optical metrology

DNNs have been increasingly used in optical metrology [28]. DNNs have been used in many processes, including pre-processing (e.g., fringe denoising [29] and enhancement [30]), analysis (e.g., phase retrieval [31] and phase unwrapping [32]), and post-processing (e.g., phase denoising [33], error compensation [34], and digital refocusing [35]).

Several DNN-based methods have shown high performance in denoising fringe patterns and optical phase maps. For fringe denoising, a deep CNN consisting of 20 layers was proposed by Yan _et al._, where the training dataset was generated from Zernike polynomials and additive white Gaussian noise. Several methods have also applied DNNs to fringes corrupted by speckle noise [36, 37, 38, 39, 40]. Similar ideas have been used to denoise optical phase maps [41, 42, 43, 44, 33].

However, no research has used DNNs to denoise sound-field images measured by optical methods. Since the spatial and temporal features of sound-field images differ from those of interference fringes and typical optical phase maps, the previous methods may not be optimal for sound-field denoising. Our contribution is that we developed DNN-based sound-field denoising methods and a training dataset that considers the physical nature of sound.

## 3 Methods

### Acousto-optic measurement data

Here, let us briefly review the principle of acousto-optic measurement [2]. The acousto-optic effect is the change in the refractive index of a medium caused by sound. If light propagates along the \(z\)-axis, the phase shift of the light propagating through a sound field in air is given by

\[\phi_{s}(x,y,t)=k_{L}\frac{n_{0}-1}{\gamma P_{0}}\int_{z_{1}}^{z_{2}}p(x,y,z,t)dz, \tag{1}\]

where \(k_{L}\) is the wavenumber of light, \(\gamma\) is the specific heat ratio, \(n_{0}\) and \(P_{0}\) are the refractive index and pressure of air in a static condition, respectively, and \(p\) is the sound pressure. The phase shift of light is proportional to the sound pressure along the laser path. When sound-field imaging is performed based on this principle, the observed data can be written as a three-dimensional array \(\Phi_{\rm noisy}\) whose elements are of the form \(\phi_{s}(x_{i},y_{j},t_{m})\), where \((i,j)\) is the pixel index and \(m\) is the time index. Any processing method that can extract \(\phi_{s}\) from noisy data can be applied.
### DNN-based sound-field denoising

The overview of the inference process is shown in Fig. 1(b). First, a time-domain FT is performed on all pixels of \(\Phi_{\text{noisy}}\) to obtain the complex-amplitude sound field, that is, \(\Psi_{\text{noisy}}=\mathcal{F}_{t}[\Phi_{\text{noisy}}]\), where \(\mathcal{F}_{t}\) denotes a 1D FT along the temporal axis. \(\Psi_{\text{noisy}}\) is the complex amplitude at the corresponding spatial position and the Fourier frequency. Then, for each Fourier frequency, the 2D complex amplitude is converted into a two-channel image with real and imaginary parts. The two-channel complex-amplitude image is normalized and inputted to the neural network. The network is trained to output a clean complex-amplitude image from the input noisy complex-amplitude image. The output image is multiplied by the reciprocal of the normalization factor to maintain the magnitude of the sound field. After processing all frequencies independently with the same DNN, the denoised complex amplitude, \(\Psi_{\text{denoise}}\), is inverse Fourier transformed, and the denoised sound field \(\Phi_{\text{denoise}}=\mathcal{F}_{t}^{-1}[\Psi_{\text{denoise}}]\) is obtained.

Since the proposed method uses DNNs to denoise two-channel input images, any network that is able to perform denoising on images can be used with it. Here, Unet-based networks are often used in optical metrology [28]. In particular, we chose NAFNet [22], which has excellent performance and can run with relatively small memory and training time. A discussion of the optimal network structure and parameters for denoising the sound field may be a subject of future work.

### Training data

Although optical sound measurements have been actively studied in recent years, no dataset exists for training a neural network on them. It is difficult to collect sound-field data under various conditions through experiments. Therefore, this study used acoustic numerical simulation to create a training dataset.

A 2D sound-field simulation with randomized parameters was used. Figure 2(a) shows a schematic illustration of the simulation. The inner rectangle is the measurement area, outside of which is the sound source area where point sources are randomly placed. To generate sound fields with diverse spatial characteristics from simple to complex, the number of point sources was varied from 1 to 5, and the position and relative amplitude of each source was randomly assigned. Each true sound field is a superposition of the sound waves generated by these point

Figure 2: (a) Sound-field data generation. Point sources are randomly generated within the sound source area, and the 2D true sound fields in the center area are generated using the Green’s function of the 2D Helmholtz equation. (b) Examples of the generated sound-field data. \(N\) represents the number of sound sources. Two examples are shown for each \(N\).
sources and can be calculated as

\[p\left(\mathbf{r},k\right)=A\sum_{i=1}^{N}a_{i}\frac{j}{4}H_{0}^{(2)}\left(k|\mathbf{r_{i }}-\mathbf{r}|\right), \tag{2}\]

where \(\mathbf{r}=(x,y)\), \(k\) is the magnitude of acoustic wavenumber, \(A\) is a constant determining the overall magnitude of the sound field, \(N\) is the number of sound sources, \(a_{i}\) and \(\mathbf{r_{i}}=(x_{i},y_{i})\) are the relative amplitude and position of the \(i\)th sound source, respectively, and \(H_{0}^{(2)}\) is a Hankel function of the second kind of order zero. Inside \(\Sigma\) is the product of the relative amplitude of the \(i\)th source and Green's function of the 2D Helmholtz equation.

The true sound fields were created by randomly selecting \(k\), \(a_{i}\), and \(\mathbf{r}_{i}\) from uniform distributions. The measurement area was a square of side length 1, and the sound source area was ten times larger than that. The random parameters were generated from uniform distributions of \(0\leq a_{i}\leq 0.1\), \(1.26\leq k\leq 40.2\), \(0.5\leq|x_{i}|\leq 10\), and \(0.5\leq|y_{i}|\leq 10\). The amplitude of the entire sound field was set to \(A=0.1\). These parameters were determined based on the authors' experience with typical experimental conditions of this measurement technology. \(a_{1}\) was set to 1 regardless of the number of sources to avoid all sources having small amplitudes. The simulated data was calculated by discretizing the measurement area into \(128\times 128\) pixels. The top row of Fig. 2(b) shows examples of the generated sound fields. It can be seen that the generated sound fields have different complexities, wavelengths, and directions of arrival.

Two types of noise were added to the training data: additive white Gaussian noise and speckle noise. The amplitudes of the white noise were randomly selected from a uniform distribution between 0 to 0.1. The method of generating the speckle noise data is shown in the Appendix. Examples of the noisy training data are shown in Fig. 2(b). Data with different amounts of noise were generated. Although the differences between white and speckle noise may be difficult to recognize, spatially correlated random patterns appear in the speckle noise images. Such speckle noise can occur, for example, in a sound field observation using electronic speckle pattern interferometry and a holographic interferometer equipped with Fresnel lenses [46].

### Implementation details

This study used almost the same network as in the original NAFNet article [22], except for the number of image channels. The network consisted of 32 blocks with widths of 32, two image channels (real and imaginary), and a 128 \(\times\) 128 pixel image size. The root mean square error was used as the loss, Adam was used as the optimizer, and the learning rate was set to 0.001. A total of 2,000 training data were created, 400 for each number of sources. The training batch size was 32, and the epoch was 50. The network trained on the white noise dataset is denoted by Ours (W), and the one trained on the speckle noise dataset is denoted by Ours (W+S). The data for evaluation consisted of 500 sound fields (100 for each number of sound sources) generated by simulation under the same conditions as those used for generating the training data. The peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) were used as evaluation metrics.

### Conventional methods

Three conventional denoising methods were used for comparison with the proposed method, i.e., 2D Gaussian filter, 2D median filter, and spatiotemporal band pass filter (ST BPF). The kernel sizes for the Gaussian and median filters were set to 7 pixels. They were applied to the real and imaginary parts of the complex amplitude image, respectively.

The ST BPF is a spatial frequency filter based on the wave equation [15]. In the wavenumber spectrum, sound components lie on the circumference of \(k=(k_{x}^{2}+k_{y}^{2})^{1/2}=\omega/c\). Therefore, noise can be reduced by removing the spatial frequencies that do not satisfy the equation. First, if the input signal is a time series of images, a 2D complex sound field for each frequency
is obtained by taking a 1D FT. Next, a fourth-order image Butterworth band-pass filter was created for and applied to each 2D complex-amplitude image. The lower cutoff frequency was set to \(0.5k\), and the higher cutoff frequency was \(1.2k\), where \(k\) is determined by the center frequency of each Fourier frequency bin. Note that since the image resolution was not very high, the bandwidth of the bandpass filter was set wide in order to avoid removing the broadened components by low-resolution 2D FT. The lower cutoff frequency was determined carefully to avoid erasing too many components near the origin in the wavenumber spectrum.

## 4 Numerical results

### Denoising of white noise data

Table 1 and Fig. 3 show the evaluation metrics and denoised sound-field images of the conventional and proposed methods for the white-noise data. The table shows that Ours (W) scored the highest in terms of PSNR and SIIM for all \(N\). Among the conventional methods, the Gaussian filter had the highest PSNR for the overall score, and ST BPF had the highest SSIM. Figure 3 shows that the Gaussian filter smoothed the noisy wavefront, but it blurred short-wavelength

\begin{table}
\begin{tabular}{c c c c c c c c c c c c} \hline \hline  & \multicolumn{6}{c}{PSNR [dB]} & \multicolumn{6}{c}{SSIM} \\ \cline{2-13} \(N\) & Noisy & Gaussian & Median & ST BPF & Ours (W) & Ours (W+S) & Noisy & Gaussian & Median & ST BPF & Ours (W) & Ours (W+S) \\ \hline
1 & 3.78 & 18.1 & 17.9 & 16.7 & **30.7** & 11.5 & 0.158 & 0.627 & 0.554 & 0.774 & **0.961** & 0.433 \\
2 & 2.38 & 19.8 & 16.5 & 17.1 & **27.7** & 13.6 & 0.122 & 0.681 & 0.441 & 0.770 & **0.930** & 0.566 \\
3 & 0.74 & 19.2 & 14.9 & 17.9 & **24.1** & 13.4 & 0.131 & 0.633 & 0.386 & 0.759 & **0.844** & 0.539 \\
4 & -0.67 & 20.3 & 13.8 & 17.7 & **25.1** & 14.7 & 0.098 & 0.672 & 0.327 & 0.753 & **0.847** & 0.586 \\
5 & -1.92 & 19.4 & 12.5 & 17.1 & **24.0** & 15.2 & 0.092 & 0.640 & 0.299 & 0.712 & **0.796** & 0.571 \\ \hline All & 0.86 & 19.4 & 15.1 & 17.3 & **26.3** & 13.6 & 0.120 & 0.651 & 0.401 & 0.754 & **0.876** & 0.539 \\ \hline \hline \end{tabular}
\end{table}
Table 1: PSNR and SSIM of denoising results for white-noise data.

Figure 3: Examples of denoised images for white-noise data. Two examples are shown for each \(N\).
sound waves. The median filter performed worse than the other methods. The ST BPF showed good overall results, but wavefront distortion remained when the noise amplitude was large. Ours (W) produced better noise reduction results than the conventional methods did, regardless of the sound-field parameters, such as the number of sound sources and acoustic wavelength, and the amount of noise. Ours (W+S) seemed to properly restore the wavefronts; nevertheless, its scores were significantly lower than those of Ours (W). This is because the overall amplitude of the denoised sound field increased, as can be seen from the images.

Figure 4 plots the scores or each denoising method on the 500 evaluation data as a function of the wavenumber to investigate the dependence of the denoising performance on the wavenumber. The Gaussian filter performed well for the low wavenumbers, but its performance deteriorated as the wavenumber increased. The ST BPF had low scores for very low wavenumbers because the spatial frequency bandpass filter unintentionally eliminated the very low wavenumber components. The proposed method outperformed the conventional methods over the entire wavenumber range, although it showed a slight decrease in performance as the wavenumber increased. The performance of Ours (W+S) leveled off, probably due to the increase in the overall amplitude.

### Denoising of speckle noise data

Table 2 and Fig. 5 show the evaluation metrics and denoised sound-field images for the speckle noise data. All methods except Ours (W+S) scored lower compared with their white-noise results. Figure 5 indicates that the spatial correlations caused by speckles remained in the images of the conventional methods and Ours (W), resulting in hazy denoised images. By contrast, Ours (W+S) removed the local amplitude distortions in the noisy data. The scatter plots of the scores are shown in Fig. 6. The PSNRs of the conventional filters and Ours (W) leveled off around 20 dB for almost all wave numbers. Ours (W+S) scored higher than the conventional filters and Ours (W) regardless of the wavenumber. These results confirm that the network properly learned the nonlinear transformation caused by speckle noise from the created training dataset.

## 5 Experiments

We denoised experimental data measured by two optical systems: PPSI [1], in which the primary noise source was white noise, and HSI using Fresnel lenses [46], in which speckle noise was superimposed.

### Parallel phase-shifting interferometry

PPSI is a system that combines a Fizeau interferometer and a polarized high-speed camera, as shown in Fig. 7(a). It measures four phase-shifted interference fringe images simultaneously,

Figure 4: (a) PSNR and (b) SSIM plotted as a function of acoustic wavenumber for white-noise data. The size of the circular markers represents the amplitude of the white noise in four levels.
which enables instantaneous and quantitative observation of sound fields. For details of the measurement technique, see, for example, [1].

In this experiment, a 12-kHz burst wave generated from a loudspeaker (FOSTEX FT48D) was observed. The sound measured by a microphone placed 20 cm from the loudspeaker is shown in Fig. 7(b). The generated sound was a three-cycle 12 kHz burst wave with a peak sound pressure of 13 Pa at the microphone position. The frame rate of the high-speed camera was set to 50 kfps, the number of frames was 1000, the image resolution was 128 \(\times\) 128, and the imaging area size was 80 mm \(\times\) 80 mm. The optical phase map at each frame was calculated using a typical arctangent operation, followed by 1D unwrapping along the time direction for each pixel. Subsequently, a time-directional low-pass filter with a cutoff frequency of 500 Hz was applied to remove low-frequency noise components. We call this data the noisy data. The denoising was performed on the noisy data by using the same conventional filters and trained DNNs as in the previous section.

Figure 7(c) shows the time-series sound field images of the noisy and denoised data. In the noisy data, random noise and oblique noise patterns appeared in addition to sound waves propagating from the left outside of the image to the right. These oblique patterns should be

\begin{table}
\begin{tabular}{c c c c c c c c c c c c} \hline \hline  & \multicolumn{6}{c}{PSNR [dB]} & \multicolumn{6}{c}{SSIM} \\ \cline{2-11} \(N\) & Noisy & Gaussian & Median & ST BPF & Ours (W) & Ours (W+S) & Noisy & Gaussian & Median & ST BPF & Ours (W) & Ours (W+S) \\ \hline
1 & 1.70 & 12.0 & 11.0 & 12.2 & 14.0 & **25.7** & 0.322 & 0.379 & 0.336 & 0.477 & 0.589 & **0.916** \\
2 & 0.49 & 13.9 & 11.2 & 13.6 & 15.4 & **23.0** & 0.242 & 0.459 & 0.290 & 0.528 & 0.631 & **0.850** \\
3 & -0.67 & 14.6 & 10.8 & 14.6 & 15.8 & **20.1** & 0.215 & 0.453 & 0.264 & 0.542 & 0.559 & **0.749** \\
4 & -1.72 & 15.0 & 10.4 & 14.6 & 16.3 & **21.4** & 0.172 & 0.492 & 0.242 & 0.557 & 0.606 & **0.760** \\
5 & -2.84 & 14.9 & 9.59 & 14.4 & 16.2 & **20.4** & 0.186 & 0.469 & 0.223 & 0.527 & 0.566 & **0.675** \\ \hline All & -0.61 & 14.1 & 10.6 & 13.9 & 15.5 & **22.1** & 0.227 & 0.451 & 0.271 & 0.526 & 0.590 & **0.790** \\ \hline \hline \end{tabular}
\end{table}
Table 2: PSNR and SSIM of denoising results for data with white and speckle noises.

Figure 5: Examples of denoised images for the data with white and speckle noises. Two examples are shown for each \(N\).
phase shift errors caused by imperfections in the optical system. The Gaussian filter and ST BPF produced smooth wavefronts for the peak wavefront of the burst wave, but noisy components remained in the low-amplitude parts before and after the peak wavefront, as can be seen in the right half of the 60 \(\mu\)s image and left half of the 240 \(\mu\)s image. In contrast, Ours (W) restored both the peak and low-amplitude wavefronts smoothly. The wavefront buried in the noise data were also visualized in the results for Ours (W). For Ours (W+S), the restored sound wave amplitudes increased, which is consistent with the numerical results in Fig. 3.

### Holographic speckle interferometry with Fresnel lens

An overview of the measurement using HSI is shown in Fig. 8(a). This experiment used a measurement system with Fresnel lenses, as proposed in [46]. It was proposed to establishs a lightweight and inexpensive large-aperture sound-field imaging system using Fresnel lenses. However, the measured sound-field images showed significant spatial distortion due to speckle noise. In the original paper, narrow spatial bandpass filters were used for noise reduction, but such narrow filters may not be so useful for practical applications. Here, we investigated the

Figure 6: (a) PSNR and (b) SSIM plotted as a function of acoustic wavenumber for data with white and speckle noises. The size of the circular markers represents the amplitude of the white noise in four levels.

Figure 7: (a) Schematic diagram of the PPSI measurement system. A three-cycle burst wave with a center frequency of 12 kHz was emitted from the loudspeaker. (b) Sound pressure waveform measured by the microphone placed 20 cm from the loudspeaker’s diaphragm. (c) Denoising results of transient sound fields measured by PPSI.
effectiveness of the proposed DNN-based denoising method.

Sinusoidal signals of 5, 10, and 15 kHz were radiated from the same loudspeaker used in the PPSI experiment. The amplitudes were adjusted so that the sound pressure level at the microphone located 20 cm in front of the loudspeaker diaphragm was 110 dB at all frequencies. The frame rate of the high-speed camera was 50 kfps, the number of frames was 1000, the image resolution was \(128\times 128\), and the size of the captured area was 100 mm \(\times\) 100 mm. The phase maps of the speckle interference fringes were estimated using the 2D FT method [47], and a complex sound field at each frequency was extracted via 1D FT along the time direction.

Figure 8(b) shows the real parts of the noisy and denoised complex amplitudes. The noisy data shows that the wavefronts were significantly distorted. The conventional methods and Ours (W) did not eliminate the distortions for all frequencies. In contrast, in the case of Ours (W+S), where speckle noise was included in the training data, it can be seen that smooth wavefronts were restored. Since the same loudspeaker as in the PPSI experiment was used, the harmonic wavefront should be smooth. Therefore, Ours (W+S) showed good noise reduction and wavefront restoration performance in speckle sound-field imaging.

## 6 Conclusions

We developed a DNN-based sound-field denoising method in which the trained network decomposes time-varying sound field data into 2D complex amplitude images and denoises each individual image. A 2D sound field simulation with random parameters was used to generate the training dataset. By taking into account the measurement process of the optical system, the network was successfully trained to remove not only white Gaussian noise but also speckle noise. We confirmed that the proposed method was effective on experimental data and that it outperformed conventional denoising methods.

There are questions to be tackled in future work. First, in this study, we employed only one architecture, NAFnet, with a fixed network size. Therefore, the effect of the choice of optical network architecture and its size should be investigated. Second, the simulation method and

Figure 8: (a) Schematic diagram of the HSI measurement system. The sound field between the two Fresnel lenses is measured. The sSinusoidal waves of 5, 10, and 15 kHz was are emitted from the loudspeaker. (b) Denoising results of harmonic sound fields measured by HSI.
the number of training data should also be investigated. The generalization abilities against the wavenumber range, complexity of sound fields, and amount and types of noise must depend on the training data set. Last but not least, it is important to extend the proposed method to different measurement situations, such as spatial 3D data, randomly sampled data, and data with occlusions to provide a versatile denoiser for optically measured sound field data.

Disclosures.The authors declare no conflicts of interest.

Data availability.Data underlying the results presented in this paper are not publicly available at this time but may be obtained from the authors upon reasonable request.
