Practical Radar Sensing Using Two Stage Neural Network for Denoising
  OTFS Signals
Our objective is to derive the range and velocity of multiple targets from
the delay-Doppler domain for radar sensing using orthogonal time frequency
space (OTFS) signaling. Noise contamination affects the performance of OTFS
signals in real-world environments, making radar sensing challenging. This work
introduces a two-stage approach to tackle this issue. In the first stage, we
use a generative adversarial network to denoise the corrupted OTFS samples,
significantly improving the data quality. Following this, the denoised signals
are passed to a convolutional neural network model to predict the values of the
velocities and ranges of multiple targets. The proposed two-stage approach can
predict the range and velocity of multiple targets, even in very low
signal-to-noise ratio scenarios, with high accuracy and outperforms existing
methods.
Ashok S Kumar, Sheetal Kalyani
# Practical Radar Sensing Using Two Stage Neural Network for Denoising OTFS Signals

###### Abstract

Noise contamination affects the performance of orthogonal time frequency space (OTFS) signals in real-world environments, making radar sensing challenging. Our objective is to derive the range and velocity from the delay-Doppler (DD) domain for radar sensing by using OTFS signaling. This work introduces a two-stage approach to tackle this issue. In the first stage, we use a convolutional neural network (CNN) model to classify the noise levels as moderate or severe. Subsequently, if the noise level is severe, the OTFS samples are denoised using a generative adversarial network (GAN). The proposed approach achieves notable levels of accuracy in the classification of noisy signals and mean absolute error (MAE) for the entire system even in low signal-to-noise ratio (SNR) scenarios.

 OTFS, delay-Doppler domain, convolutional neural network, generative adversarial network.

## I Introduction

Orthogonal time frequency space (OTFS) signaling has been identified as a promising signal waveform for fully using the capabilities of the integrated sensing and communication (ISAC) system [1, 2]. The OTFS signal modulates the data in the delay-Doppler (DD) domain. The target's range and velocity characteristics, which may be derived from the DD domain, are the essential parameters to be calculated in radar signal processing.

A 2D correlation-based approach to evaluate the Doppler and delay indices for radar sensing has been studied in [3]. The advantages of using OTFS waveform for velocity and range estimates in radar sensing applications have been investigated in [4, 5] and [6]. A single target scenario has been considered in [4], in which the root mean square error (RMSE) performance of the range and velocity as a function of signal-to-noise ratio (SNR) up to -15 dB has been analyzed for radar target estimation. The work in [5] mentions the estimation of range and velocity RMSE as a function of radar SNR in a multipath scenario. The work reported in [7] exploited the application of three distinct sparse algorithms in the estimation of range and velocity of target using OTFS signaling. All the studies mentioned above examine the range and velocity RMSE of the target by considering fixed SNR levels. Unlike the existing state- of-the-art methodologies, we calculated the range and velocity RMSE of each target by using a two-stage noise reduction method for OTFS signals.

For the first stage, a convolutional neural network (CNN) model is proposed in our work for the classification of noise as severe or moderate [8]. The proposed CNN model effectively classifies the noise without incorporating any preprocessing steps for noise removal. In wireless communication systems, deep neural networks (DNNs) significantly improve data interpretation, leading to better signal quality and system performance [9, 10, 11]. The GAN-based denoising models have several advantages over traditional denoising methods. The model can rival the performance of artificial denoising methods. The advantages include better performance, enhanced generalization capabilities, the ability to study complex patterns in the data, and automation of the entire process [12, 13, 14]. The aforementioned advantages serve as the driving force behind the incorporation of GAN into the second stage of our proposed method for denoising.

In summary, this letter presents a two-stage neural network model comprised of CNN and GAN with the goal of estimating the range and velocity for radar target detection by means of OTFS signaling. In real-world radar applications, the severity of noise in the signal is unpredictable, which makes radar target detection difficult. The existing literature falls short of providing a system dealing with radar target estimation in extremely noisy conditions. Simulation results show that our system has the capability to reduce the values of mean absolute error (MAE), range RMSE, and velocity RMSE of the target significantly. The system can even operate in extremely noisy conditions ranging from 0 to -20 dB SNR, thus expanding the SNR range for radar target detection.

## II System Model

In this section, we first describe the OTFS based system.

### _Otfs_

For each OTFS frame, \(N\) and \(M\) represent the number of time slots and the number of sub-carriers respectively. \(T\) represents the symbol duration and \(\delta f_{s}\) is the subcarrier frequency spacing. For a given \(\delta f_{s}\), the total bandwidth is \(B=M\delta f_{s}\) and the duration of one OTFS frame is given by \(NT\). The information bits are mapped to a symbol set in the DD domain. The information symbols are generally QAM symbols. The symbol set corresponding to \(l^{th}\) delay and \(k^{th}\) doppler bins is \(A_{\mathrm{DD}}[k,l]\), for \(k=0,\ldots,N-1\) and \(l=0,\ldots,M-1\).

The DD domain symbols are mapped to time-frequency (TF) domain symbols using inverse symplectic finite Fourier transform (ISFFT), operation as

\[A_{\mathrm{TF}}[n,m]=\frac{1}{\sqrt{NM}}\sum_{k=0}^{N-1}\sum_{l=0}^{M-1}A_{ \mathrm{DD}}[k,l]e^{j2\pi\left(\frac{nk}{M}-\frac{ml}{M}\right)} \tag{1}\]
where \(n=0,\ldots,N-1\) and \(m=0,\ldots,M-1\). The TF symbols are translated to the time domain transmit signal, \(x(t)\) by using the Heisenberg transform,

\[x(t)=\sum_{n=0}^{N-1}\sum_{m=0}^{M-1}A_{\mathrm{TF}}[n,m]g_{tx}(t-nT)e^{j2\pi m \delta f_{x}(t-nT)} \tag{2}\]

where \(g_{tx}\) is a pulse-shaping waveform.

The time domain signal \(x(t)\) is passed through the linear time-varying channel, which has \(P\) targets in the DD domain. The \(p^{th}\) target has reflection coefficient \(h_{p}\), delay \(\tau_{p}\) with \(0<\tau_{p}<T\) and Doppler shift \(\nu_{p}\). The complex base-band channel response, \(h(\tau,\nu)\) in the DD domain can be expressed as

\[h(\tau,\nu)=\sum_{p=0}^{P-1}h_{p}\delta\left(\tau-\tau_{p}\right)\delta\left( \nu-\nu_{p}\right) \tag{3}\]

For integer delays and Doppler, \(\tau_{p}=\frac{l_{p}}{M\delta f_{x}}\) and \(\nu_{p}=\frac{k_{p}}{NT}\), where \(l_{p}\) and \(k_{p}\) denote the corresponding delay and Doppler indices of the \(p^{th}\) target. The received signal \(r(t)\) is given by

\[r(t)=\iint h(\tau,\nu)e^{j2\pi\nu(t-\tau)}x(t-\tau)d\tau d\nu+w(t) \tag{4}\]

where \(w(t)\) denotes the additive white Gaussian noise (AWGN) process with one side power spectral density (PSD), \(N_{0}\). The received signal \(r(t)\) is converted back to the TF domain using Wigner transform,

\[B_{\mathrm{TF}}[n,m]=\int_{-\infty}^{\infty}r(t)g_{rx}^{*}(t-nT)e^{-j2\pi m \delta f_{x}(t-nT)}dt \tag{5}\]

where \(g_{rx}(t)\) is the pulse-shaping filter at the receiver. The TF domain signals \(B_{\mathrm{TF}}[n,m]\) are then converted to DD domain symbols \(B_{\mathrm{DD}}[k,l]\) using symplectic finite Fourier transform (SFFT), which is given by,

\[B_{\mathrm{DD}}[k,l]=\frac{1}{\sqrt{NM}}\sum_{n=0}^{N-1}\sum_{m=0}^{M-1}B_{ \mathrm{TF}}[n,m]e^{-j2\pi\left(\frac{mk}{N}-\frac{m}{N}\right)} \tag{6}\]

In view of the fact that \(B_{\mathrm{DD}}\) contains information symbols, we are not able to identify the target areas of interest directly. Instead, a 2D correlation-based approach has been used between \(B_{\mathrm{DD}}\) and \(A_{\mathrm{DD}}\) to obtain the delay and Doppler index [3]. The matrix \(V\) contains information about the correlation between the transmitted and received signals at different delay and Doppler indices. The accumulated correlation coefficient under different delay and Doppler indices is given by,

\[\begin{split} V[k,l]=\sum_{n=0}^{N-1}\sum_{m=0}^{M-1}B_{ \mathrm{DD}}^{*}[n,m]A_{\mathrm{DD}}\left[[n-k]_{N},[m-l]_{M}\right]\\ \times\gamma[n-k,m-l]e^{j2\pi\frac{(m-1)k}{NM}},\end{split} \tag{7}\]

where \(k\in[0,N-1]\) and \(l\in[0,M-1]\), and \(\gamma[k,l]\) is a phase offset given by

\[\gamma[k,l]=\begin{cases}1,&l\geq 0,\\ e^{-j2\pi\frac{k}{N}},&l<0.\end{cases} \tag{8}\]

### _Dataset Description_

We describe the following datasets which we use to train the proposed deep learning model.

* **Transmitted dataset:** The transmitted dataset contains the transmitted OTFS signals \(x(t)\) that are generated by the transmitter and sent out into the environment. This signal is used to probe the environment and detect the objects or targets that reflect the signal to the receiver.
* **Low Noise dataset:** The low noise dataset has the signal \(r(t)\), which is obtained by using the equation (4). Typically, deep learning applications operate under the assumption of completely clean dataset with no noise. However, in practical scenarios, such datasets are rarely available. Hence we use low noisy datasets in our work. We have separately created datasets with 5 dB, 20 dB, and 40 dB SNR values. This is for comparing MAE, range RMSE, and velocity RMSE of the target at different SNR values. These datasets are used as the input to the GAN only during the training phase
* **Corrupted/ Noisy dataset:** The corrupted dataset has the signals \(r(t)\), after it has been corrupted with AWGN noise, where SNR ranges from 0 to -20 dB.
* **Label:** The estimated target location is obtained in the DD domain by correlating the corrupted signal with the transmitted signal. This is obtained by using the equation (7). The true target location indicates the actual target location. The estimated target value is compared with the true target value to generate the labels. The label '0' denotes a match between the estimated and true target values which indicates moderate noise. The label '1' denotes a mismatch between the estimated and true target values which correspond to severe noise.

Fig. 1. shows the DD matrices for radar sensing after performing the 2D correlation between moderately corrupted signal and transmitted signal in the DD domain. In this example, one target is considered with \(M=N=28\). The delay and Doppler indices of the targets are 7 and 12 respectively. The same scenario has been considered in Fig. 2. by performing the 2D correlation between the severely corrupted signal and the transmitted signal in the DD domain. It is seen that we cannot identify the location of the target exactly from the DD domain matrix.

### _Proposed CNN for classification of noise_

In the proposed system, the input to the CNN consists of transmitted dataset and corrupted dataset. The proposed CNN architecture which is used to classify noise as moderate or severe is shown in Fig. 3. The network starts with an image input layer. This layer is succeeded by a convolution layer with 32 filters of size \(13\times 13\). The padding is set to be the same. The Batch normalization layer is inserted between the layers which is then followed by the ReLU layer. The CNN then continues with a series of similar layers alternating between convolutional layers, batch normalization layers, and ReLU layers. The filter size also gradually increases from 32 and doubles up to 256 in the subsequent layers. A dropout of value 0.5 is also added after this layer. Two fully connected layers of
512 nodes and one fully connected layer of 256 nodes follow the convolution layer. In the proposed model, the activation function used at the output layer is the softmax layer. The final classification layer then computes the loss and accuracy of the network during training and evaluation. In this case, the loss function employed is the cross-entropy loss. The CNN architecture classifies the noise as moderate or severe. If the noise is moderate, then by using equation (7), the location of the target can be identified. Consequently, we can derive the velocity and range of the target from the DD domain. On the other hand, if the CNN outputs severe noise as the classification, all the corresponding samples are aggregated and passed through the GAN for denoising.

### _Denoising OTFS Signals using GAN_

GANs use two neural networks called a Generator Network \(G\) and a Discriminator Network \(D\), which compete against each other to create the desired result. The inputs to the discriminator and generator are real data \(u\) and random variable \(w\) respectively. The discriminator gives a value \(D(u)\) suggesting the possibility that \(u\) is a real sample. The main purpose of the discriminator is to maximize the probability of labeling the real samples as real and the generated fake samples \(G(w)\) as fake. The objective of the generator is to produce fake samples \(G(w)\) which are as close as possible to that of real samples, so that the discriminator fails to separate between fake and real samples. Hence a GAN can be defined as a minimax game in which \(G\) wants to minimize the value function \(\tilde{V}(D,G):\) while \(D\) wants to maximize it.

\[\min_{G}\max_{D}\tilde{V}(D,G)=\mathbb{E}[\log D(u)]+\mathbb{E}[\log(1-D(G(w)))] \tag{9}\]

The GAN is trained using pairs of low noise and corrupted OTFS signals. Fig. 4. shows the block diagram of GAN for denoising OTFS signals. The generator network is trained to generate denoised signals from the corrupted signals. The input to the discriminator network is low noise signals and generated signals from the generator. During training, the generator network attempts to minimize the difference between the generated and low noise signals, while the discriminator network aims to maximize the difference between the generated and low noise signals.

The discriminator network starts with an input layer followed by the convolution layer which has 32 filters of size (3,3) and stride (2,2). The padding is set to the same. The output of the convolution layer is then fed through a Leaky ReLU activation function. The same convolution layers are repeated with increasing filter sizes of 64, 128, 128, 256, and 512. A dropout regularisation layer with 0.5 is inserted at the output to prevent overfitting. The output layer is a fully connected layer (Dense) with a single unit and a sigmoid activation function. The corrupted signal is preprocessed before feeding to the Generator. The preprocessing is done

Fig. 1: The DD matrices for radar sensing after performing the 2D correlation between moderately corrupted signal and transmitted signal in the DD domain. The below plot corresponds to the 3D plot by taking the magnitude of \(V\) along the \(z-\)axis

Fig. 2: The DD matrices for radar sensing after performing the 2D correlation between severely corrupted signal and transmitted signal in the DD domain. The below plot corresponds to the 3D plot by taking the magnitude of \(V\) along the \(z-\)axis
by using Gaussian filtering and the preprocessed signal is normalized to a range of [0,1]. This is then passed through a series of two sets of convolution layers with 32 filters each with size (3,3), each succeeded by a max pool layer. This is repeated with two more batches of convolution layer and max pool layer with the filter size of 64 and 128. Each layer uses the ReLU activation function and the same padding. The upsampling path also consists of 3 sets of 2 convolution layers with filter sizes of 64, 32, and 3 respectively. The output layer produces the denoised signals after training. It represents the generated signals which are then fed to the discriminator.

## III Simulation Results

In the simulations, we consider a single target scenario with \(P=1\). A DD domain grid with \(M=N=28\) and \(\delta f_{s}\) = 150 kHz is considered. The carrier frequency \(f_{c}\) is taken as 60 GHz. The velocity and range resolution can be calculated by \(V_{res}=\frac{Bc}{2M\)\(M\)\(f_{c}\) and \(R_{res}=\frac{c}{2B}\), where \(c\) is the speed of light. The maximum velocity and range are given by \(V_{max}=\frac{c\delta f_{s}}{2f_{c}}\) and \(R_{max}=\frac{cT}{2}\). The dataset comprising 50000 complex OTFS samples, each with a length of \(MN\) are taken for low noise, corrupted, and transmitted signals. We have also created three separate datasets corresponding to low noise signals with different SNR levels of 5 dB, 20 dB, and 40 dB. Each dataset contains 50000 samples and is utilized for analyzing MAE, range RMSE, and velocity RMSE of the target. Out of 50000 samples, 40000 samples are used for training the system. The plot in Fig. 5. shows the variation in loss across the number of iterations for the CNN. Notably, the proposed CNN achieved a remarkable training accuracy of \(98.8\%\). Concurrently, the loss reduced progressively, reaching a low value of 0.05.

### _System Validation and Performance Analysis_

For testing the system, a total of 10000 transmitted and corrupted samples are given to CNN. The test accuracy for the CNN is found to be \(97.89\%\). If the CNN produces severely noisy results, associated samples are aggregated and sent through the GAN for denoising. Table 1. shows the comparative analysis of MAE, range RMSE, and velocity RMSE of the target by utilizing the samples of corrupted signals in the range 0 to -20 dB SNR along with low noise signals of different SNR values. In this work, we have achieved

Fig. 4: Block diagram of GAN for denoising

Fig. 3: CNN Architecture for classification of noise
a MAE value of 0.68 in the DD domain by using low noise signals with 40 dB SNR and corrupted signals. Thus, for a maximum distance of 1000 m and a maximum velocity of 375 m/s, we obtain a range RMSE and a velocity RMSE of 69.5 m and 26.06 m/s respectively. Similarly, the MAE, range RMSE, and velocity RMSE value of the target under the same scenario by using low noise signals with 20 dB and 5 dB SNR is depicted in the Table. 1.

### _Comparison with the state-of-the-art methods_

The works [5] and [6] address modified versions of maximum likelihood (ML) estimation algorithms for target detection using OTFS signaling. In [5] and [6], the maximum range RMSE (at an SNR of up to -15 dB) is determined to be 0.7 m and 12 m, respectively. Similarly, the maximum velocity RMSE at -15 dB SNR is found to be 3 m/s and 10 m/s, respectively. The work in [7] performs target sensing up to -10 dB by utilizing sparse algorithms and they could obtain both the maximum range RMSE and velocity RMSE as 1. These non-zero values of RMSE in [4]-[6] indicate inaccurate target localization in radar sensing. In the SNR range from 0 to -15 dB, our approach delivers highly accurate results, with both range RMSE and velocity RMSE effectively reduced to zero. The work in [6] and [7] does not address SNR levels below -15 dB. Due to our dual-stage technique, we are able to work even with SNR as low as -20 dB. Despite [5] addressing SNR levels below -15 dB, which resulted in range RMSE and velocity RMSE values of 100 m and 120 m/s at -20 dB, our approach demonstrates superior performance with lower values as indicated in Table 1. It is evident that the values of MAE, range RMSE, and velocity RMSE are low when considering corrupted signals ranging from 0 to -20 dB SNR. Hence, radar target detection in the highly challenging, low SNR environments, is now possible due to our proposed approach.

## IV Conclusions

In this work, we have proposed a two-stage approach for denoising OTFS signals for radar sensing. The first step involves the classification of noisy OTFS samples as moderate or severe with the help of a CNN. The CNN accurately distinguishes the noisy samples and provides a solid foundation for the subsequent denoising process. The second step focuses on denoising the identified noisy samples with the help of a GAN. The proposed system has yielded promising results demonstrating its effectiveness in both the classification and denoising processes even in very low SNR environment.
