Paper Title: Fluctuating Line-of-Sight Fading Distribution: Statistical Characterization and Applications.







Fluctuating Line-of-Sight Fading Distribution: Statistical Characterization and Applications 


    
        Thanh Luan Nguyen1, 
        Georges Kaddoum1^,2
        
    
    1 Department of Electrical Engineering, École de Technologie Supérieure (ÉTS), Montréal, QC, Canada 

    2 Artificial Intelligence & Cyber Systems Research Center, Lebanese American University
    

      Emails: 
        1thanh-luan.nguyen.1@ens.etsmtl.ca,
        1^,2georges.kaddoum@etsmtl.ca, 

    August 5, 2024
===========================================================================================================================================================================================================================================================================================================================================================================================



§ ABSTRACT 
We introduce the fluctuating Line-of-Sight (fLoS) fading model, characterized by parameters K, k, λ, and Ω. 
    The fLoS fading distribution is expressed in terms of the multivariate confluent hypergeometric functions Ψ_2, Φ_3^(n), and Φ_3 = Φ_3^(2) and encompasses well-known distributions, such as the Nakagami-m, Hoyt, Rice, and Rician shadowed fading distributions as special cases. An efficient method to numerically compute the fLoS fading distribution is also addressed.
Notably, for a positive integer k, the fLoS fading distribution simplifies to a finite mixture of κ-μ distributions.
    Additionally, we analyze the outage probability and Ergodic capacity, presenting a tailored Prony's approximation method for the latter. 
Numerical results are presented to show the impact of the fading parameters and verify the accuracy of the proposed approximation. Moreover, we illustrate an application of the proposed fLoS fading distribution for characterizing wireless systems affected by channel aging.



κ-μ fading, confluent hypergeometric functions, outage probability, channel capacity, Prony's method




§ INTRODUCTION









The advent of sixth-generation (6G) wireless networks is expected to bring forth various applications for ubiquitous, high-speed connectivity <cit.>. 
    To name a few, the use reconfigurable intelligent surfaces (RIS) can configure radio environments for enhanced performance. Moreover, high-mobility communication (HMC) ensures broadband services for users traveling at speeds of up to 1000 km/h. 
In addition, the utilization of terahertz (THz) bands, ranging from 0.1 THz to 10 THz, offer transmission rates reaching multiple terabits per second. 
    However, despite these advancements, significant challenges persist within each of these technologies. 
For instance, RIS-aided communication requires compelling applications that demonstrate substantial performance gains over existing technologies like mmWave and massive multiple-input-multiple-output (mMIMO) systems <cit.>.
    HMC faces issues like channel aging, where the system relies on outdated channel state information (CSI) for information processing <cit.>. 
THz communications encounter blockages and lack convincing theoretical frameworks and practical channel models <cit.>. 
    Therefore, to fully realize the potential of these emerging technologies and address the associated challenges, the integration of novel fading channel models that accurately simulate real-world 6G environments is essential. 

The literature presents numerous generalized fading models that effectively depict the statistical behavior of mobile radio signals. 
    Various distributions, including the Rayleigh, Rice, Nakagami-m, Hoyt (Nakagami-q), and κ-μ distributions, describe short-term signal fluctuations <cit.>. 
Long-term variations, usually referred to as shadowing, are represented by the lognormal distribution, assuming independent attenuation from obstacles <cit.>. 
    However, practical scenarios introduce dependencies among obstacle attenuations due to phenomena like diffraction and refraction, challenging the independence assumption. 
While the lognormal model is widely accepted, alternative approaches like the Nakagami-m distribution provide tractable substitute <cit.>. 
    Moreover, in the context of dynamic environments, such as in vehicle-to-vehicle (V2V) communications, Gaussian, Rayleigh, and Rician distributions are favored for their ability to account for non-Gaussian shadowing with non-zero skewness in the dB scale <cit.>. 
Despite these models, significant theoretical gaps persist in characterizing generalized shadowing and other Line-of-Sight (LoS) variations due to various Radio Frequency (RF) propagation phenomena, leading to the need of comprehensive models for capturing real-world signal propagation phenomena.

This paper seeks to address these gaps by providing a novel approach for characterizing the fluctuations of the dominant specular component, i.e., the LoS component, using generalized distributions. Specifically, we propose a novel fluctuating LoS (fLoS) fading distribution, where LoS fluctuations are modeled by a noncentral chi-square distribution. This fLoS fading distribution encompasses classical models, like the Hoyt, Rice, Nakagami-m, Rician shadowed, and mixture of κ-μ distributions, as special cases.
    Our main contributions are summarized as follows[Our results are reproducible with MATLAB code, uploaded at https://github.com/thanhluannguyen/fluctuatingLoS]:


    
  * We introduce a novel probability density function (PDF) and cumulative distribution (CDF) for the proposed fLoS fading characterized by 4 parameters K, k, λ, and Ω.
    
  * We offer numerical methods for tackling the derived distributions due to their high computational complexity. 
        Notably, for integer k, the fLoS fading distribution can be expressed as a mixture of κ-μ distributions.
    
  * We derive the outage probability (OP) and Ergodic capacity (EC) by introducing a Prony's method to approximate various functions and compute the EC for wireless systems.
    
  * We present a use case for the fLoS fading distribution and we analyze the wireless system under the influence of channel aging. 
    






The paper is organized as follows: Section <ref> introduces the physical model of the fLoS fading channel. Subsequent analysis of the fLoS fading distribution, including the PDF, CDF, moments, and moment generating function (MGF), is presented in Section <ref>. Section <ref> presents the performance analysis and a potential application for the fLoS fading distribution. Section <ref> details the numerical results. The paper is concluded with a summary of findings in Section <ref>.



§ PHYSICAL MODEL



We consider the first-order scattering model, introduced in <cit.>, to present 
the received signal under the fLoS fading model with LoS fluctuations as follows:


    S = ω_0 ξ e^jϕ_0 + σ G,


where ω_0 e^jϕ_0 is the LoS (specular) component with constant amplitude ω_0, ϕ_0 is a random variable (RV) uniformly distributed in [0, 2π). The term σ G, where G ∼ CN(0,1), represents the diffuse scattering, or non-LoS component, with total power σ^2. 
    Here, K = |ω_0|^2/σ^2∈ [0, ∞), such that ω_0 = √(K)/√(K+1) and σ = 1/√(K+1), is the ratio between the power of the dominant LoS component and the average power of the scattering components.

In various researches, the component ξ is often assumed as the lognormal shadowing, or normal (Gaussian) shadowing in decibels. 
    This model assumes independent attenuations from independent obstacles, i.e., scatterers and reflectors, within local clusters, common in heavy shadowing environments. 
Specifically, if each obstacle (reflector or scatterer) independently attenuates the signal by A_m along a path, the total attenuation due to M independent obstacles is ξ = ∏_m^M A_m or ξ|dB = ∑_m^M ξ_m|dB in dB. With sufficiently large M, the total attenuation (in dB) follows a Gaussian distribution due to the Central Limit Theorem (CLT) regardless of the distribution of A_m distribution <cit.>. 
    Yet, in reality, the obstacles' attenuations are often correlated due diffraction and receiver body vibrations. In high frequency bands (e.g., mmWave, THz), molecular absorption loss significantly influences obstacle attenuation <cit.>. Consequently, the characterization of the shadowing distribution is useful when M is sufficiently small to prevent the probability of ξ=0 approaching one. However, when M is small, the CLT cannot be applied to characterize ξ with a lognormal distribution. Despite the widely accepted lognormal assumption, there exist other tractable shadowing models <cit.>. 
    In this manner, we consider ξ^2 follows a (scaled) noncentral chi-squared distribution with 2k degrees of freedom (d.o.f.), 2λ noncentrality parameter, and scale factor Ω/2, denoted as ξ^2 ∼χ_k^2(Ω; λ).
The PDF of ξ^2 is:


    f_ξ(ξ) = e^-x/Ω-λ/Ω(
            x/λΩ)^k-1/2
        I_k-1(
            2 √(λ x/Ω)), x > 0.


where I_v(x) is the vth order modified Bessel function of the first kind. 
It is noted that for Ω = 1/k+λ with λ→ 0, (<ref>) reduces to the Gamma distribution with unit mean and shape factor k, as f_ξ(x) = k^k/Γ(k) x^k-1 e^-k x, x > 0, and the model (<ref>) is simplified to the well-known Rician shadowed fading, where k depicts the shadowing severity <cit.>.
    We must note here that while the term "shadowing" is used, the fLoS model also includes variations in the direct LoS path due to other factors, such as movement or dynamic environments.


The noncentral chi-square distribution is closely related to the κ-μ fading power distribution with parameters (; κ, μ). 
    These two distributions can be deduced from one another by substituting suitable parameters. 
Specifically, let F_κμ(γ̅; κ, μ) be a RV following the κ-μ fading power distribution, we have the following relations:


    χ_k^2(Ω; λ)
           =^d  F_κμ(Ω(k+λ); λ / k, k) , 
     F_κμ(; κ, μ)
           =^d χ_μ^2(γ̅ [μ (1+κ)]^-1; μκ).















§ STATISTICAL CHARACTERIZATION



In this section,  we derive exact analytical formulas for the Moment Generating Function (MGF), the PDF, the CDF, and raw moments of the instantaneous SNR under fLoS fading conditions.
    For normalization purposes, let Ω = 1/k+λ, so that {|S|^2} = 1 and =  |S|^2, where  is the average SNR.



 §.§ Probability Functions




Considering ∼ F_(; K; k, λ, Ω) with non-negative real values K, k, λ, and Ω, to characterize the SNR under fLoS fading, where = {} denotes the average SNR. Then, its MGF is obtained as:


    M_(s) 
           =  1/1 - σ^2  s M_ξ^2( 
            |ω_0|^2  s/1- σ^2  s) 
       =  [1-σ^2  s]^k-1exp(
            λΩ |ω_0|^2  s /1-(σ^2+Ω |ω_0|^2)  s)/[1-(σ^2+Ω |ω_0|^2)  s]^k
        ,


for s < (σ^2 +Ω |ω_0|^2)^-1.


    We can characterize the SNR based on (<ref>) as:


    |ξ   ∼|  CN( ω_0 ξ e^jϕ_0, σ^2 ) |^2 
    
       ∼σ^2 /2[
                 N^2( √(2)ω_0 σ^-1ξcosϕ_0, 1 )
    
               
            +  N^2( √(2)ω_0 σ^-1ξsinϕ_0, 1 )
        ] 
       ∼^(a)χ_1^2(
            σ^2 , |ω_0|^2 σ^-2ξ^2
        ),


where (a) is due to the fact that the sum of two statistically independent squared normal (Gaussian) RVs with unit variances follows the noncentral chi-square distribution. 
Hence, conditioned on ξ, the SNR follows the noncentral chi-square distribution with 2 d.o.f., noncentrality parameter 2 |ω_0|^2/σ^2ξ^2, and scale factor σ^2/2.
    The conditional MGF of  on ξ is:


    𝐌_|ξ(s|ξ) = 1/1 - σ^2  sexp( |ω_0|^2  s/1- σ^2  sξ^2),


for σ^2 s < 1. The marginal MGF of the SNR is obtained by taking the expectation of 𝐌_|ξ(s|ξ) over ξ, which results in (<ref>). 
    Finally, applying _ξ^2(s) = 1/(1-Ω s)^k e^Ωλ s/1-Ω s yields (<ref>), which completes the proof of Lemma <ref>.



Considering ∼ F_(; K; k, λ, Ω) with non-negative real values K, k, λ, and Ω, where = {} denotes the average SNR. Then, the nth moment of  is


    {^n }
        =   n! (σ^2 )^n ∑_i=0^nni( Ω |ω_0|^2/σ^2)^i
        L_i^k-1(-λ),


where L_n^α(x) denotes the generalized Laguerre polynomial.




Considering ∼ F_(; K; k, λ, Ω) with non-negative real values K, k, λ, and Ω, where = {} denotes the average SNR. Then, the PDF of  is:


    f_(x)    =
        1/σ^2  [1+Ω|ω_0|^2 σ^-2]^k exp(
            -   x/σ^2  - λ)
        
         ×Ψ_2(
            k; λ/1+Ω |ω_0|^2/σ^2, Ω |ω_0|^2/σ^2+Ω |ω_0|^2x/σ^2 ),


for x > 0, where Ψ_2(a; w, z) ≜Ψ_2(a; 1, a; z, w) with Ψ_2(·) being the confluent hypergeometric function of two variables <cit.>.



The PDF of  is obtained by taking the inverse Laplace transform of M_(-s) from to x-domain as:


    f_(x) 
           = 𝐋^-1{ M_(-s); s, x }
       = exp( -x ^-1/σ^2 + Ω|ω_0|^2)
    
           ×𝐋^-1{ M_( -s + ^-1/σ^2 + Ω|ω_0|^2); s, x },


where the last equality is due to the frequency shifting property of the Laplace transform. After that, utilizing <cit.> and <cit.>, we get:


    𝐋^-1{(a+s)^k-1/s^k e^a b/s; s, x
        }   = Φ_3(1-k; 1; -a x, a b x) 
       = e^-ax-bΨ_2(k; b, a x),


and after some mathematical manipulations, we obtain (<ref>). This completes the proof of Theorem <ref>.




Considering ∼ F_(; K; k, λ, Ω) with non-negative real values K, k, λ, and Ω, where = {} denotes the average SNR. Then, the CDF of  is:





















    F_(x)    =
        x/σ^2 
        A^k
        exp(
            -   A/σ^2  x - Bλ)
        
         ×Φ_3^(3)(
            1, 1-k; 2; A/σ^2  x,
            - B/σ^2 x, 
            A B λ/σ^2  x
        ),


for x > 0, where A ≜σ^2/σ^2+Ω |ω_0|^2, B ≜Ω |ω_0|^2/σ^2+Ω |ω_0|^2, and Φ_3^(n)(·) is the n-variate confluent hypergeometric function <cit.>.



The CDF of  is obtained by taking the inverse Laplace transform of s^-1 M_(-s) as:


    F_(x) 
           = 𝐋^-1{ s^-1 M_(-s); s, x }
       = exp( -x ^-1/σ^2 + Ω|ω_0|^2)
    
           ×𝐋^-1{1/s M_( -s + ^-1/σ^2 + Ω|ω_0|^2)/1-^-1/σ^2 + Ω|ω_0|^21/s; s, x }.



Note that Φ_3^(n)(·) is expressed as <cit.>:


    Φ_3^(n)(b_1, …, b_n-1; c; d_1 t, …, d_n t)
    
           
    =   lim_b_n →∞Φ_2^(n)(b_1, …, b_n; c; d_1 t, …, d_n-1 t, d_n b_n^-1 t).



Hence, using the Laplace transform of Φ_2^(n)(·) in <cit.>, the Laplace transform of Φ_3^(n)(·) is obtained as:


    L{t^c-1/Γ(c)Φ_3^(n)(b_1, …, b_n-1; c; d_1 t, …, d_n t) }
           
    =   lim_b_n →∞1/s^c( 1-d_n/s1/b_n)^-b_n∏_i=1^n-1( 1-d_i/s)^-b_i
           
    =   1/s^c e^d_n/s∏_i=1^n-1( 1-d_i/s)^-b_i.



Upon applying the above identity, we obtain (<ref>). This completes the proof of Theorem <ref>.




The expressions in (<ref>), (<ref>), and (<ref>) are novel, and their accuracy is validated through comprehensive Monte Carlo simulations. 
    However, the newly proposed fading model's expressions are quite challenging from a computational perspective as compared to the κ-μ fading case. 
In section <ref>, we introduce an efficient method to numerically compute these functions.
    The relationship of the fLoS fading distribution with other well-known distribution is summarized in Table <ref>, where Ω = 1/k+λ.



 §.§ Integer value of d.o.f. parameter, k

When the parameter k in the fLoS model is a positive integer, the PDF expression is simplified. Specifically, using the property of the Φ_3(·) function in <cit.>, we can derive a more tractable expression for the PDF, as detailed in the subsequent corollary.



Considering ∼ F_(; K; k, λ, Ω) with non-negative real values K, λ, Ω, and k ∈ℤ^+ is a positive integer, where = {} denotes the average SNR.
Then, the PDF of  is:
























    f_(x) 
           =   A^k/σ^2exp( - A x/σ^2  - Bλ) 
        ∑_j=0^k-1(-1)^j/j!  (1-k)_j
        
         ×( B/Aλx/σ^2 )^j/2
            I_j( 2 √( A B λx/σ^2 )), x > 0.






We observe that the PDF is expressed as a finite mixture distribution, greatly simplifying the computational complexity of the fLoS fading distribution. Examining Corollary <ref> and utilizing the noncentral chi-square distribution definition in (<ref>) and its relation to the κ-μ fading model, we observe the fLoS fading distribution as a finite mixture of κ-μ fading power distributions as:


    f_(x)    = _j=0^k-1 C_j 
        f_κμ(; κ, μ; x), x > 0. 
    
    F_(x)    = _j=0^k-1 C_j 
        F_κμ(; κ, μ; x), x > 0.


where  := ( Bλ + j + 1 ) A σ^2,
    μ := j + 1,
    κ := Bλ/j+1,
    C_j ≜(1-k)_j/j! (-B/A)^j A^k with ∑_j=0^k-1C_j = 1.








Here, f_κμ(; κ, μ; x) and F_κμ(; κ, μ; x) are the PDF and CDF of the κ-μ fading power as introduced in <cit.> and <cit.>, respectively.




Due to the finite mixture of κ-μ distributions, performance metrics for the fLoS fading channel (with integer k) can be readily derived from existing results for the κ-μ fading case, without resorting to further mathematical manipulations.


§ PERFORMANCE ANALYSIS AND APPLICATION




 §.§ Outage Probability

The outage probability (OP), P_ out, indicates the probability that the instantaneous SNR drops below a set threshold, _ th, given by P_ out = { < _ th} <cit.>. 
    Consequently, the OP under fLoS fading is directly obtained by computing the CDF of the SNR, provided in (<ref>), as P_ out = F_(_ th).
In this context, when →∞ (high transmission SNR), we neglect Φ_3^(3)(·) in (<ref>) to obtain the asymptotic OP as:


    P_ out|_⇑
        = A^k e^-Bλ_ th (σ^2 )^-1.


A close observation of (<ref>) indicates that the fLoS fading model's diversity order, which is the rate at which the OP decreases in the high-SNR regime, is d = 1.





 §.§ Unified Ergodic Capacity

The Ergodic capacity (EC) per unit bandwidth, in bps/Hz, is given by C   =  1/ln 2∫_0^∞ln(1+) f_()  d <cit.>.
    However, the equation is difficult to derive due to the SNR PDF, f_(), involving confluent hypergeometric functions.
To address this limitation, we propose a unified based method to approximate C [bps/Hz]. 
    Our method tailors the Prony's algorithm presented in <cit.> to approximate f(x) = ln(1+x) as:


    f(x) = 
        _s=1^S_k=1^M_s c_s,k e^-T_s,k x≜f̂(x), x ∈ [l_s, u_s].



The tailored Prony's approximation method is detailed in Algorithm <ref>.





















From (<ref>) and the fact that {e^s} = _(s), the EC per unit bandwidth is approximated as:


    C   = 1/ln2_s=1^S_k=1^M c_s,k[
            F(T_s,k, u_s) - F(T_s,k, l_s)
        ],


where F(T,x) = ∫_0^x e^-T s  f_(s)  d s, or <cit.>


    F(T,x) = 𝐋^-1{_( -T - p )/p ; p, x },


which is derived in a similar manner as F_(x) in 
(<ref>).

Observing that ln(1+x) →ln(x) as x →∞, we directly derive the asymptotic EC per unit bandwidth as:


    C|_⇑
        = _s=1^S_k=1^Mc_s,k e^T_s,k/ln 2[
            F(T_s,k, u_s) - F(T_s,k, l_s)
        ].





 §.§ Multiple-input single-output (MISO) systems under channel aging

The SNR distribution over fLoS fading channel finds application in many real-world scenarios.
Within these scenarios, the intended physical interpretation of the fLoS fading distribution may vary, yet the distribution itself remains useful in characterizing SNR distributions effectively. 









In downlink MISO systems, with a high-speed moving receiver Rx, channel aging poses a significant challenge, causing discrepancies between current and estimated channel state information (CSI) <cit.>. 
    This mismatch degrades the wireless system performance and reliability. 
Specifically, the true CSI 𝐡 conditioned on the estimated CSI 𝐡̂ is given by 𝐡 | 𝐡̂ = ρ𝐡+ ρ̅𝐳, where 𝐳∼ CN(0_N, 𝐈_N), with N being the number of transmit antennas, ρ being the temporal correlation coefficient, and ρ̅ = √(1-ρ^2) <cit.>. 
    The effective fading from transmitter to receiver, h_, under Maximal Ratio Transmission (MRT) beamforming, is h_|𝐡̂∼ CN(ρ‖𝐡̂‖, ρ̅^2), where ‖𝐡̂‖ is the L2-norm of the estimated CSI.
Assuming Rician fading: 𝐡̂∼𝒞𝒩( √(κ)/√(κ+1)ȟ, 𝐈_N/κ+1) ∈ℂ^N, where κ is the Rician factor and ȟ represent the LoS components. Hence, the SNR at Rx is ≜P/N_0 W L_ |h_|^2, where P [W] is the Rx's transmission power, N_0 [W/Hz] is the noise spectral density, W [H] is the channel bandwidth, and L_ denotes the effective path loss between Tx and Rx.
    Based on (<ref>) and noting that ‖𝐡̂‖^2 ∼χ_N^2(1/κ+1, Nκ), the true distribution of the received SNR, , is given by:


    ∼ F_( 
        ; ρ^2/ρ̅^2, N, κ N, [κ+1]^-1),


where ≜P L_/N_0 W. Thus, when Rx moves with high speed and causes channel aging, the effective channel's true diversity order between Tx and Rx remains 1 regardless the number of transmit antennas, as discussed in the previous section.



§ NUMERICAL RESULTS



Although the confluent hypergeometric functions Φ_3^(n)(·), Φ_3(·) = Φ_3^(2)(·), and Ψ_2(·) are not available in MATLAB nor in Mathematica, we can numerically compute them as:


    Φ_3^(n)(b_1, …, b_n-1; c; d_1 t, …, d_n t)
    
           = Γ(c)/t^c-1∫_L-jT^L+jT s^-ce^st+d_n/s/j2π∏_i=1^n-1( 1-d_i/s)^-b_i d s , 
       Ψ_2(a; d; a; y, xt)
        = e^y+xtΦ_3(d-a; d, a; -xt, x y t),


where L = max(d_1, …, d_n-1, 0)+ϵ with ϵ>0 and T≫ 0 are manually chosen to ensure the computation accuracy. For example, we set ϵ = 1/2 and T = 10^3 in all the simulations.



 §.§ Effect of fading parameters












We now demonstrate how the fading parameters of the fLoS fading model influence the distribution shape. Due to space constraints, we concentrate on the SNR PDF and use Ω = 1/k+λ to ensure {ξ^2} = 1 and {} =.

Fig. <ref> illustrates the changes in the PDF as the d.o.f. parameter varies. Increasing k reduces the probability for having very low SNR values by concentrating the SNR around its mean value . Additionally, higher values of k alleviate the fading severity, with k = 0 indicating the most severe fading conditions.

Fig. <ref> depicts the changes in the PDF as the noncentrality parameter λ varies. Similarly, increasing λ reduces the probability for very low SNR values by shifting the distribution to the right causing the peak to move away from zero, while the PDF is not concentrated around its mean as compared to increasing the d.o.f.. Smaller values of λ also result in distributions with heavier tails, reducing the probability for having low SNRs. 



 §.§ Performance Analysis



Fig. <ref> presents the OP as a function of the normalized average SNR, /_ th, with different instances of the noncentrality parameter, λ. Analytical and asymptotic results for the OP are obtained by applying (<ref>) and (<ref>), respectively. 
    Simulation results are presented validate the expressions under the current settings. 
We observe that the OP decreases as λ is increased, especially with increased K, and that the asymptotic OP results are highly accurate for sufficiently large /_ th.





Fig. <ref> depicts the EC considering various values for k. Simulation results (markers) verify the accuracy of (<ref>) and (<ref>). Moreover, the capacity of the AWGN case is included as a benchmark, as it always upper bounds the capacity of a fading channel due to Jensen’s inequality. As observed, the performance gap with respect to the AWGN case is reduced as k is increased.



 §.§ Application






Fig. <ref> depicts the coverage probability, P_ cov = 1 - P_ out, versus the time instant index n for varying normalized Doppler shifts, where the data transmission starts at n = 0.
    The downward trend illustrates a decreasing coverage probability with increasing n. Additionally, as f_d T_s increases, the first zero position shifts leftward, indicating a reduction in the coherence time. 
The lower limit, e^- / _ th, signifies an effective fading channel behavior similar to Rayleigh fading, one of the most unfavorable fading scenarios.



§ CONCLUSION



We introduced the new fLoS fading model, motivated primarily by its ability to encompass the lognormal shadowing and shadowed Rician fading models. This fLoS fading distribution simplifies into a finite combination of the more straightforward κ-μ distribution when k is a positive integer. The performance analysis of wireless communication systems under the influence of the proposed fLoS fading model was examined using the OP and EC, where a unified Prony's method is proposed to approximate the EC. Finally, we presented an application for the fLoS fading distribution in characterizing wireless systems under the influence of channel aging.

IEEEtran




 Paper Title: Machine Learning In-Sensors: Computation-enabled Intelligent Sensors For Next Generation of IoT.














Machine Learning In-Sensors: Computation-enabled Intelligent Sensors For Next Generation of IoT  






    
    Andrea Ronco, Lukas Schulthess, David Zehnder, Michele Magno
    Dep. Information Technology and Electrical Engineering 

    ETH Zürich

    andrea.ronco@pbl.ee.ethz.ch
    
  

    August 5, 2024
===========================================================================================================================================================================================



§ ABSTRACT 



Smart sensors are an emerging technology that allows combining the data acquisition with the elaboration directly on the Edge device, very close to the sensors.





To push this concept to the extreme, technology companies are proposing a new generation of sensors allowing to move the intelligence from the edge host device, typically a microcontroller, directly to the ultra-low-power sensor itself, in order to further reduce the miniaturization, cost and energy efficiency.

This paper evaluates the capabilities of a novel and promising solution from STMicroelectronics. The presence of a floating point unit and an accelerator for binary neural networks provide capabilities for in-sensor feature extraction and machine learning.
We propose a comparison of full-precision and binary neural networks for activity recognition with accelerometer data generated by the sensor itself.

Experimental results have demonstrated that the sensor can achieve an inference performance of 10.7 cycles/MAC, comparable to a Cortex-M4-based microcontroller, with full-precision networks, and up to 1.5 cycles/MAC with large binary models for low latency inference, with an average energy consumption of only 90/inference with the core running at 5 MHz. 







Smart Sensors, IoT, Low Power, Edge Computing, Machine Learning





§ INTRODUCTION




Artificial Intelligence and Machine Learning paradigms are increasing their presence in the area of the Internet of Things (IoT) and Smart Sensors to face the challenging task of automatically extracting information from the data generated by "traditional" sensors spread in the physical world <cit.>.
An emerging trend is to bring machine learning on low-power IoT devices, in contrast with the Cloud Computing paradigm, where the data processing is performed in the cloud.
This trend is motivated by the huge amount of data produced by a growing number of IoT devices.
Transferring all the data to the cloud would have several drawbacks including high energy requirements for data transmission, and, more generally, reliability, latency, and privacy concerns <cit.><cit.>. 














Major challenges arise when processing data at the Edge, mostly due to the highly constrained resources available on smart IoT devices<cit.>.
Thus, bringing intelligence to the edge creates fascinating challenges for industrial and academic researchers <cit.>.
In fact, most of today's low-power IoT smart sensors are composed of a low-power microcontroller (MCU), a set of sensors, and an optional but common wireless interface to transmit the findings of the device to a remote gateway or backend <cit.>. 




Many academic and sensor technology companies, including Bosch, TDK, and many others, are proposing a new generation of smart sensors that embed ultra-low power digital signal processors in the same integrated circuit.
In particular, there is interest in combining the sensing part with a small neural network core to process the sensor data on the fly, increasing the overall efficiency and reducing the system latency.




This novel approach is based on the intuition that waking up external processors periodically might have an impact on the power consumption, which is avoidable by moving the intelligence from the host processor to the sensor itself.

This paper presents the evaluation of a novel sensor with an Intelligent Sensor Processing Unit (ISPU) designed and produced by . The ISPU includes both full precision and binary neural network accelerators as well as the capability of DSP for feature extraction of the raw data. The main contribution of this paper is the design of benchmark neural networks (both full precision and binary) to evaluate the new sensor from under different loads to characterize the performance, including the features extraction on the ISPU, and the usability of this solution.
We compare different neural network architectures using both full-precision and binary implementations, with the aim to evaluate the power efficiency of the core in different conditions.
Experimental evaluation in terms of Multiply-Accumulate (MAC) operations per cycle and power in different conditions has been performed to have a fair comparison with a popular microcontroller for smart sensors based on ARM Cortex-M4F.






























§ STRED INTELLIGENT SENSOR PROCESSING UNIT



The inertial module is composed of a low-power, always-on, 6-axis Inertial Measurement Unit (IMU) and the novel Intelligent Sensor Processing Unit (ISPU), as represented in Figure <ref>.
The main novelty of this sensor is the presence of a fully programmable 32 core inside the sensor itself.

The processing core (ISPU) is optimized for power consumption and machine learning, and features 40 of RAM memory (for both program and data), a Floating Point Unit (FPU) for full-precision operations, and an additional Binary Neural Network accelerator with support for convolutions, to enable efficient BNN execution in the sensor.
The sensor offers a high-performance mode and a low-power one, with different sampling frequency ranges, while the ISPU can run at 5 or at 10.

While the processing core allows the implementation of any algorithm, compatibly with the computational resources, the ISPU is designed with a focus on Binary Neural Networks, which are a competitive alternative to full-precision models <cit.>, especially when coupled with ad-hoc accelerators <cit.> <cit.>.

The core has been designed with a wake-up time of only 4 cycles, to allow efficient event-based processing with minimal latency. It wakes up when new sensor data are available, but it is also possible to trigger the execution of algorithms from the external host MCU if needed.

The core can be programmed in C with user-defined algorithms, or with the support of the tools provided by , which allow to deploy Keras <cit.> models and take advantage of the BNN accelerator.







§ EVALUATION SCENARIO AND PROPOSED NEURAL NETWORK

To evaluate the potential of in-sensor intelligence with the ISPU under a realistic real-world setting, we chose an activity recognition task.
Specifically, we opted for an activity classification of different actions on an office chair, as this allows us to use directly the data from the accelerometer hosted in the same die of the ISPU. 
The set of activities we wanted to recognize is the following:

    
  * Idle: no motion is detected on the chair.
    
  * Stand up: a person stands up leaving the chair empty.
    
  * Sit down: a person sits on an empty chair.
    
  * Rotate: the chair spins around the shaft.
    
  * Move: the chair is moved around.



For our demonstration, we consider a prediction pipeline composed of a Feature Extractor followed by a Neural Network, as shown in Figure <ref>.
This well-known combination <cit.> allows us to evaluate the performance of the ISPU for both traditional feature extraction/signal processing tasks and machine learning loads.





 §.§ Feature Extraction

We adopted a feature extraction algorithm based on standard time-domain statistical features, which are suitable for real-time systems <cit.> and are invariant to the sampling rate of the sensor.

Since the ISPU is triggered at every acquisition, we consider 1 acquisition as our time unit to simplify the explanation below.
Every acquisition yields three 16 ADC samples, one per axis. The samples are stored in three dedicated circular buffers of length N=32.

With a periodicity of T=32 (when the buffers are full), we evaluate for each axis the following set of features: Mean, Median, Variance, Maximum, Minimum.
The set of features S_t is stored in a shift buffer which contains the last L=2 sets, from S_t to S_t-(L-1).
Together they compose the input of the neural networks described later.


The configuration we used in our application generates a total of 30 features, which compose the input for the neural networks described below.





 §.§ Neural Networks

We compare different neural network architectures using both full-precision and binary implementations, with the aim to evaluate the power efficiency of the ISPU in different conditions. We generated networks with growing complexity by adding hidden layers with 32 or 64 units.
The full-precision networks have an input size of 30 and the hidden layers have ReLU activation. 
The input of the binary network is padded with two zeros to comply with hardware limitations of the BNN core, which requires the input size of the layers to be multiple of 32. Binary activation is used in the hidden layers. All the models begin with a Batch Normalization layer and terminate with a fully connected layer with SoftMax activation.
The model variations are described in Table <ref>, along with the network complexity measured in Multiply-Accumulate operations.



To train the networks, we acquired a dataset with three different chair types. For each chair, we asked 25 different people to perform the required 5 activities. The models were trained for 100 epochs.
We evaluated the models with standard train/validation/test split, and achieved for all models an accuracy between 96 and 98 for the full-precision ones, and between 93 to 97 for the binary versions, confirming the expectations to experience a drop in accuracy due to the extreme quantization.
The models were deployed on the sensor with the tools provided by STMicroelectronics, while the feature extraction was implemented directly in C.























































































































§ EXPERIMENTAL RESULTS































 
 
 
 
















 §.§ Execution Time and Power Consumption

Since the toolchain does not include an accurate performance profiler, we evaluate the execution time by measuring the average duration of the ISPU phase in the power profile on about 100 samples per configuration.


The feature extraction has an average duration of 6.57, and is common to all the models.

As visible in Figure <ref>, the overhead of the feature extraction is considerable and dominates in the simplest network architectures.
We believe a consistent part of the overhead might be caused by the integer-to-floating-point conversions, which to our knowledge are not optimized.





In Table <ref> we compare the cycles/MAC operation for each network architecture on both the ISPU and an STM32L4R9, an MCU based on the Cortex-M4 core. On the latter, we only evaluate the full-precision models, since the architecture does not have native support for binary networks.

The data show that small networks, in general, perform worst than large ones, as the additional overhead of operations such as Batch Normalization/Softmax has a greater impact.
This is even more relevant in the case of binary networks, because of the aforementioned overhead of data conversion needed at the input and the output of the network.
For the same reason the binary models struggle to provide any speedup for small to medium models, granting only about 25 speedup for the largest model we could compare (Float_(2,64)) on the sensor, as visible in Figure <ref>.

To test this thesis, we also evaluated a single binary model with 4 hidden layers and 256 hidden units per layer to maximise the BNN accelerator utilization. While it was not possible to test the full-precision version of this model on the sensor for memory limitations, the binary one achieved 1.48 cycles/MAC operation, a promising speedup of about 7x on the previously measured full-precision performance.





































§ CONCLUSION




We proposed an evaluation of on-sensor activity classification with the novel sensor with integrated ISPU and analyzed the performance of the core under different conditions, proposing neural networks for realistic application scenarios, and exploiting both the sensing and the on-board processor. Our evaluation shows that the ISPU is suitable for running small full-precision networks, and is mainly limited by the available memory for what concerns this task. The cycles/MAC metric also shows that the performance of the core is comparable with the performance of a Cortex-M4 core with intensive full-precision loads with an inference energy of only 90. Our experiments also show that it is possible to run large binary models with a speedup of around 7x when using the dedicated BNN accelerator.







00
FANN-on-MCU Wang, X., Magno, M., Cavigelli, L., & Benini, L. (2020). "FANN-on-MCU: An open-source toolkit for energy-efficient neural network inference at the edge of the Internet of Things." IEEE Internet of Things Journal, 7(5), 4403-4417..
EdgeComputing G. Premsankar, M. Di Francesco and T. Taleb, "Edge Computing for the Internet of Things: A Case Study," in IEEE Internet of Things Journal, vol. 5, no. 2, pp. 1275-1284, April 2018, doi: 10.1109/JIOT.2018.2805263.
edge_computing_privacy A. Alwarafy, K. A. Al-Thelaya, M. Abdallah, J. Schneider and M. Hamdi, "A Survey on Security and Privacy Issues in Edge-Computing-Assisted Internet of Things," in IEEE Internet of Things Journal, vol. 8, no. 6, pp. 4004-4022, 15 March15, 2021, doi: 10.1109/JIOT.2020.3015432.
Magno2009 Magno, M., Brunelli, D., Thiele, L., Benini, L. (2009, August). Adaptive power control for solar harvesting multimodal wireless smart camera. In 2009 Third ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC) (pp. 1-7). IEEE.
Jelicic2010Jeličić, Vana, Michele Magno, Davide Brunelli, Vedran Bilas, and Luca Benini. "An energy efficient multimodal wireless video sensor network with eZ430–RF2500 modules." In 5th International Conference on Pervasive Computing and Applications, pp. 161-166. IEEE, 2010.
ContiWear Conti, F., Palossi, D., Andri, R., Magno, M., & Benini, L. (2016). Accelerated visual context classification on a low-power smartwatch. IEEE Transactions on Human-Machine Systems, 47(1), 19-30.
EdgeComputing2 W. Shi, J. Cao, Q. Zhang, Y. Li, and L. Xu, “Edge Computing: Vision and Challenges,” IEEE Internet of Things Journal, vol. 3, pp. 637–646, Oct. 2016. Conference Name: IEEE Internet of Things Journal.
bnn_good G. Cerutti, R. Andri, L. Cavigelli, M. Magno, E. Farella, L. Benini, "Sound Event Detection with Binary Neural Networks on Tightly Power-Constrained IoT Devices", 2021.
bnn_yoda R. Andri, L. Cavigelli, D. Rossi and L. Benini, "YodaNN: An Ultra-Low Power Convolutional Neural Network Accelerator Based on Binary Weights," 2016 IEEE Computer Society Annual Symposium on VLSI (ISVLSI), 2016, pp. 236-241, doi: 10.1109/ISVLSI.2016.111.
bnn_xor F. Conti, P. D. Schiavone and L. Benini, "XNOR Neural Engine: A Hardware Accelerator IP for 21.6-fJ/op Binary Neural Network Inference," in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 37, no. 11, pp. 2940-2951, Nov. 2018, doi: 10.1109/TCAD.2018.2857019.
keras Keras. The Python Deep Learning API. https://keras.io/. [Online; accessed 1/2/2022]
features_nn Y. Chen and Y. Xue, "A Deep Learning Approach to Human Activity Recognition Based on Single Accelerometer," 2015 IEEE International Conference on Systems, Man, and Cybernetics, 2015, pp. 1488-1492, doi: 10.1109/SMC.2015.263.

features J. M. H. Priyadharshini, S. Kavitha and B. Bharathi, "Classification and analysis of human activities," 2017 International Conference on Communication and Signal Processing (ICCSP), 2017, pp. 1207-1211, doi: 10.1109/ICCSP.2017.8286571.
stm32_datasheet STM32L4F5/7/9 Datasheet:
 www.st.com/resource/en/datasheet/stm32l4r5vi.pdf [Online; accessed 1/2/2022]
































































































































































































































 Paper Title: Enabling Next-Generation V2X Perception: Wireless Rigid Body Localization and Tracking.











Submitted to the IEEE Vehicular Technology Magazine Special Issue
N. Führling et al.





 Enabling Next-Generation V2X Perception: 
 Wireless Rigid Body Localization and Tracking

    Niclas Führling0000-0003-1942-8691, Hyeon Seok Rou0000-0003-3483-7629, Graduate Student Members, IEEE,

Giuseppe Thadeu Freitas de Abreu0000-0002-5018-8174, David González G.0000-0003-2090-8481, Senior Members, IEEE, and Osvaldo Gonsa0000-0001-5452-8159




N. Führling, H. S. Rou and G. T. F. Abreu are with the School of Computer Science and Engineering, Constructor University, Campus Ring 1, 28759, Bremen, Germany (emails: [nfuehrling, hrou, gabreu]@constructor.university).

D. González G. and O. Gonsa are with the Wireless Communications Technologies Group, Continental Automotive Technologies GmbH, Guerickestrasse 7, 60488, Frankfurt am Main, Germany (emails: david.gonzalez.g@ieee.org, osvaldo.gonsa@continental-corporation.com).

    August 5, 2024
=========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================



§ ABSTRACT 

V2X perception describes a suite of technologies used to enable vehicles to perceive their surroundings and communicate with various entities, such as other road users, infrastructure, or the network/cloud.

With the development of autonomous driving, V2X perception is becoming increasingly relevant, as can be seen by the tremendous attention recently given to ISAC technologies.

In this context, RBL also emerges as one important technology which enables the estimation of not only target's positions, but also their shape and orientation.

This article discusses the need for RBL, its benefits and opportunities, challenges and research directions, as well as its role in the standardization of the 6G and B5G applications.




V2X perception, RBL, tracking, B5G, ISAC, wireless systems.







§ INTRODUCTION














WL can be regarded as an early example of ISAC that demonstrates how communication signals can also be used to extract environmental information (i.e., the location of users) using signals originally indented for communications <cit.>.

These capabilities become particularly relevant in certain industry verticals, such as intelligent transportation systems (ITS) and automotive, where there is an increasing need for V2X perception and convergence of previously independent functions, such as sensing, communication, and computing.

Indeed, WL has been successfully used in a wide range of applications including security, smart homes, industrial automation, robotics, vehicular networks, localization of UAV, and more.

In terms of the fundamental concept or metrics for WL, i.e., the type of information extracted from wireless signals for the purpose of localization, many SotA techniques have been proposed, including, methods based on finger-prints, RSSI, AoA, radio range, mere connectivity or hop-counts, and combinations of them.



The WL literature also exhibits vast diversity in terms of the underlying mathematical approaches used in the design of positioning algorithms, with examples ranging from purely algebraic methods exploiting MDS, and its further developed SMDS variant, methods based on optimization-theoretical techniques, such as SDP, PoCs, FP and MC, to schemes based on ML, which gain more popularity in the recent years.

Finally, in addition to the aforementioned methods, abundance also exists in terms of offered features, such as robustness against noise, bias, LOS and NLOS conditions, and mitigation of effects, including not only information scarcity, uncertainty of anchor points, co-existence of near- and far-field waves, but also physical problems, such as missing links for measurements that are needed for the positioning algorithms.

These challenges will be further discussed in Section <ref>.

Despite the healthy breadth of topics covered by the literature, one important practical aspect that has not been fully addressed by most of the SotA WL methods (nor commercial wireless systems, e.g., 5G) is the fact that, in many use cases, targets could be better represented as 3D objects.

Aiming to address this matter, a growing literature is emerging, in which each target is modeled not as a single point, but as a group of inter-connected points with a fixed and known arrangement, i.e, a rigid body <cit.>.

In our view, a rigid body representation of objects would enhance most of the applications mentioned before, such as control, mobility, and safety for autonomous robots, vehicles, and UAV.

Thanks to the inherent inclusion of target shape and orientation models, which can be either used as prior information or jointly estimated within the solution of the localization problem, these RBL techniques, have been shown to provide higher accuracy than conventional (point-based) WL methods.

Thus, RBL expand the notion of a single reference position to position, translation, and rotation, which are very valuable information in many realistic scenarios.

In view of the later, a significant amount of work and schemes for RBL have appeared.




Remarkably, anchorless localization methods will become more relevant in the future, taking into account the fact that sidelink-based positioning has recently been added to the 5G specifications in 3GPP release 18, as well as <cit.>, which is a positioning technique purely operating between UE.

With the recent popularity of autonomous driving, extending the idea of rigid body localization even further leads to rigid body tracking approaches that can be used for collision detection.





In that sense, in <cit.>, a two-stage approach was used to estimate rotation, translation, angular velocity and translational velocity by range and Doppler measurements, making use of various WLS minimizations.


Emphasizing the distinction between WL/ISAC <cit.>, where wireless connectivity is a fundamental component of the technology, and other approaches based on, e.g., inertial sensing 
or machine vision, which are outside of the scope of this article,

it can be said based on the above that with respect to RBL, the current focus of the WL/ISAC research community is essentially extending/applying the multitude of techniques found in the vast ML literature to the RBL problem,  which could be jointly referred to as W-RBL.

The rest of the article is organized as follows:

Section <ref>, provides industry and standardization perspectives on the need for adopting RBL features in beyond-5G systems, e.g., 6G, as well as potential use cases and applications.

Section <ref> offers a brief introduction of RBL, defines the most relevant mathematical relationships, and discusses the transition from point-based localization to rigid body localization.

Next, in Section <ref>, main applications, challenges and opportunities that come along with the usage of rigid bodies are discussed in detail.

Finally, Section <ref> closes the article with final remarks and future research directions.






§ RIGID BODY LOCALIZATION IN BEYOND-5G AND 6G



Positioning services have been standardized in existing 3GPP commercial networks, such as Long term Evolution (LTE, 4G) and New Radio (NR, 5G), mainly as a necessary complement to conventional Global Navigation Satellite Systems (GNSS), for certain mission critical applications with stringent requirements in terms of positioning accuracy, latency, availability, and/or trustworthiness/reliability. In general terms, 5G positioning features include network-based positioning, sidelink-based positioning (without direct use of network signals), and hybrid-enhancements which leverage positioning capabilities in other available radio access technologies (RATs), such as ultra-wide band (UWB), GNSS, Bluetooth, WiFi, etc. A variety of methodologies have been deployed including time difference, round trip time (RTT), carrier phase measurements, angle of arrival/departure (AoA, AoD), etc., and improvements have been made in terms of signaling overhead, coverage scenarios, integrity, accuracy, etc. Since the beginning of 5G-Advanced (5G-A, release 18 and beyond) positioning enhancements based on Artificial Intelligence and Machine Learning (AI/ML) have also been studied and are currently being specified. However, a more general framework for RBL has not been considered yet in 5G systems, and hence, it is an interesting (and required) positioning expansion/enhancement to be developed for  <cit.>, i.e., 6G. 





 §.§ Standardization Perspectives and Applications


At the time of writing, the timeline for 6G has been sketched <cit.> and process that will shape it within the 3GPP has also been kicked off with a workshop on 6G use cases <cit.>. The 6G study items at the technical specification group service and system aspects (TSG SA) will be approved in September 2024 based on early inputs provided by a wide range of industry sectors (verticals), mobile network operators, regional alliances, and the International Telecommunications Union (ITU). Sensing and (advanced) positioning (potentially including RBL), have been clearly identified among the strongest drivers for 6G, and both of them have been framed within the usage scenario[IMT-2030 comprises 6 usage scenarios: 3 of them are evolution from existing 5G service types (Immersive Communication, Hyper Reliable and Low-Latency Communication, and Massive Communication) and 3 of them are expansion towards new usage scenarios, including ISAC, Ubiquitous Connectivity, and Artificial Intelligence and Communication.] ISAC. Thus, RBL could certainly be one of the most interesting novelties in 6G. 


Sensing and advanced positioning are key to support use cases in several verticals, such as ITS, automotive, and automated/smart  factories. Complex applications requiring localization and tracking, object reconstruction, and environment perception can be benefited from RBL capabilities. 
In the context of industrial applications <cit.>, RBL can enhance real-time high-accuracy digital twins and industrial metaverse, which in turns enable logistics, mobile robots, and factory automation. For the automotive sector, its applicability is also clear in many use cases <cit.> requiring real time environmental modeling and/or intelligent automated driving systems, such as automatic valet parking, lane merging/maneuvering, smart intersections, protection of vulnerable road users, and automatic charging/fueling of (electric) vehicles. In addition, RBL is also very compatible with the use of (next-generation)  in connected/digital roads <cit.>.   

To emphasize the potential role of RBL in future 6G systems, Figure <ref> provides an overview of past and future milestones, highlighting the capabilities and use cases specified in .






§ RIGID BODY LOCALIZATION SYSTEM MODEL





 §.§ From Wireless Point-based Localization to Wireless Rigid Body Localization (W-RBL)


As described before, there are many different approaches with different types of information that have been proposed for wireless localization.

In this section, we aim to describe the process of extending a standard point-based localization method to incorporate the rigid body framework in Sec. <ref>.

Whereas single point localization tend to use a single measurements to a set of anchors, the same can be applied to rigid bodies, where each sensor in the rigid body performs a measurement to the set of sensors, or vice versa.

The main difference lies in the fixed construction of the rigid body that need to be incorporated in the corresponding methods, having obtained information through measurements of all the individual sensors.

Thus, any localization method can effectively be reformulated to incorporate the RBL constraints by an appropriate insertion of the RBL framework given by equation (<ref>), opening significant directions of improving the existing SotA methods.





 §.§ Background of Rigid Body Localization



To gather a basic understanding of rigid bodies, we will shortly present the underlying mathematical system model of RBL.

Given a rigid body, the position of the k-th node within a rigid body can be described by a 2D coordinate vector 𝐜_k∈ℝ^2 × 1,









which captures the x- and y-coordinates of the k-th node of the rigid body respectively, for all K points defining the rigid body.

Next, following the SotA system model of the standard RBL framework <cit.>, a change in position and orientation of a RBL can be described by the joint affine transform of the K coordinates of a rigid body, given by 



    𝐒   ≜ [𝐬_1, ⋯, 𝐬_k, ⋯, 𝐬_K] 
       = 𝐑·[𝐜_1, ⋯, 𝐜_k, ⋯, 𝐜_K]^≜ 𝐂 +  [𝐭, ⋯, 𝐭, ⋯, 𝐭] 
       =𝐑𝐂+ (𝐭⊗1_1× K) ,


where 𝐒∈ℝ^2 × K consists of the new coordinates of the rigid body nodes after the transform, 𝐑∈ℝ^2 × K and 𝐭∈ℝ^2 × 1 respectively describe the rotation and translation equally to the K original coordinates, and ⊗ is the kronecker product.

In addition, another important property of rigid bodies is the fixed relative Euclidean distances between the K nodes in 𝐂, which must also be satisfied in 𝐒.

Therefore, such information is incorporated in RBL method as a necessary constraint.

The implication for RBL is that the known structure of the rigid body in 𝐂 will be exploited to aid the estimation of the localized positions in 𝐒, where the affine transform 𝐑 and 𝐭 is unknown, and only some noisy observation of 𝐒, or other kinds of measurements related to 𝐒 are available.

The above model and properties can easily be extended to the 3D scenario by adding another dimension to the coordinate vector and correspondingly in eq. (<ref>), with out a loss of generality, where a simple illustration of a rigid body transformed by a rotation and a translation is visualized in Figure <ref>, by vehicle turning on a street.


    




     

     
     

     
     






§ CHALLENGES AND OPPORTUNITIES



With the main idea of rigid bodies and a clear mathematical description defined, this chapter aims to discuss various approaches problems and solutions to RBL, drawing an analogy to single-point localization schemes and presenting which problems have been solved for those schemes, but are open to investigate for the special case of rigid body targets.

This includes the discussion for multiple different localization approaches, the scenario where the needed observations are incomplete, thoughts about the optimal sensor deployment, as well as some insights into next-generation extensions to anchorless localization and tracking.





 §.§ Localization Approaches




  §.§.§ Algebraic Approaches




Inspired by the aforementioned discussion, it can be considered what type of measurements can be used and combined for localization, which measurements are the most useful, and how one can benefit from the known shape of the rigid body.

Combining all these measurements leads to different algebraic and geometric approaches resulting in diverse estimation methods, with the most well known examples being triangulation, MDS, and SMDS approaches <cit.>.

Therefore, to give an insight on how these methods can be adapted to rigid bodies, Fig. <ref> illustrates how different type of measurements can be combined for single-point localization, as well as RBL.

On the left hand side, a single target is localized via range, AoA and ADoA measurements, where by combination of the later the position can be exactly defined.

On the right hand side, a multi-point rigid body is localized by the combination of the same measurements.

What can be observed is that due to the known shape of the rigid body one might be able to save certain measurements, since the conformation matrix adds a constraint that helps in the localization process.

Additionally it can be seen that for a rigid body, angle measurements are much more relevant than range measurements, since two points can have the same distance to an anchor, as illustrated, while the angles change.

However, which kind of measurements are the most effective and how to process them is still open to investigate.






  §.§.§ Optimization Approaches


In terms of optimization, a lot of SotA methods, can, and have been applied to rigid bodies.

Belonging to these applications, methods like least squares, SDP, PoCs, FP and MC schemes have been deployed <cit.>.

In contrast to the pure algebraic approaches, optimization based methods tend to describe the desired problem mathematically in a way such that through common techniques, such as gradient descent methods, an optimization problem dealing with the main objective, as well as, for example, noise constraints can be solved.

Depending on the problem statement itself, these approaches can lead to efficient estimations, possibly leading to closed-form solutions, which can also be adapted to rigid body frameworks.



  §.§.§ Message Passing


The algebraic and classical optimization methods described above, while providing optimal solutions with possibly even closed-form expressions, typically require high complexity due to inherent matrix inversions and exhibit low flexibility and robustness to various additional/incompleteness of information in the system.

Alternatively, in light of the fundamental system model given in eq. (<ref>), tailored message passing algorithms <cit.> may be designed to solve the RBL problem of estimating the 3D rotation angles and translation vector.

Due to their flexibility in application and design, message passing algorithms have been thoroughly investigated and exploited in vast areas of engineering and science, including channel coding applications, data estimation, distributed machine learning and much more, and have been shown to be able to provide Bayes-optimal solutions under proper conditions.

The fundamental uniqueness of the method lies in the concept of passing locally computed messages in a factor graph structure shown in Fig. <ref>, where different nodes correspond to different parts of observed information and/or unknown variables of the original system, under which the key messages – typically the sufficient parameters of probability distributions – are computed at each node and transmitted (passed) over the graph edges, which represent the existence of a dependency between the nodes (i.e., between the observation and the variable).

In hand of the received messages, different nodes recompute and update the variable beliefs, repeating the message passing iterations until convergence.

Due to this distributed nature, the message passing algorithms are known to be robust to incomplete data and irregularity in the system model, whose framework is flexible to support the design of an estimator given the underlying system.

However, they may suffer from self-interference and divergent behavior if proper design and mitigation techniques are not applied in the formulation of the messages.

In light of the above, we highlight the message passing technique as a promising solution to the RBL problem, which fundamentally involve observation data such as range, angle, and power measurements, whose values are related to the sensor position variables.

Furthermore, the unique relationship (constraint) between the position variables in the form of the rigid body conformation can efficiently be represented by an additional layer of edges between the variables, as exemplified in Fig. <ref>.

In addition, the inherent probability distribution-based operation of the message passing fits nicely with the notion of localization with multiple distributions and confidence ellipses (soft-decisions), not to mention the robustness to unavailable observations often arising in wireless sensing scenarios, as will be elaborated in the following section.








 §.§ Incomplete Observations



As hinted by the prior sections, independent of the metric used for localization, a common issue regularly occurring is the observation of incomplete measurements, for instance due to blocked paths.

Just to name one example, when considering range measurements from anchors to sensors, a LOS path might be blocked due to the structure of the rigid body, which leads to zeros occurring in the observed distance matrix, shown in Figure <ref>.

In the case of range measurements these zeros are especially harming, since putting a zero is equivalent to stating that the measured distance is exactly zero, which poisons the algorithms.

Therefore, these zeros have to be handled with a lot of care, since either the algorithm has to be adjusted to be aware of the missing information, or something has to be done to reconstruct the lost data.

One possible solution to this problem, which highly benefits from the structure of the rigid body, captured in the conformation matrix C are MC solutions.

MC literature <cit.> offers a wide range of different methods that can be used for completion, often used in scenarios such as recommender systems in computer science, wave channel estimation in wireless communication, but also localization algorithms in signal processing.

By knowing the conformation structure of the rigid body itself, as well as the positions of the anchors, it is possible to construct a hollow squared EDM, containing the intra-distances of the anchors to one another, the intra-distances of the points in the rigid body itself, as well as the incomplete measured cross-distances.

This information alone is sufficient to complete the matrix with high accuracy to improve the performance significantly.

Recent contributions have proposed a discrete-aware variation that makes use of the knowledge that the values to complete can be chosen from a finite discrete alphabet set.

In theory this can be implemented for a rigid body as well, where after measuring
the incomplete distance matrix, the discrete sets can be found by taking into account all possible rotations of the rigid body, only being possible because the structure of the body is fixed and known.

Even though the discrete-aware MC algorithm can be performed once the sets are found, the construction of the sets itself becomes a combinatorial problem for larger rigid bodies, where more research is required to find suitable algorithms for the constructing of the set.






        




 §.§ Optimal Performance and System Setup


Another important aspect in terms of real life scenarios and industrial standards is the sensor deployment on the rigid body itself, i.e. how many sensors to use and where to put them on a specific vehicle.

Since the effect has not been studied sufficiently in the SotA, future work on this topic needs to be done, with a precise mathematical analysis of the effects.

For illustration, a simple set of simulations have been performed with a 3D vehicle that was localized via a conventional least squares minimization problem <cit.>, with the amount of sensors and their position varying.

Results of this can be found in Figure <ref>, illustrating the translation estimate RMSE for a varying number of sensors with different noise levels.

It can be observed that for two sensors only, no satisfying result can be obtained, where four sensors are sufficient to locate the 3D rigid body, with a large performance gain.

Furthermore, when the number of sensors increases to ten, the error decreases and starts to converge.


In terms of positioning the sensors on the rigid bodies the specific used case has to be defined first before, deploying the sensors.

As an example, take the scenario presented in Section <ref>, where some measurements might be missing due to NLOS paths.

In these type of scenarios on has to make sure that the sensors are positioned in such a way that the amount of possible NLOS paths is minimized, depending on the scenario.

On the one hand, if the sensors will connect to anchors positioned high up in road side units it might be useful to place the sensors on top of the rigid body.

On the other hand, if the rigid body is connecting to other rigid bodies, it is advisable to place them in the front and the back, such that they can connect to other rigid bodies in all possible directions.

However, for conventional single-point localization scenarios, multiple SotA methods have investigated the problem of optimal sensors and positioning.

To name one example, <cit.> investigates the optimal anchor positioning using frame theory.

To summarize, it was found that for single targets the anchors can placed to minimize the frame potential, i.e. how closely a set of anchors in a Hilbert space approximates an orthonormal basis.

For multiple targets, the anchors can be placed sequentially optimizing the the position of the anchor with respect to each of the targets position, which is build on the problem of single-target localization using AoA measurements.

As a result, <cit.> proposed the optimal anchor placements in 2D and 3D for different amount of anchors, forming tight frames that were proved to be optimal for single targets.

However, to translate the idea on rigid bodies, Fig. <ref> first presents optimal placements for 2D and 3D scenarios, before extending the idea, illustrating difficulty of placing two sensors only for rigid bodies of higher dimensions, where for a 2D rigid body the anchors need to be placed in 3D space to be optimal, however, for a 3D object this is not possible anymore.

With the work that has been done, it is left to do further research and investigations on how the concepts can be applied to RBL, keeping in mind that the frame theory presented only applies to the anchors surrounding the rigid bodies, put not the placement of sensors on the rigid body itself.

Similarly the effect of sensor uncertainty of the anchors, as well as the sensors on the rigid body itself need to be studied in more depth.







 §.§ Towards Next-Gen Rigid Body Localization and Tracking




  §.§.§ Anchorless Multi-Rigid Body Localization


Another more recent relevant approach in terms of autonomous vehicles proposes a relative multi-rigid body localization method in an anchorless scenario, estimating the relative translation and rotation between two rigid bodies by measuring the cross-body LOS distances between the bodies, which is visualized in Fig. <ref>.

This relative multi-rigid body localization is especially useful in scenarios where there are no, or only a few anchors deployed, such that the bodies can find the relative translation and rotation even without any external sensors, which can be useful for collision detection for vehicles or UAV or for other autonomous driving scenarios, only to name a few possible use-cases.

However, there exists one major flaw related to a real life implementation of such a system, which is the following.

As explained in (<ref>) and Section <ref>, each rigid body has a known conformation matrix C.

The problem occurs when localization methods need the conformation matrices of all vehicles in order to work, which is not the case in real life, since a body can indeed measure the distances to other sensors, but does not necessarily know the other bodies structure.

To solve such issues for real life applications regulations on how knowledge on C_i should be acquired and distributed would be required.

Otherwise, egoistic methods need to be developed, which are able to perform a localization purely by measurements, without any prior knowledge about the rigid body itself, such as the one proposed in <cit.>.




Nevertheless, common SotA methods, such as MDS can be used to obtain this knowledge and solve the problem, whereas other problems, such as missing measurements due to blocked paths can be solved by methods as described in Section <ref>.





  §.§.§ From Rigid Body Localization to Rigid Body Tracking


Inspired by the aforementioned work and as a logical next step towards autonomous driving, one has to extend the concept from rigid body localization to rigid body tracking.

While for single-point based localization methods it is more intuitive to connect the model to time, adding a time constant to the rigid body system model can become a more complicated task.

Single-point based systems can easily incorporate time by applying a velocity to their point, which is not directly possible  for rigid bodies.

In terms of rigid body tracking, a lot of research has been done in the field of visual tracking, for example tracking robots, faces or general gestures by cameras.

In that sense many approaches, such as feature based tracking, edge based tracking, or sensor fusion has been proposed, which performs very well for its applications.

Nevertheless, as described before, we are focusing on W-RBL and therefore have to search for a different approach.

What can be done to model the motion of the entire body instead of visually capturing its motion is to add an angular velocity and a translational velocity <cit.>.

Mathematically this can be described in simple form per sensor as


    ṡ_i=[ω]^×Rc_i+ṫ_i,


where ṡ_i denotes the velocity of the individual sensor in the inertial frame, [ω]^× being the cross product operator matrix applied to the angular velocity and ṫ_i being the translational velocity.

Another important aspect to keep in mind is what type of tracking to pursuit, since multiple approaches are known and can be called tracking.

The most common tracking approach is to a notion of time in the system model, which coherently influencing the velocity or similar parameters.

These type of tracking applications are commonly solved via SotA methods, such as Kalman filters.

Another more relevant type of tracking with respect to rigid body applications is to look at moving rigid bodies in time from frame to frame, only working at one time instance at a time, such as shown in equation (<ref>).

This type of tracking can be used especially effective in frameworks such as platooning problems.






§ CONCLUSION AND OUTLOOK



In this work we have provided a detailed discussion about the usage of RBL, which is a highly promising technique for future V2X perception and autonomous driving applications.

In that sense, we have presented the general system model of a rigid body framework, with the transition from a single point localization to RBL, enabling the estimation of the orientation of the target instead of only the position.

Furthermore, challenges and opportunities in a vast array of fields and applications have been discussed.

The discussion about incomplete measurements directly connects to the sensor deployment part, which has shown how the amount of anchors and their placement in general can influence the performance, leading to conditions where incomplete measurements occur which can be addressed by different methods, such as matrix completion.

Further, it was discussed how modern signal processing methods, including message passing can be redesigned to fit the RBL problem, resulting in significant complexity reductions/gains.

Even more relevant for future applications, especially for autonomous driving in rural areas where not many anchors are available, we discussed how multiple rigid bodies can perform localization in anchorless scenarios with respect to each other
and how the whole framework of RBL can be extended to rigid body tracking.

With this in mind, future work and applications might develop scenarios where two rigid bodies belong to one object, which have a soft connection joint between them <cit.>, for instance in the case of large busses or trains with multiple movable sections.

Finally, as discussed in Section <ref> and illustrated in Figure <ref>, positioning applications have been standardized in all existing 3GPP commercial networks, where advanced positioning has been set as a goal for 6G, which offers great potential for future RBL applications. 



IEEEtran

10

url@samestyle








LimaAccess2021
C. De Lima, D. Belot, R. Berkvens, A. Bourdoux, D. Dardari, M. Guillaud, M. Isomursu, E.-S. Lohan, Y. Miao, A. N. Barreto, M. R. K. Aziz, J. Saloranta, T. Sanguanpuak, H. Sarieddeen, G. Seco-Granados, J. Suutala, T. Svensson, M. Valkama, B. Van Liempd, and H. Wymeersch, “Convergent communication, sensing and localization in 6G systems: An overview of technologies, opportunities and challenges,” IEEE Access, vol. 9, pp. 26 902–26 925, 2021.

ChepuriTSP2014
S. P. Chepuri, G. Leus, and A.-J. van der Veen, “Rigid body localization using sensor networks,” IEEE Transactions on Signal Processing, vol. 62, no. 18, pp. 4911–4924, 2014.

DongTWC2023
X. Dong, G. Wang, K. C. Ho, and L. Huang, “Rigid body localization in unsynchronized sensor networks: Analysis and solution,” IEEE Transactions on Wireless Communications, vol. 23, no. 4, pp. 2901–2916, 2024.

Chen_2015
S. Chen and K. C. Ho, “Accurate localization of a rigid body using multiple sensors and landmarks,” IEEE Transactions on Signal Processing, vol. 63, no. 24, pp. 6459–6472, 2015.

Nic_RBL
N. Führling, H. S. Rou, G. T. F. de Abreu, D. González G., and O. Gonsa, “Egoistic MDS-based rigid body localization,” in 2024 IEEE International Symposium on Modeling and Optimization in Mobile, Ad Hoc, and Wireless Networks (WiOpt), 2024 (submitted).

02:00074
ITU-R, International Telecommunication Union - Radiocommunication Sector, “M.2160-0: Framework and overall objectives of the future development of IMT for 2030 and beyond,” Nov. 2023.

01:00133
3GPP TSG Chairs, “RP-240823: Additional considerations for 6G timeline,” Mar. 2024, 3GPP, Technical Specification Group Radio Access Network, Meeting #103.

01:00134
3GPP, “SWS-240025: Summary of the Workshop,” May 2024, Stage 1 Workshop on IMT-2030 Use Cases, Rotherdam, NL.

01:00135
——, “SWS-240006: 5G-ACIA Considerations on 6G Use Cases and Requirements,” May 2024, Stage 1 Workshop on IMT-2030 Use Cases, Rotherdam, NL.

09:00055
5GAA, “C-V2X Use Cases and Service Level Requirements. Volume III.” Jan. 2023, White paper.

15:00020
——, “A Visionary Roadmap for Advanced Driving Use Cases, Connectivity Technologies, and Radio Spectrum Needs,” Sep. 2020, White Paper, version 1.0.

Feng_2021
O. Y. Feng, R. Venkataramanan, C. Rush, and R. J. Samworth, “A unifying tutorial on approximate message passing,” 2021.

Chen_2022
Z. Chen and S. Wang, “A review on matrix completion for recommender systems,” Knowledge and Information Systems, vol. 64, no. 1, pp. 1–34, 2022. [Online]. Available: <https://doi.org/10.1007/s10115-021-01629-6>


zhao2013optimal
S. Zhao, B. M. Chen, and T. H. Lee, “Optimal sensor placement for target localisation and tracking in 2D and 3D,” International Journal of Control, vol. 86, no. 10, pp. 1687–1704, 2013.

führling2023softconnectedrigidbodylocalization
N. Führling, H. S. Rou, G. T. F. de Abreu, D. González G., and O. Gonsa, “Soft-connected rigid body localization: State-of-the-art and research directions for 6G,” 2023. [Online]. Available: <https://arxiv.org/abs/2309.05002>








Niclas Führling
   received the B.Sc. degree in electrical and computer engineering from Jacobs University Bremen, Bremen, Germany in 2022. He is currently pursuing the M.Sc. degree in electrical engineering with the University of Bremen, with a focus on communication and information technology, while working on a research project at Constructor University, focusing on 6G connectivity. His current research interests are wireless communications, signal processing, and positioning.




Hyeon Seok Rou
 (S'19) has received the B.Sc. degree in Electrical and Computer Engineering from Jacobs University Bremen (currently Constructor University), Germany, in 2021 - where he is currently pursuing a Ph.D. in Electrical Engineering, funded by a research project from the Wireless Communications Technologies group at Continental A.G. His research interests lie in the fields of joint communications and sensing (JCAS), Bayesian statistics, multi-dimensional modulation schemes, and mmWave/sub-THz MIMO wireless communications.




Giuseppe Thadeu Freitas de Abreu
(Senior Member, IEEE) is a Full Professor of Electrical Engineering at Constructor University, Bremen, Germany. His research interests include communications theory, estimation theory, statistical modeling, wireless localization, cognitive radio, wireless security, MIMO systems, ultrawideband and millimeter wave communications, full-duplex and cognitive radio, compressive sensing, random networks, connected vehicles networks, and many other topics. He currently serves as an editor to the IEEE Signal Processing Letters and the IEEE Communications Letters.





David González G.
(S’06–M’15–SM’18) received a Ph.D. degree in Signal Theory and Communications from the Universitat Politècnica de Catalunya, Spain. Since 2018, David is with Continental AG, Germany, focusing on diverse aspects of wireless communications, vehicular communications (V2X), and integrated sensing and communications (ISAC). David also serves as delegate in the 3GPP for 5G standardization, ETSI, and 5GAA.





Osvaldo Gonsa
received the Ph.D. degree in electrical and computer engineering from Yokohama National University, Japan, in 1999, and the M.B.A. degree from the Kempten School of Business, Germany, in 2012. He is currently the Head of the Wireless Communications Technologies Group, Continental AG, Frankfurt, Germany. And since 2020 also serves as a member for the GSMA Advisory Board for automotive and the 6GKom Project of the German Federal Ministry of Education and Research.
  




 Paper Title: TinyChirp: Bird Song Recognition Using TinyML Models on Low-power Wireless Acoustic Sensors.


IEEEexample:BSTcontrol




TinyChirp: Bird Song Recognition Using TinyML Models on Low-power Wireless Acoustic Sensors 
Corresponding author: zhaolan.huang@fu-berlin.de. 



    
    Z. Huang1,
    A. Tousnakhoff1,
    P. Kozyr2,
    R. Rehausen3,
    F. Bießmann25
    R. Lachlan4,
    C. Adjih6,
    E. Baccelli61
    
    1Freie Universität Berlin
    2Berlin University of Applied Sciences
    
    
    
    
    
    3Woks Audio GmbH
    4Royal Holloway University of London
    
    
    
    
    
    5Einstein Center for Digital Future
    6Inria, France
    
    
    
    
    

    

    August 5, 2024
======================================================================================================================================================================================================================================================================================================================================================================================================================================



§ ABSTRACT 
Monitoring biodiversity at scale is challenging. Detecting and identifying species in fine grained taxonomies requires highly accurate machine learning (ML) methods. Training such models requires large high quality data sets. And deploying these models to low power devices requires novel compression techniques and model architectures. While species classification methods have profited from novel data sets and advances in ML methods, in particular neural networks, deploying these state of the art models to low power devices remains difficult. Here we present a comprehensive empirical comparison of various tinyML neural network architectures and compression techniques for species classification. We focus on the example of bird song detection, more concretely a data set curated for studying the corn bunting bird species. The data set is released along with all code and experiments of this study. In our experiments we compare predictive performance, memory and time complexity of classical spectrogram based methods and recent approaches operating on raw audio signal. Our results demonstrate that our TinyChirp approach can robustly detect individual bird species with precisions over 0.98 and reducing the energy consumption such that a deployment time on a single battery charge can be extended from 2 weeks to almost an entire season of 18 weeks.  



TinyML, Microcontroller, Acoustic Sensors, Machine Learning




§ INTRODUCTION


The typical data pipeline with bio-acoustic research requires the deployment of sensors, in which each node is battery-powered and left in the field to record environmental sound, continuously, for a long period (e.g. for a whole season). Thereafter, the recorded data is manually collected by researchers on-site, from each node, and further analyzed in laboratory. In the case of avian species for instance, only the targeted bird species is relevant within the recorded data, and the rest of the data is to be discarded ultimately. This is both a cumbersome process and a waste of resources, not only at this downstream stage in the lab, but also upstream in the field, on each node, while recording large amounts of irrelevant data. In this pipeline, continuous recording of audio thus creates a bottleneck in terms of memory and energy budgets available on individual sensors – typically limited to a small battery, and an SD card, respectively, driven by rudimentary software running on a low-power microcontroller. Such hardware is very energy-efficient, but very limited in memory resources, with RAM memory budgets in the order of 500 kiloBytes <cit.>, which yields specific constraints on software embedded on such devices <cit.>.

Meanwhile, as AI achieves excellent performance in pattern recognition of audio and biosignals, more and more bioacoustic researchers leverage ML  related methods to improve accuracy and efficiency. 
However, until recently, such ML models, e.g. BirdNet <cit.> were confined to lab use only, as their resource requirements (GigaBytes of RAM) are way beyond the capacity of microcontroller-based hardware available on sensors in the field. However, recent advances in TinyML, a lively field of research targeting machine learning for microcontrollers, offer a glimpse of hope that pattern recognition of bioacoustic signals might become possible, over longer periods, as required by the aforementioned use cases. 






However, ..., which required to:
[inline]to be finish

    
  * train ML models.
    
  * deploy models on sensor node.
    
  * evaluate the resource consumption of models on boards.


This paper thus introduces ....


Paper Contributions. In this paper, we explore the possibility of using TinyML in practice, on common low-power microcontroller hardware, for a concrete use case: monitoring corn bunting birds' songs, in a rural area in the UK, over several months in a row, using a fleet of energy-efficient acoustic sensors. We aim to answer the following questions: (Q1) Can we pre-screen the audio on the low-power sensor node, so as to only store the targeted bird songs on each sensor? (Q2) How does that extend the device's lifetime, in terms of memory and energy budgets? More in detail, our contributions are as follows:


    
  * We propose a pipeline to record, recognize and store specific bird songs on low-power microcontroller-based devices, which can be further scale-up to universal species;
    
  * We developed a neural network with high accuracy on bird song recognition. Furthermore, we optimize the network with partial convolution technology to minimize the memory consumption on deployment;
    
  * We propose a two-stage classification approach to reduce computational and storage cost and enhance the overall accuracy;
    
  * We provide experimental results on both the classification performance of models and resource consumption on low-power devices;
    
  * We publish a new data set of curated audio recordings snippets of corn bunting bird songs;
    
  * We provide open source code[see <https://github.com/TinyPART/TinyBirdSounds/>] to reproduce the results on common MCU.



§ BACKGROUND & RELATED WORK




Bird Song Recognition –
Bird song detection has significantly advanced through various machine learning techniques, particularly deep learning methods. Recent research has explored different strategies for recognizing bird songs, with substantial progress in preprocessing audio data for recognition.
One prevalent preprocessing technique involves transforming the audio signal into a spectrogram using STFT <cit.>. This spectrogram can subsequently be converted into a mel spectrogram <cit.>,  by applying a mel scale, followed by a log scale transformation <cit.>. This process effectively redefines the audio recognition task as an image recognition task. CNN <cit.> and Residual Neural Networks (ResNets) <cit.> are widely utilized for such image recognition tasks in bird song detection. For instance, one study employed CNNs to recognize the songs of 17 bird species <cit.>, while another study utilized ResNets to classify the songs of 19 bird species <cit.>. BirdNET <cit.>, a well-known neural network for bird song recognition, is derived from the family of residual networks and can recognize 6,000 of the world’s most common species at the time of writing.

Another approach involves transforming the audio signal into Mel-Frequency Cepstral Coefficients (MFCCs), which are then used as input for bird song classification <cit.>. Alternatively, feature extraction using wavelet decomposition can be employed for this purpose <cit.>. Additionally, certain studies combine different architectures; for example, CNN and Transformer <cit.>, and CNN and VGG <cit.> to enhance performance. While these methods are effective for bird song recognition, their computational requirements are typically too demanding for execution on microcontrollers.

Bioacoustic Audio Datasets –
High-quality and diverse datasets are crucial for training models to accurately recognize birds and differentiate between species by their unique songs. The Xeno-canto database [see <https://www.xeno-canto.org>] is one of the largest and most comprehensive collections of bird sounds, containing over 500,000 recordings from more than 10,000 species. In <cit.>, 2,699 bird song audio recordings from the Xeno-canto database, ranging in length from 20 to 0.74 seconds, were used to classify 17 bird species.
Another extensive collection of bird audio recordings is available in the Macaulay Library [see <https://www.macaulaylibrary.org/>], renowned for its high-quality bird audio recordings and detailed metadata, which includes information about species, location, and context. The combination of the Macaulay Library and Xeno-canto audio recordings was used to train BirdNet <cit.>.
In <cit.>, the public dataset CLO-43DS, containing recordings of 43 different North American wood-warblers, was utilized. Another notable collection is the Birdsdata dataset with audio files of 20 bird species from the Beijing Academy of Artificial Intelligence (BAAI) repository, which is frequently used in research and model training <cit.>.
To complement datasets focused on bird species, the Google AudioSet <cit.> and Urbansound8K <cit.> datasets are used to include non-bird species sounds.



TinyML –
For models to operate on low-power devices, they must be compact and computationally efficient. Studies have demonstrated the use of lightweight CNNs for speech recognition and age classification <cit.>, water leakage detection <cit.>, acoustic defect localization <cit.>, fall detection for the elderly <cit.> and other tasks <cit.>. Tiny vision transformers have also been employed for classification tasks in various studies <cit.>. Moreover, quantization techniques can be applied to models to reduce their size to fit within the constraints of low-power devices <cit.>.

Creating spectrograms from audio signals can be power-intensive; hence, some studies use raw time-series data as input for neural networks <cit.>. For instance, single-channel EEG was used as input for a model based on CNN and Transformer for sleep stage classification <cit.>. Raw audio signals were used for urban sound analysis <cit.>. Time-series data offer advantages for TinyML applications, including reduced computational load and suitability for low-power devices.


Embedded Software Platforms for TinyML – The widely used model transpiler TVM (Tensor Virtual Machine <cit.>) has recently been extended with uTVM, providing automated transpilation and compilation for models output by major ML frameworks (TFML, Pytorch, etc.). As such uTVM exposes low-level routines and  optimizes these for execution on different processing units, including for targets such as a large variety of microcontrollers. Prior works such as <cit.>, <cit.> or MLPerfTiny <cit.> focused on the production, performance and analysis of standard benchmark suites of representative TinyML tasks on different microcontrollers. Conversely, prior work such as U-TOE <cit.> or RIOT-ML <cit.> provide embedded operating system integration of TinyML, facilitating TinyML benchmarking and continuous deployment over low-power wireless network links such as IEEE 802.15.4 or BLE.



§ BIOACOUSTIC MONITORING SCENARIO



As depicted in Figure <ref>, a network of battery-powered, autonomous recording units (ARUs, i.e. microcontroller-based acoustic sensors) is deployed across a monitored area. An example of ARU is given in <cit.>.

These sensors remain in the field for an entire season, typically around six months. Locations are often remote and hard to access and devices may be distributed over a relatively wide area, making visiting them time-consuming. Birds typically do not sing consistently during the whole day, while each bird has several different song types that need to be regularly sampled. Moreover, each individual bird moves around its individual territory, singing from a range of song posts, not all of which will be adequately recorded from any one location. 

On the network aspect, the sensors are distributed approximately evenly, ensuring that each sensor can be wireless connected by more than two neighboring sensors. This arrangement facilitates approximate triangulation through distance estimation based on signal strength. The placement process can be streamlined using LEDs and a basic ultra-low-power wireless protocol, such as IEEE 802.15.4 or LoRa. For space reason, we do not detail further networking aspects in the paper, but rather focus on the on-device machine learning aspects. We nevertheless keep an eye on the total memory footprint on the device, to ensure that the operating system including application TinyML code, the OS and a typical wireless low-power network stack fit in typical resource budgets on common microcontroller-based boards.

Although our work is designed around one very specific real-world research scenario (monitoring common Corn Bunting bird songs) the architectures should be readily applicable to other audio data classification tasks. Examples of related scenarios to which our method could be extended include, surveying remote areas for targeted endangered species of conservation value, or broader surveys of songbird biodiversity.  Similar devices would even be useful in the search for very elusive species such as Ivory-billed Woodpecker <cit.>. 




 §.§ Corn Bunting Monitoring Use-Case




SongBeam <cit.> microcontroller-based recorders have been used to monitor Corn Bunting (Emberiza calandra) birds in Oxfordshire, UK. 
A set of 30-40 devices have been deployed since 2022, currently comprising 3 complete breeding seasons (February - July).

SongBeam devices are based on an ARM Cortex-M microcontroller, run on 4 D-cell batteries and record 4-channel WAV files onto 128  microSD cards. To maintain such a fleet of sensors, devices are currently checked approximately every 2 weeks, especially because memory space is soon exhausted on the microSD cards. 

The above deployment has been used to produce a significant part of the dataset we present in Section <ref>.
Our experience using SongBeam and a preliminary analysis of the raw dataset we produced shows that less than 10% of recording time contains useful recordings. 
Given the extended deployment period (6 months), remaining low-power while improving storage efficiency is thus paramount. 
Moreover, current prototypes use solar power, further emphasizing that the bottleneck is downloading data from the microSD cards to free space on-board. 

This description of the limitations of using SongBeam recorders to monitor Corn Buntings is likely to apply to many biomonitoring scenarios: solar power may allow for extending deployment lifetime in remote locations, but storage of recordings is a limiting factor.






Describe Robert's scenario.
A bunch of battery-powered, microcontroller-based acoustic sensors are disseminated across an area to be monitored. The sensors are left in the field for a whole season at a time ( 6months). Power and storage efficiency are paramount.
To emphasis the network aspect, we can assume that the sensors are placed approx. evenly so that each sensor hears N>2 other sensors beacons, so that approx. triangulation is possible via (signal strength) estimated distance. Placement can be simplified via LED and a basic ultra low-power wireless protocol such as IEEE 802.15.4 or LoRa.


§ EVALUATION METRICS



On the one hand, as the basic part of bird song recording, the model should identify the target bird song as much as possible, potentially requiring a complex structure with a large size. On the other hand, the limited resource budget allows only simple models to be deployed on low-power IoT devices.

Thus we considered two orthogonal types of metrics to evaluate the prediction performance on bird song classification and resource usage on low-power IoT devices of TinyML models. Preliminary results are presented in Section <ref>.




 §.§ On-device Resource Usage


We performed extensive profiling of all models on low-power devices to capture their effective memory and time complexity in a real-world application scenario. The metrics are helpful to investigate the energy efficiency of the models, complementing the commonly used metrics for predictive performance. 

Memory (RAM) Consumption –
This metric measures the amount of dynamic memory space (primary RAM) consumed by the model during inference. It reflects the memory footprint of the model activation and is important for low-power devices that have limited memory resources. Efficient memory utilization allows for the deployment of larger and more complex models on such devices.

Storage (Flash memory) Consumption –
This metric quantifies the amount of storage space, typically in terms of Flash memory region, required to store the compute instruction and associated parameters. 

Minimizing storage consumption allows for accommodating multiple models on the device or orchestrating with other essential applications.

Computational Latency –
This metric measures the time consumption of performing inference for each input sample at the model level. It reflects the inference speed of the model on the low-power device and plays a crucial role in real-time or latency-sensitive applications. Core clock frequency, cache strategies and communication latency between memory and working core have a great impact on this indicator.

Energy Consumption – 
This metric is crucial for battery-operated and resource-constrained devices, where efficient energy usage can significantly impact device longevity and performance. Energy consumption encompasses both active power (when the device is triggered to perform tasks) and idle power (when the device is in sleep mode).



 §.§ On-device Prediction Performance



Accuracy –
It is defined as the ratio of correctly predicted instances to the total instances in the dataset. While accuracy is intuitive, it can be misleading, especially in highly imbalanced datasets. 

To address this limitation, accuracy should be combined with other metrics that offer a more detailed view of model performance.

Precision and Recall –
They are two fundamental metrics of classification models used to evaluate the ability to distinguish true positives (TP) from false positives (FP) and false negatives (FN), particularly in the context of imbalanced datasets. 


F-Score –
There is often a trade-off between precision and recall; increasing one can lead to a decrease in the other. 

As two common instances, F_1-score weighs them evenly, while F_2-score treats recall as two times more important than precision, applying in scenarios where false positives are more tolerant than false negatives.

ROC Curve –
While the above metrics are sensitive to class imbalance – which occurs in many species classification and detection tasks – there are other metrics that are more robust towards class imbalance. The receiver-operator characteristic (ROC) curve depicts the  TPR and FPR across different decision thresholds.



The AUC is a key metric derived from the ROC curve that quantifies the overall performance of the model and is invariant with respect to class imbalance. An AUC of 1.0 indicates perfect classification, while an AUC of 0.5 suggests no discriminative ability.  


move to discussion part
    - Confusion matrix
    - ROC Curve -> different cut-off value





§ METHODOLOGY


We used a combination of project-specific and publicly accessible data to train and validate our classification models. In the data pre-processing phase, the raw audio signals were further segmented, labeled, down-sampled and divided into different groups to generate classification datasets. The corresponding Mel-spectrograms of the pre-processed audio segments were also created for spectral-based methods (models). During the pre-processing stage, we conducted a pilot analysis to establish guidelines and determine specific hyper-parameters for downsampling and spectrogram generation.




 §.§ Data Acquisition


The publicly accessible data originated from Xeno-Canto, Macaulay, Google AudioSet, while the project-specific data were previously collected in a long-term research of bird song patterns. We published the resulting dataset[see <https://www.kaggle.com/datasets/zhaolanhuang/tinychirp>].



    
  * Oxfordshire Corn Buntings. This library contains recordings of corn buntings along a transect of approximately 20km in Southern England. Corn buntings sing in a mosaic-like pattern of geographical variation called dialects; our sample contains approximately 6 different dialects. The recordings were performed with directional parabolic microphones as described in <cit.>.  
    
    
    
  * Macaulay Library. This library contains the necessary audio recordings for our target species as well as the other identified species. There are a total of 796 entries of audio recordings for the target species. We filtered the audio files by the song tag and removed 16 entries due to unclear metadata and missing catalog numbers. For other species as non-target, we limit the selection to a maximum of 30 recordings per species and choose those with the highest average community rating, acquiring a total of 1468 recordings.
    
    
  * Xeno-Canto. This library also contains recordings of target species and other species. We retrieved the recordings only marked with song and rated with the best quality level, resulting in 303 recordings of target species and 9622 recordings of other species as non-target.
    
    
  * Google AudioSet. This dataset provides environmental sounds which contain non-bird sounds. 


After gathering all audio recordings from the above libraries, we used BirdNET to identify and chop corn-bunting segments from all datasets, and labeled them as target bird songs; all segments were truncated or zero-padded to the length of 3 seconds. Meanwhile, we chose other 3-s segments randomly and ensured no overlap with corn-bunting segments, with labeled as non-target bird songs. Table <ref> shows the distribution of target and non-target segments over all datasets.



 §.§ Data Pre-processing



This phase contains the following steps:


    
  * We divided the segments into training, validation and test sets with the ratio of 80:10:10.

    
  * All segments were downsampled to 16  using zero-order holder, which constituted the audio dataset.
    
    
  * We transformed the audio segments into Mel-Spectrograms, and grouped them with the same splitting ratio to form the spectral dataset.



Pilot Analysis –

We averaged the STFT spectrograms of target and non-target segments in the training dataset to investigate their frequency characteristics, as depicted in Figure <ref>. Obviously, a bright band lays on the spectrogram of target segments roughly between 4000 and 8000 , hints that a sample rate with 16  should be sufficient to preserve all frequency components of corn-bunting song according to Nyquist–Shannon theorem. A highpass filter can also be applied to eliminate low-frequency noise without damaging target songs. Thus, we downsampled the segments from 48  to 16  to reduce the resource consumption and improve the efficiency of classification phase. In the baseline model we also designed a highpass filter to enhance SNR.




Mel-Spectrogram –

Mel-spectrogram is commonly used in audio classification tasks due to its ability to closely approximate human auditory perception. To create log Mel-spectrograms, we first generated magnitude spectrograms of STFT for all downsampled audio segments. The STFT was performed with a Hann window with a width of 1024 samples and a step size of 256 samples. This process generated 184 window frames and 513 frequency bins. Thereafter, the magnitude spectrograms were mapped onto the Mel scales with 80 Mel bins. In creating the Mel-spectrograms, we employed a sampling rate of 16  and focused on the frequency range of 80-8,000 Hz. This frequency range was chosen because it encompasses the typical frequency range of most bird songs, including that of the target Corn Bunting. Lastly, we converted the Mel-spectrograms into a logarithmic scale to obtain log Mel-spectrograms, with the shape of 184 × 80. 









| Data         | # Bird | # Non-Bird |
| ————— | —— | ———- |
| Roberts         |        |            |
| Xeon-Canto      |        |            |
| Macaulay        |        |            |
| Google AudioSet |        |            |


§ BASELINE & DECISION STRATEGY


We propose an approach to optimizing the screening process by combining a high-speed signal processing step (hereafter named "baseline") with the precision of an TinyML model, ensuring reliable results while maintaining resource-consumption efficiency.



 §.§ Baseline


We developed a lightweight pre-selector signal processing step, not relying on machine learning. First, the audio segments are normalized by min-max schema. Thereafter, a 9th-order highpass Butterworth filter with the cut-off frequency of 7000  is imposed on the normalized segments, in order to suppress noise and non-relative sound in low-frequency and to preserve only target components. Finally, we calculate the signal power P of the filtered segments,


    P = 1/N∑_n=1^N x(n)^2,

where x(n) and N denote the data points and the length of a segment, respectively. 
We define two thresholds: t_low and t_high. If P is smaller than t_low, the microcontroller remains in the idle/low-power state, and the audio sample is discarded directly. Else, P is compared to t_high. Depending on this result, and on the decision strategy (see below), the audio sample is either stored or further analyzed via inference using the machine learning model.





 §.§ TinyChirp Decision Strategy





We employ a double-phase approach for enhanced accuracy and efficiency of bird song recognition, combining the baseline with a TinyML model for further verification. The process is as follows:

    
  * Step 1: Baseline Filtering – Pre-screening is conducted using the baseline pre-processing step described in Section <ref>, designed to provide a quick and efficient preliminary analysis. If P is smaller than t_low, the microcontroller remains in the idle/low-power state, the audio sample is discarded and the process aborts. Else proceed to Step 2.
    


 

    
  * Step 2: Inference-based Classification – If the power-saving flag is set and if P is smaller than t_high a higher-accuracy classification is conducted via inference  using TinyML model. If the audio sample is not classified as the target, the audio sample is discarded and the process aborts. Else proceed to Step 3.


  * Step 3: Storage – Without further processing, the audio sample is logged, i.e. stored on the SD card. 

    


Note that the decision strategy can therefore be configured with two "knobs". On one hand the power-saving flag determines skipping (or not) some of the computation and can thus decrease energy consumption. On the other hand, different TinyML models can be inserted in Step 2.



Next, we thus study two families of TinyML models for Step 2: the first category of model takes a Mel-Spectrogram as input, while the second category of model takes time-series as input. Table <ref> presents a summary of the explored TinyML model structures. 



This study mainly investigated the performance of two types of ML models – CNN and Transformer – on time-series and Mel-spectral input. Due to the compute- and memory-intensive characteristic of ML methods, two major optimization strategies were leveraged to compress the models and improve their efficiency, as discussed in Section <ref>. 

This study also presents a baseline model built with classical digital signal processing methods, as performance reference of ML-based models. Furthermore, we integrated baseline and ML-based model into ensemble one to gain a optimal compromise between accuracy and efficiency resource consumption? on low-power devices. 






§ CLASSIFICATION BASED ON SPECTROGRAMS



In contrast to the computer vision community, where the transition from frequency decomposition based feature extraction to neural feature extraction on raw image data introduced in 2012 led to a significant increase in predictive performance <cit.>, neural feature extraction on raw audio time series did not contribute to a comparable breakthrough yet. Instead most ML techniques for audio pattern recognition (somewhat surprisingly) were relying on image processing pipelines – sometimes with neural networks trained on ImageNet data. Basically, image classification is performed, using a neural network, analyzing the spectrogram rendering of the audio trace.

For this reason, we initially consider the below two models, based on state-of-the-art neural network architectures.

CNN-Mel –
This model contains two 2D convolutional layers for Mel-spectral inputs as feature extraction and two fully connected layers as classifier. Both convolutional layers utilize 4 filters with a 3 × 3 kernel size and ReLU activation function, followed by 2 × 2 max pooling to reduce spatial dimensions. The output of the classifier are normalized by Softmax function as well.

SqueezeNet-Mel – 
This model is based on SqueezeNet <cit.>, an advanced backbone focused on optimizing the efficiency of computer vision applications by strategically reducing parameters, with comparable performance to AlexNet <cit.>. It leverages a so-called Fire module to achieve higher efficiency compared to standard convolutions with only a slight decrease in accuracy. The Fire module significantly reduces the number of parameters by the squeeze layer (1×1 convolutions), which exceedingly reduces the number of channels before the more parameter-heavy 3×3 convolutions in the expand layer. This reduction in intermediate channels means fewer parameters overall.
We aligned its input shape with the Mel-spectrogram and tailored the output for binary classification. Also offering potential for further compression using quantization techniques, SqueezeNet is thus appealing given memory constraints.





Limitations of Spectrograms on Microcontrollers
Practical experience on low-power microcontroller-based devices has shown however that producing and manipulating mel-spectrograms from audio signal streams on such devices is problematic. It inserts an additional step in the processing pipeline, which increases latency and memory requirements. Prior work (such as <cit.>, section 6.2) concludes that the CPU bottleneck is not just the model inference time, but also the spectrogram calculation, which, compared to inference alone, almost doubles latency. Other previous work such as <cit.> measures on a quite powerful STM32F7 microcontroller that computing and writing in memory the mel-spectrogram of 30 columns and 40 frequency bands takes approximately 1 second.  Note that these spectrogram dimensions are much smaller than our requirements (184 columns × 80 frequency bands) hence even more latency can be expected in our case.

For these reasons, we next explore pipelines which skip the spectral pre-processing stage as described below.



SqueezeNet<cit.> is a CNN characterized by comparable performance to AlexNet while using approximately 50 times fewer parameters. The compact architecture of SqueezeNet is achieved by employing eight unique building blocks called the Fire Modules. Each Fire Module starts with a "squeeze" convolution using 1x1 filters to reduce input channel dimensions, thereby cutting down on computational load. This initial step compresses data into fewer channels. Following this, an "expand" convolution layer mixes 1x1 and 3x3 filters: the former expands channel numbers, while the latter captures spatial details. Moreover, SqueezeNet offers the potential for further compression using quantization techniques, making it an efficient choice given memory constraints.

The implemented SqueezeNet-Time-Series architecture uses a reduced number of feature maps in compare to initial SqueezeNet. This compressed model has in the first convolutional layer 0.3 times the number of filters that initial model has in this layer. In the fire modules in convolutional layers, we use the integer value of 0.3 times the number of squeeze filters given to this layer in initial model. Since the number of expand filters depends on the number of squeeze filters and is four times greater, it also undergoes compression. The structure of the fire module in SqueezeNet-Time-Series is presented in Table 2. Therefore, if the initial number of squeeze filters in the fire module is x, the compressed number of squeeze filters in this fire module will be int(0.3× x). Then both the expand layers will have 4× int(0.3× x) filters.

Additionally, since the initial model uses 2D convolutions and time-series are 1D arrays, we converted all 2D convolutional layers to 1D convolutional layers. The model was trained using time-series data for 10 epochs with the Adam optimizer and cross-entropy loss function. After training, this model was further quantized using different post-training quantization techniques.





For spectrogram approach we aimed to create an ultra-compact CNN-Mel-Spec model, comprising just two convolutional layers and two fully connected layers. The input for the model is a log Mel spectrogram with a size of 184x80. Both convolutional layers utilized 4 filters with a 3x3 kernel size and ReLU activation function, followed by max pooling with a 2x2 pooling window to reduce spatial dimensions. After the last convolutional layer and max pooling, the data were reshaped into a one-dimensional vector for input into the subsequent fully connected layer. The first fully connected layer had 8 neurons and utilized ReLU activation. The final fully connected layer contained two neurons with a linear activation function. For optimization, we employed the Adam optimizer with a learning rate of 0.001, and the model was trained over 15 epochs, using cross-entropy as the loss function. In an effort to reduce model size, we converted it to TFLite format and applied several post-training quantization techniques: float16 quantization, dynamic range quantization, and full integer quantization.













§ CLASSIFICATION BASED ON TIME-SERIES


Contrary to the models described in the previous section and inspired by the success of neural feature extraction in the computer vision domain <cit.>, the pipelines we aim at next take directly the raw audio signal time-series as input. More precisely, we designed the three models described below.

CNN-Time –
This simple model performs feature extraction with two sequential (1D) temporal convolutional layers followed by average pooling. A max pooling is inserted in-between to reduce the dimension of feature maps and enhance the non-linearity of the network. Two fully connected layers act as classifier at the end of the network, with the probability outputs normalized by Softmax function.  

Transformer-Time –
Inspired by <cit.>, we designed an efficient, attention-based model with only one temporal convolutional layer and one single-head transformer. The convolutional and pooling layers serve as feature extraction to transfer raw audio signal into embeddings for the transformer. The activations of the fully connected layer are normalized by Softmax function as well.



SqueezeNet-Time –
This model is based on SqueezeNet <cit.>. We tailored this basis to align the input shape with the audio segment, and narrowed the output channels for binary classification. To adapt the backbone to time-series input, all 2D convolutional layers are replaced by temporal convolutional layers. Furthermore, to have a more compact structure, we decreased the filter number of all convolutional layers by roughly 70%, as presented in Table <ref>.





§ ADDITIONAL MODEL OPTIMIZATIONS FOR TINYML


In this study, we used two optimization techniques to compress the models and further reduce their memory consumption: model quantization on the one hand and on the other hand partial convolution, as described below.







 §.§ Model Quantization

Quantization reduces the model size and inference time by converting the weights and activations from higher precision (e.g., 32-bit floating-point) to lower precision (e.g., 8-bit integer) <cit.>. In this study we adopted PTQ with weights and activations quantized in 8-bit integer. To avoid substantial loss of prediction performance, the training dataset was used to find the optimal scale factor and zero-point of the activations.



 §.§ Partial Convolution

We observed that the outputs of the Conv1D layer require significant memory, which impedes deployment on resource-constrained tiny devices. As presented in Table <ref>, the peak memory consumption occurs at the first Conv1D layer both in CNN-Time and Transformer-Time with 768  and 3 , respectively. This indicates that deployment on resource-constrained tiny devices is impractical.

To address this issue and inspired by <cit.>, 

we exploited the fact that average pooling can be computed iteratively point-by-point on the output channels of the final Conv1D layer over a small sliding window of inputs. Each point in the output channels depends only on a small subset (kernel window) of the outputs from the previous layer. That is, for a block of L Conv1D layers following by an average pooling layer, its output y can be iteratively computed on the input sequence x(n), n= 1 … N as following:

    y_j(k)    = y_j(k-1) + 1/N_L A^L_j(k), 
    
        k    = 1 … N_L, j = 1 … C_L . 
     
        A^l_c(n)    = ∑_i=1^C_l-1 W^l_c(i) · [ A^l-1_i(n-K^l/2) ⋯
       =4A^l-1_i(n) ⋯ A^l-1_i(n+K^l/2) ]^T 
    
        A^0(n)    = x(n) , 
    
        l    = 1 … L, c = 1 … C_l, n = 1 … N_l, N_0 = N

where y_j(N_L) is the result of the average pooling of the j-th channel; A^l_c(n) denotes the n-th point in channel c of the l-th Conv1D layer calculated with kernel weights W. Each Conv1D layer contains C_l channels with filter size of K^l and output size of N_l. Without loss of generality, the formula of partial convolution contains only the linear components and considers only one-dimension case for the sake of simplicity; it can be generalized in high-dimension and integrated with non-linear building blocks (e.g., stride, non-linear activations, pooing layers, etc.).

Figure <ref> provides a comparison between classic convolution and partial convolution. Unlike classic convolution where entire channels (C × N) are computed and stored before being processed by the next layer, partial convolution requires only a small part of the channels (C × K, K << N) for each layer. In our case with K=3 and N=48000, it can be expected roughly 16000 × smaller memory consumption, making the implementation more suitable for tiny devices. We applied this strategy to the Conv1D layers combined with average pooling in the CNN-Time and Transformer-Time architectures.










§ TINYCHIRP PERFORMANCE EVALUATION


In the following, we conducted a comprehensive evaluation of the discriminative performance of TinyChirp with different TinyML models, as well as of resource consumption and computation time on typical low-power boards based on microcontrollers.



 §.§ Classification Performance Comparison







Our evaluation process began with the generation of ROC curves and the corresponding AUC on the training dataset, providing a clear picture of the models' abilities to distinguish between classes. As shown in Figure <ref>, the spectrogram-based models – CNN-Mel and SqueezeNet-Mel – achieved the best classification performance with AUCs of 1.0 and 0.99, respectively, followed by time-series models – CNN-Time and Transformer-Time – both with 0.98 AUC. Unexpectedly, with a more complex structure, SqueezeNet-Time performs worst (lower AUC than the baseline).


We next measured accuracy, precision, recall, F_1- and F_2-score, shown in Figure <ref>. Spectrogram-based models achieve the highest metrics for the whole range of threshold values. For time-series models, Transformer-Time worked overall better than CNN-Time; Again, SqueezeNet-Time performs worst (consistent with the above ROC analysis). 


Next, we focused on finding the threshold settings for optimal  F_2-score prioritizing recall, i.e. reducing the likelihood of mistakenly discarding records of the targeted birds). The results are shown in Table <ref> for training data, thereafter verified on test (previously unseen) data as shown in Table <ref>. Note that this threshold influences energy consumption: more target bird classification leads to more energy consumption (as data needs to we stored on to SD card in this case).

These results are promising and allowing to focus only on the relevant audio segments (10% of the total recording time), as the classification performance is compelling. Thus, TinyChirp can save 90% of SD card space compared to the initial monitoring scenario (Section <ref>), potentially extending the deployment time to 18 weeks. However, complementary experiments must now be carried out on targeted hardware, so as to evaluate other key metrics: computation time, energy consumption and memory footprint on typical microcontroller-based devices. The next sections focus on that part.




presents the optimized setting with corresponding metrics on the training dataset. This approach ensured that our models were fine-tuned to prioritize recall, thereby reducing the likelihood of missing critical positive cases. Also we emphasize that the decision threshold is directly related to energy consumption in the use case considered here: more positive predictions will lead to higher energy consumption. Hence high precision is desirable for low energy consumption, but recall should be traded with caution against energy consumption.

After determining the optimal threshold setting on the training dataset, we validated this threshold using the test dataset, as described in Table <ref>. This step was crucial to ensure that the threshold not only performed well on the training data but also generalized effectively to unseen data. By confirming the optimal threshold on the test dataset, we demonstrated the robustness and reliability of our TinyML models' predictive performance.













 §.§ Performance Evaluation on Microcontrollers


Experimental Setup – For our measurements, we used a common low-power off-the-shelf board: the nRF52840 Development kit (nrf52840dk). This board is based on an ARM Cortex-M4 processor, with a typical memory budget: 1  of Flash memory and 256  of RAM. As embedded software base we used RIOT-ML <cit.>. 

Throughout the experiments, we monitored the (RAM/Flash) resource footprint using built-in diagnostic tools provided by RIOT-ML and RIOT <cit.>, as well as external measurement equipment as described below.

Energy consumption was measured using an ampermeter to gauge the board's energy efficiency. This involved recording the current draw of the MCU and the external storage system (SD card) in different stages. We used a voltage regulator to supply 3.3  DC output and a logger to capture detailed current profiles over time.

Scope of the Measurements – Besides the model inference stage, we also considered the resource footprint of pre-processing during bird song recognition. Pre-processing refers to down-sampling and calculation of Mel-spectrogram as described in Section <ref>.

Also, the energy consumption in MCU's lowest power mode (idle stage) was measured as the reference of absence of sound with sufficient intensity.



Measurement Results on Microcontrollers – The first striking observation is that  SqueezeNet-Time and SqueezeNet-Mel were not deployable due to OOM (exceded memory budget) and were thus excluded in this experiment. All other models fit in the constraints of memory and storage budget, even when discounting the memory footprint of the OS and the network stack for wireless communication. 

The resource footprint of different stages and models measured on an nrf52840dk board are shown in Table <ref>.  We observe that the model inference stage takes the bulk of the overall energy consumption, as it requires the most process time and MCU in full power mode. 
As expected, the idle stage requires the least power, as all peripherals and the MCU are in the lowest power (standby) mode (until woken up by sound with sufficient amplitude). 











Spectrogram vs Time-series – Considering only the prediction performance in Section <ref>, it would seem natural to choose CNN-Mel as the best candidate, since it also provides the best inference latency. However, taking pre-processing time into account (which for CNN-Mel includes producing a spectrogram) shows a different picture. It takes roughly 2.4 seconds in total for a 3 second audio snippet, which is dangerously near the real-time constraint and costs high power consumption. With the best prediction performance and the lowest total compute latency among time-series models, the Transformer-Time now appears like the best choice to deploy on tiny devices.



Next, we consider jointly the overall resource consumption and the classification performance to discuss variants for the decision strategy described in Section <ref>:

Baseline Only –
With the Baseline t_low (Step 1 and Step 3 only) setting, the required computational resources are minimal, while achieving a recall of 0.99; however, 
due to the low precision, many false positives are stored, leading to the lowest storage efficiency.

TinyChirp Skipping Baseline –
With this variant, the TinyML model is applied immediately, without pre-screening by the baseline (Step 2 and Step 3 Only), leading to high predictive performance and high storage efficiency, but requiring higher energy consumption. Even when using the Transformer-Time with the lowest energy footprint, this requires more than 9 times the energy required for the baseline. This variant is ideal for devices currently running out of storage space while still with substantial battery left.

TinyChirp –
This variant refers to the full series of Step 1-3 described in Section <ref>, whereby the samples not discarded by the baseline are double-checked by the TinyML model. This approach offers a good compromise of compute, storage and energy footprint, by discarding early, without impacting too much the overall prediction performance achievable by the TinyML model on its own. 

TinyChirp with Power-saving –
This variant (Step 1-3 with the power-saving flag enabled) provides a more energy-efficient twist to TinyChirp.

However, the overall precision is determined by the threshold setting t_high, which causes an extra 40% of false positives to be stored onto the SD card, leading to lower storage efficiency. This variant seems more suitable for currently running out of battery, but with sill sufficient storage resources.






§ CONCLUSION

In this paper we present an empirical study of a bioacoustic monitoring use case showing that machine learning is usable directly on autonomous recording units based on low-power microcontrollers. We focused on a concrete use-case: monitoring Corn Bunting birds in rural UK. We publish a data set of the recordings, based on which we demonstrated that TinyML can perform on-device high accuracy classification for bird song classification. Compared to the traditional approach used so far, our approach TinyChirp can extend from 2 weeks to 18 weeks – almost a full season – the time intervals until which researchers must harvest overfull data storage embedded on the recording units deployed in the field. Our experiments also observe hands-on how performing inference directly on the audio time-series is the better approach on microcontroller-based hardware, compared to inference on mel-sepctrograms typically used in audio pattern recognition. Beyond the specific use case of monitoring Corn Bunting birds' songs, based on our general-purpose TinyML approach and our open-source code implementation integrated into the RIOT operating system, TinyChirp is applicable to a wide range of similar bioacoustic use cases, using a large variety of low-power microcontroller-based hardware.




§ ACKNOWLEDGMENT

The research leading to these results partly received funding from the MESRI-BMBF German/French cybersecurity program under grant agreements No. ANR-20-CYAL-0005 and 16KIS1395K. We acknowledge the receipt of media from the Macaulay Library at the Cornell Lab of Ornithology and the Xeno-Canto database.  Felix Bießmann received support from the Einstein Center Digital Future, Berlin,
the German Research Foundation (DFG) (528483508 - FIP 12), the Federal Ministry of Economic Affairs and Climate Action (RIWWER 01MD22007H), the German Federal Ministry of the Environment (GCA 67KI2022A) and the German Federal Ministry
of Education and Research (KIP-SDM 16SV8856).


IEEEtran 









 Paper Title: Rapid and Power-Aware Learned Optimization for Modular Receive Beamforming.




An Empirical Study on Challenges of Event Management in Microservice Architectures
    Yongluan Zhou
    August 5, 2024
==================================================================================






§ ABSTRACT  

mimo systems play a key role in wireless communication technologies. 
A widely considered approach to realize scalable mimo systems involves architectures comprised of multiple separate modules, each with its own beamforming capability. Such models accommodate cell-free massive mimo and partially connected hybrid mimo architectures. A core issue with the implementation of modular mimo arises from the need to rapidly set the beampatterns of the modules, while maintaining their power efficiency. This leads to challenging constrained optimization that should be repeatedly solved on each coherence duration. 
In this work, we propose a power-oriented optimization algorithm for beamforming in uplink modular hybrid mimo systems, which learns from data to operate rapidly. We derive our learned optimizer by tackling the rate maximization objective using projected gradient ascent steps with momentum. We then leverage data to tune the hyperparameters of the optimizer, allowing it to operate reliably in a fixed and small number of iterations while completely preserving its interpretable operation. We show how power efficient beamforming can be encouraged by the learned optimizer, via boosting architectures with low-resolution phase shifts and with deactivated analog components.  Numerical results show that our learn-to-optimize method notably reduces the number of iterations and computation latency required to reliably tune modular mimo receivers, and that it allows obtaining desirable balances between power efficient designs and throughput.













§ INTRODUCTION



Massive mimo technology is at the core of future wireless communications, with extremely large implementations expected to enable meeting the constantly growing demands in throughput and coverage <cit.>. However, implementing communications with hundreds or thousands of antenna elements gives rise to several challenges, including issues related to cost, power consumption, deployment, and scalability <cit.>. To overcome these challenges, a leading approach to massive mimo  adopts modular designs. In such cases, an extremely large mimo  array is implemented using many distinct modules, each equipped with a (smaller) antenna array, and possibly possesses local beamforming capabilities. These modular architectures can take different forms, e.g., distributed cell-free massive mimo <cit.>, centralized interconnections of multiple panels <cit.>, or radio stripes <cit.>.

While modular implementations of massive mimo alleviate cost and power issues and support distributed implementation, they also introduce design and algorithmic challenges. In particular, beamforming is not carried out by a single cpu that manipulates the signal at each element; instead, it is divided between the cpu and a set of separate arrays termed panels, that are possibly constrained in their hardware capabilities, and are likely to be subject to strict power constraints. Moreover, the setting of the modular beampattern is done on each coherence duration, necessitating rapid adjustments of the modules <cit.>.

Since modular beamforming is divided between a cpu and multiple panels, it can be viewed as a form of hybrid beamforming <cit.>. Hybrid beamforming divides the processing of the signals between conventional digital processing and an additional constraint antenna interface, typically representing analog hardware <cit.>. Such hybrid mimo is often studied in the context of RF chain reduction <cit.>, most commonly assuming phase shifter based analog processing <cit.>. However, hybrid mimo can  also represent emerging antenna architectures based on true-time-delay <cit.>, vector modulators <cit.>, metasurfaces <cit.>, or leaky waveguides <cit.>. 

The main task of hybrid beamforming can be viewed as the conversion of csi into a beampattern achievable through a divisible architecture <cit.>. Various methods have been proposed for tackling this task (see, e.g., overview in <cit.>). As hybrid beamforming can be represented as a constrained optimization problem, a conventional approach achieves this via iterative optimizers, see, e.g., <cit.>. However, these approaches typically involve lengthy optimization with numerous iterations, which can be challenging to carry out efficiently within the extremely short time frames required for determining beamforming (e.g., on the order of 0.1 milliseconds <cit.>).

To facilitate rapid beamforming setting, recent works proposed using deep learning tools, taking advantage of their ability to learn to tackle challenging optimization problems with fixed and limited latency <cit.>. This can be achieved by training conventional deep learning architectures, such as cnn, to map mimo csi into  analog and digital precoders <cit.>, or by
combining optimization with deep learning for hybrid beamformers <cit.> as a form of model-based deep learning <cit.>. However, while hybrid beamforming can be viewed as encompassing modular beamforming, its associated algorithms are not tailored for exploiting modular structures, and do not account for the limited power considerations imposed in such forms of distributed mimo systems. Moreover, current learning-aided designs often focus on a given configuration, and the optimizer needs to be retrained when using a different array (or adding another module), limiting scalability. Finally, existing works typically focus on downlink beamforming, with the uplink being much less studied.

Uplink modular beamforming was considered in <cit.>. There, the focus was to characterize trade-offs of its decentralization under narrowband communications and sparsity-based power-oriented constraints for loss-less processing, rather than establishing algorithms for tuning modular beamformers. In practice, the regimes of interest are typically those where the distributed and power constrained operation is likely to yield some rate loss compared to centralized fully digital mimo systems. This gives rise to the need to design methods for tuning hybrid modular receive beamformers for such practical and challenging settings.

In this work, we develop a rapid and power-aware tuning algorithm for setting modular receive mimo beamformers. 
Our approach combines principled optimization with machine learning  via model-based deep learning <cit.> to realize  uplink modular beamforming optimizers operating reliably with fixed and low latency. By doing so, we are able to utilize data to optimize the optimizer while maintaining its comprehensibility and transferability across different configurations. 

We focus on modular beamformers implemented using conventional phase shifters, and leverage the interpretable operation of the optimizer to (i) facilitate scalable design, where the same learned optimizer can be utilized with different numbers of modules; and (ii)  encourage power efficient designs for each panel. The latter is achieved by enabling deactivation and low-resolution implementation of the active phase shifters at each module. 
We formulate the modular uplink beamforming task as a constrained maximization of the sum-rate objective in multiple-access channels. We then specialize pga steps with momentum for the considered setting, being a suitable candidate accelerated iterative optimizer for directly optimizing the rate objective, based on which we develop our learning-aided optimizer. 

Our main contributions are summarized as follows:

    
  * First-order optimization for modular receive beamforming:
We formulate the receive modular beamforming task as a constrained optimization, and derive the corresponding first-order iterative optimizer based on pga steps. We specifically characterize the corresponding gradient steps and projection methods, which enable (lengthy) tuning of modular receive beamformers.
    
    
  * Rapid and scalable learned optimization: To enable rapid tuning, we convert the identified pga steps into a machine learning model operating with a fixed and low latency. This is achieved by fixing the number of iterations while using the hyperparameters of pga with momentum as trainable machine learning parameters. We show that the interpretable operation of the  resulting design allows it to be applied to different number of modules (which can be much larger than those used in training), thus facilitating scalability. 
    
    
  * Power-aware learned optimization: We  leverage the fact that the resulting machine learning model is comprised of interpretable pga iterations to incorporate two forms of power-aware considerations: the ability to boost sparse modular architectures with deactivated phase shifters, and the support of low-resolution phase shifters. As the resulting constraints render pga non-differentiable, we approximate its operation for learning purposes via surrogate projection mappings that enable its representation as a machine learning architecture.
    
    
  * Extensive experimentation: We extensively evaluate our learned optimizer for both synthetic channels as well as channels obtained from the physically compliant quadriga model <cit.>. Our numerical studies systematically show that learned optimization enables reliable tuning of modular beamformers with reductions of 10-30 × in iterations compared to conventional optimizers, and that the power-aware design enables efficient operation with only a minor degradation in the accuracy compared to unconstrained designs. 



The rest of this paper is organized as follows: Section <ref> presents the system model of uplink modular beamforming and formulates its design problem. Section <ref> presents our learned optimization method without accounting for power considerations, which are introduced into the method in Section <ref>. Empirical results are reported in Section <ref>, and Section <ref> concludes the paper.


Throughout this paper, we use  boldface lowercase and boldface uppercase letters for column vectors and matrices, respectively, while [M]_i,j is the (i,j)th element of a matrix M.
We use calligraphic fonts for sets, while ℤ and ℂ denote the integer and complex numbers, respectively. The operations (·)^T and (·)^H are used for transpose and conjugate transpose, respectively. I_N refers to the identity matrix of size N. The gradient of the function f(·) with respect to the matrix W is denoted by ∇_Wf(·), while δ_·,· is the Kronecker delta function, and ∡ (·) is the (element-wise) phase operator (in radians).








§ SYSTEM MODEL




In this section we introduce the system model for receive beamforming with modular hybrid mimo receivers. We commence with presenting the considered models for the communication channel and the receiver processing in Subsections <ref>-<ref>, respectively, after which we formulate the beamforming design problem in Subsection <ref>.



 §.§ Communication System



We consider a multi-band multi-user uplink mimo system. The setting involves K single antenna users that communicate with a (possibly distributed) mimo receiver,  over B frequency bins. The transmitted signals are received by M antenna elements, that are divided into multiple panels. The channel output at the bth frequency
is given by

    y[b]=H[b] s[b] + w[b],      b=1,…,B.

In (<ref>), H[b]∈ℂ^M× K is the channel matrix at frequency bin b, while  s[b] ∈ℂ^K represents the transmitted symbols, which are comprised of i.i.d. entries with power ρ_s>0. The signal w[b] ∈ℂ^M is awgn with variance σ_w^2>0.




 §.§ Modular Hybrid Receive Processing



The signal received by the M antenna elements is processed by a centralized cpu. The resulting digitally processed signal is obtained from the channel output in (<ref>) by undergoing modular receive beamforming, where the M antennas are divided into P panels. This processing, illustrated in Fig. <ref>, includes two stages: (i) Panel-wise beamforming, and (ii) Panel-cpu connectivity.



  §.§.§ Panel-Wise Beamforming

The first stage is composed of receive beamforming performed by each panel separately. Each panel has N antennas and L output ports, where the data processing takes place within the pth antenna panel and is described via a N× L frequency-invariant matrix W_p, for each p=1,…,P. The panels are assumed to be hardware limited, which we model using the set of feasible mappings W⊂ℂ^N× L, such that W_p ∈W. 

We consider the following types of analog constraints:

	
  *  Unconstrained Phase Shifters: Here, the analog processing is comprised of phase shifters, where the phase of component can be set arbitrarily, i.e., 
	
    W_ uph = {W∈ℂ^N × L: |[W]_n,l| = 1, ∀ (n,l)}.

	
	
  *   Sparsified Phase Shifters: As phase shifters are active components with non-negligible power consumption, a candidate approach to maintain power-efficient operation at the panels is to allow some of the phase shifters to be turned off <cit.>. This type of feasible set is represented as
	
    W_ sph = {W∈ℂ^N × L: |[W]_n,l| ∈{0,1}, ∀ (n,l) }.

	While the formulation in (<ref>) does not indicate how many phase shifters are inactive, such a requirement can be imposed by constraining the ℓ_0 norm of W to achieve a desired amount of inactive components. 
	

	
	
	
  *   Quantized Phase Shifters: An alternative  approach to limit the power consumption on the panel side is to restrict the phase shifters to support only a discrete set of feasible phases, i.e., support quantized phases shifts <cit.>. Such a constraint with Q-level quantized phases is modeled by the feasible set
	
    W_ qps^Q  = {W∈ℂ^N × L :       ∡[W]_n,l/2π· Q ∈ℤ ,
    	
       |[W]_n,l| = 1,  ∀ (n,l) }.


Note that for all considered feasible sets, the phase shifting components remain frequency-agnostic, i.e., W_p is invariant of the frequency index b. Moreover, we note that  approaches <ref> and <ref>, which aim to improve the power efficiency of the panels, are complementary. Accordingly, one can restrict the analog processing at the panels side to be both sparse and quantized, i.e., have some of the phase shifters deactivated, while the active ones take discrete phase values.


 



  §.§.§ Panel-CPU Connectivity

We model the connectivity to the cpu  via the binary matrix A∈{0,1}^L· P × T, with T being the number of cpu inputs. This matrix is also frequency invariant, and its entry [A]_l · p,t denotes whether the lth output of the pth panel is connected to the tth cpu input. 


The cpu input  z[b] is related to the channel output y[b] via

    z[b]=A^H W^H  y[b],

where W=  blkdiag{W_1,…,W_P}. This constraint comes from the decentralized structure of our communication system, which takes place within each panel. Note that when A is the identity matrix and L=1, this formulation specializes the common form of partially-connected hybrid beamformers <cit.>. 




 



 §.§ Problem Formulation



We aim to design the modular receive beamformer detailed in (<ref>) to maximize the achievable rate. To formulate this, we define the equivalent channel at the bth frequency bin as 


    G≜WA∈ℂ^M× T.

For the uplink setup detailed in Subsection <ref>, the average achievable sum-rate is given by (see, e.g., <cit.>)

    R(G; {H[b]}) = 1/B∑_b=1^B log|I_M +   ρ_s/N_0G(G^HG)^-1
       G^H
        H[b]H^H[b]|.




Using the rate expression in (<ref>), we can formulate the setup of modular receive beamforming as a constrained optimization problem. Since the matrix A represents the hardware connectivity between the panels, it is assumed to be constant. Accordingly, for a given channel realization {H[b]}_b=1^B}, the modular beamforming task is formulated as the recovery of 

    W^⋆   = max_W=  blkdiag{W_1,…,W_P} R(WA;  {H[b]}) 
       subject to W_p ∈W,   ∀ p∈{1,…,P}.


The problem as it is formulated in (<ref>) considers a generic setting for the feasible panel-wise  computation set W. As such, it accommodates both the unstructured setting <ref> as well as the power-oriented <ref>-<ref>. However, when required to operate with sparse panels <ref>, one is likely to also impose an additional constraint on the overall number of active phase shifters  in W. Similarly, imposing <ref> assumes that one operates with a fixed and known resolution dictated by Q.

 

In some cases (e.g., when T ≥max (⌊ MK-L/K+1 ⌋,K), B=1, and while allowing W = ℂ^N × L), one can identify the optimal setting <cit.>. Even in such cases,  tackling (<ref>) typically involves an iterative optimization procedures. In practice, the modular beamformer must be set within a coherence duration (which can be of the order of 0.1 milliseconds <cit.>). To account for this requirement, we limit the number of iterations for these calculations, allowing at most J iterations. To facilitate identifying an optimizer that can reliably tackle (<ref>) within J iterations, we assume access to a data set D composed of |D|>0 past channel realizations, i.e., 
 
    D={{H_i[b]}_b=1^B}_i=1^|D|.



 






§ UNCONSTRAINED UNFOLDED MODULAR BEAMFORMING


In this section, we study modular beamforming design while focusing on unconstrained phase shifters, i.e., constraint <ref>. The  unfolded modular beamforming algorithm, which is geared towards enabling rapid adaptation, serves as a starting point for considering the more restrictive constraints <ref>-<ref> in Section <ref>. In particular, we  first formulate our baseline optimizer of pga with momentum in Subsection <ref>. Then, we convert it into a machine learning model in Subsection <ref>, and provide a discussion in Subsection <ref>.





 §.§ PGA with Momentum



A direct approach to tackle (<ref>) employs first-order methods <cit.>, e.g., pga. This scheme starts from some initial guess W_0, and iteratively takes gradient steps towards the maximization of (<ref>), projected to meet the constraints. pga relies on the ability to compute the gradient of R(WA;  {H[b]}) with respect to W, denoted ∇_WR(W,A). The gradient of the rate is stated in Theorem <ref>.

The gradient of (<ref>) with respect to W is 
    
    
    ∇_W    R(W,A) =
            1/B∑_b=1^B
            (ρ_s/σ_w^2A((G^HG)^-1G^H 
       ×H[b]H[b]^H)
            (I +ρ_s/σ_w^2(G(G^HG)^-1G^H 
       ×H[b]H[b]^H))^-1(I-G    (G^HG)^-1G^H)) ^H.




    The proof is given in Appendix <ref>. 


In addition to the gradient in (<ref>), one should also be able to project onto the feasible constraints in (<ref>)  to formulate  pga.
For the non-power-constrained feasible set (<ref>), the projection operator for a matrix W based on <ref> is


    [P_W_ uph(W)]_i,j   = [W]_i,j/|[W]_i,j|·δ_⌊i/N⌋, ⌊j/L⌋.

In (<ref>), the element-wise operation divides each entry of the matrix W by its own magnitude, and then sets it to zero if it falls out of the main block diagonal. This effectively keeps each panel-wise processing made up of phase-shifters only.


Although first-order methods often require a large number of iterations to converge, they can typically be accelerated by introducing momentum. Momentum is also useful for dealing with non-convex objectives, as in (<ref>), since it can prevent getting trapped in local minima and saddle points. The resulting pga with momentum is summarized as Algorithm <ref>.










 §.§ Unfolded PGA with Momentum


Algorithm <ref> allows tackling (<ref>) while meeting the requirement to operate with J iterations. However, iterative optimizers such as pga are designed to tackle convex optimization problems and to operate with asymptotically large number of iterations. There, convergence can be guaranteed with fixed  hyperparameters {μ,β} (i.e., independent of the iteration index j), which can be tuned manually. In Algorithm <ref>, we apply this convex optimizer to a non-convex problem (<ref>) while operating with a fixed and small number of iterations J. In such cases, the  setting of the hyperparameters {μ_j,β_j} greatly affects performance <cit.>. This motivates leveraging the available  data in (<ref>) 
to tune these hyperparameters, by converting Algorithm <ref> into a discriminative machine learning model <cit.>, and proposing a dedicated learning mechanism. We next detail the resulting machine learning architecture, its proposed training procedure, and how it can be scaled to large modular settings.




  §.§.§ Architecture



Algorithm <ref> operates with exactly J iterations. It can thus be readily converted into a machine learning model via deep unfolding for learning hyperparameters <cit.>. Here, each iteration is treated as a layer in a multi-layer architecture, whose trainable weights are the hyperparameters. 

We increase the abstractness of the trainable architecture by allowing it to use different values of μ and β not just per iteration, but also per entry in W. Accordingly, the scalar values  {μ_j, β_j} that multiply matrix quantities in Algorithm <ref> are replaced with block-diagonal matrices α_j, β_j that multiply the matrix quantities in Algorithm <ref> element-wise.
The resulting architecture obtained from Algorithm <ref> is illustrated in Fig. <ref>, and has 2· J · P · N · L trainable parameters,  written as
 
    θ=  {α_j, β_j}_j=0^J-1.







  §.§.§ Training



Converting Algorithm <ref> into a machine learning model facilitates tuning θ  in (<ref>),
which are the hyperparameters of Algorithm <ref>. This is achieved using the dataset D in (<ref>), containing past channel realizations taken from the same communication system. Letting W_J(θ) be the modular beamforming matrix computed by Algorithm <ref> with parameters θ, we seek the hyperparameters 

    θ^⋆ = max_θ∑_{H[b]}∈D R(W_J(θ)A;  {H[b]}).


The objective in (<ref>) evaluates θ  based on the output of the unfolded architecture, i.e., W_J. To facilitate the learning of θ, we exploit the interpretability of the architecture, and particularly the fact that each layer should output a gradually improved modular beamformer. Accordingly, we propose a loss measure for optimizing θ that accounts for the output of the intermediate iterations (layers) in a monotonically growing fashion. Specifically, we use a loss function with logarithmic contribution growth between iterations <cit.>, given by

    ℒ_𝒟(θ) = -∑_{H[b]}∈D∑_j=1^Jlog(1+j)/|D| R(W_j(θ)A;  {H[b]}).

The resulting training algorithm using mini-batch stochastic gradient descent is stated as Algorithm <ref>.
The negative sign in the loss  is added because our objective is to maximize the rate, rather than minimize it.



  














  §.§.§ Transferability Across Configurations



Modular beamforming architectures are scalable,  stemming from the ability to achieve extremely large mimo configurations by adding panels. A core gain of our interpretable learning-aided optimizers is the fact that they support such scalability. Specifically,  what we learn are hyperparameters of an optimization algorithm, which can be applied to different configurations. This indicates that our unfolded optimizer shares this transferability across configurations, thus inherently supporting large and adaptive scale deployments.

In particular, the architecture and training procedure detailed above are formulated for a given modular configuration. Nonetheless, a learned optimizer is transferable, and a learned optimizer trained for a small configuration can be scaled to applied to much larger ones. Specific scalable parameters of interest are the numbers of users K and of panels P:

    
  * Scaling K: we note that the learned hyperparameters in (<ref>) are invariant of the number of users. Thus, a learned optimizer trained for a small number of users can be readily applied to a setting with a large number of users.
    
  * Scaling P: While the setting of the hyperparameters in (<ref>) depends on the number of hyperparameters, as we allow panel-specific hyperparameters, one can scale by reusing hyperparameters across panels <cit.>. For instance, a learned optimizer trained to set {α_j, β_j} for a configuration with P panels can be applied to a setting with c· P a positive integer c, by e.g., setting its hyperparameters to be {I_c ⊗α_j, I_c ⊗β_j }, where ⊗ denotes the Kronecker product, or by assigning the added panels an average of the learned hyperparameters.

The above approaches to transfer a learned optimizer across different configurations are empirically demonstrated to yield effective and scalable optimization in Subsection <ref>.















 §.§ Discussion



The proposed unfolded pga  is designed to learn from data how to traverse the optimization loss surface, allowing to achieve improved sum-rate within J iterations, as shown in Section <ref>.
The resulting learned θ, which represents the per-iteration hyperparameters used for setting the modular beamformer, may exhibit certain counter-intuitive characteristics, such as negative gradient steps, as means of coping with non-convexity. As the resulting algorithmis a learned implementation of a principled optimizer, we share its scalability, which is highly desirable in modular architectures. 

While our formulation considers panels comprised phase-shifters, one can consider alternative hardware, such  as distributed active metasurfaces <cit.> and phase-time arrays <cit.>. 




The fixed and limited number of iterations impacts the computational complexity of setting the modular beamformers for a given channel realization. To quantify this complexity, we examine a single iteration of Algorithm <ref>. The computational burden is mainly dominated by the gradient computation step, formulated in (<ref>). This computation has a complexity order (in complex multiplications) of

    𝒪(B(T^3 + M^2(T + K)) + M (T^2 + LPT + TK)).

The remaining steps in the algorithm include the update step and the projection step (<ref>). The complexity order of these steps is  𝒪(MLP) for both, which is negligible compared to the gradient computation step. Considering that optimization procedure  operates over B frequency bins, the overall complexity of the PGA+M algorithm with a predefined J iterations, as implemented by the proposed learned optimizer, is of the order of

    𝒞_PGA+M = 𝒪(  JB(   T^3 + M^2(T + K) 
     
        + M (T^2 + LPT + TK))).


The asymptotic complexity characterization in (<ref>) is comprised of multiple summands, where in general none of the terms can be consistently ignored. Nonetheless, in an expected massive modular mimo setting, where the overall number of antenna elements is larger than both the number of users (K) and the number of cpu inputs (T), the dominating term is the one growing quadratically with M, and thus  𝒞_PGA+M≈𝒪(JBTM^2). 
When delving further into the analysis, it becomes evident that the dominant terms (in complex products) arise from matrix multiplication operations.
However, these computations can often be parallelized to reduce computational complexity, making the complexity of matrix inversion, represented by the 𝒪(JBT^3) term, the dominant factor. While it is possible to further reduce the number of multiplication operations through techniques such as sparse and low-rank approximations of the gradients, we leave this aspect to future research.








§ POWER-AWARE UNFOLDED MODULAR BEAMFORMING




Our analysis in Section <ref> focused on the unconstrained setting <ref>.
In this section, we use the unconstrained design for maximizing the rate (<ref>) as a starting point, on top of which we develop a power-aware design. Specifically, we leverage the interpretability of the solution described in Section <ref> to incorporate constraints <ref>-<ref>. We focus on conventional (non-learned) gradient-based optimization subject to  <ref>-<ref> in Subsections <ref> and <ref>, respectively. Then, we  formulate  pga with momentum and the unfolded algorithms tailored for these designs in Subsections <ref> and <ref>, respectively, and conclude with a discussion in Subsection <ref>.





 §.§ Sparsified Modular Beamformer




As detailed in Section <ref>, power efficiency can be enhanced by turning off some of the phase shifters.
Supporting deactivated phase shifters can be mathematically accommodated by constraint <ref>, i.e., as promoting  sparsity of the matrix W. 
To meet this constraint, we rewrite our constrained optimization from (<ref>) as 


    W^⋆   = max_W=  blkdiag{W_1,…,W_P} R(WA;  {H[b]}) 
       subject to  ||W||_0 ≤ S_max
                 W_p ∈W_ sph,   ∀ p∈{1,…,P},

where S_max≤ N· L · P represents  the maximal allowed number of active phase shifters. 
In (<ref>), each phase shifter can be either active or turned off (by (<ref>)). The ℓ_0 constraint guarantees that at most S_max phase shifters are active.

Due to the inherent challenges associated with gradient-based optimization under ℓ_0 constraints, we adopt the common approach of relaxing it into an ℓ_1 constraint (see, e.g., <cit.>). The relaxed optimization problem is 

    W^⋆   = max_W=  blkdiag{W_1,…,W_P} R(WA;  {H[b]})  - λ ||W||_1 
       subject to W_p ∈W_ sph,   ∀ p∈{1,…,P}.

In (<ref>),  λ>0 is an objective regularization coefficient <cit.> whose purpose is to encourage the solution to hold the sparsity constraint in (<ref>). Setting the exact value of such hyperparameters to meet a desired sparsity level S_max is typically elusive, and involves repeating numerical trials to identify the exact value that strikes a desired trade-off between performance (i.e., rate) and sparsity. Nonetheless, once λ is fixed, the gradient of the relaxed objective in (<ref>) can be computed as 

    ∇_W(R(WA;   {H[b]}) -  λ||W||_1) =
       ∇_W R(WA; {H[b]}) - λ· e^j∡W.

where ∇_W R(·) is given in Theorem <ref>.







Relaxing (<ref>) into (<ref>) enables computing the objective gradients via  (<ref>). In order to enable pga-based optimization, one should also define the corresponding projection on the feasible set 𝒲_ sph. To that end, we adopt a flexible extension of the projection operator in (<ref>) to account for deactivated phase shifters, by introducing a magnitude threshold ζ>0. The resulting projection step is given by 

    [𝒫_𝒲_ sph(W)]_i,j=
    [W]_i,j/|[W]_i,j|·δ_⌊i/N⌋, ⌊j/L⌋    |[W]_i,j| ≥ζ, 
    
    0     |[W]_i,j| < ζ.




Projection via (<ref>) is a crucial  step in the tackling the relaxation of (<ref>), ensuring that we set some elements of W to 0 and constrain their magnitude to be either 0 or 1, as described in <ref>.
At the same time, this step preserves the block diagonal structure of W. A candidate choice of ζ is ζ = 0.5; Setting ζ = 0 makes this projection step coincide with the projection step in (<ref>).





 §.§ Quantized Modular Beamformer



A complementary approach to mitigate power consumption of analog modular processing is by discretizing the phase resolution with a few bit representation. Incorporating the resulting constraint <ref> into the rate maximization objective (<ref>), yields the resulting constrained optimization formulation

    W^⋆   = max_W=  blkdiag{W_1,…,W_P} R(WA;  {H[b]}) 
       subject to W_p ∈W^Q_ qps,   ∀ p∈{1,…,P}.

In (<ref>), the set of feasible phase shifts is restricted to take Q quantization levels, using the definition of W^Q_ qps in (<ref>). 

We note that the objective in (<ref>) is the same as in the unconstrained case (<ref>), and thus its gradient can be obtained from Theorem <ref>. Consequently, to formulate the suitable pga steps for tackling (<ref>), one must specify the quantization-aware projection operator. A direct approach to guarantee discrete phase components is by setting each phase element into its nearest discrete value, resulting in 

    [𝒫_𝒲_ qps^Q(W)]_i,j   = e^j ϕ_i,j^⋆·δ_⌊i/N⌋, ⌊j/L⌋
    where ϕ_i,j ^ ⋆   = min_ϕ/2π· Q ∈ℤ 
    |ϕ - ∡[W]_i,j|.

The projection operator in (<ref>) can be viewed as uniform scalar quantization of the phase of each element of W. 





 §.§ Power Aware PGA





Here we present a baseline pga algorithm for the updated set of constraints, as described in <ref> and <ref>. 
This pga algorithm is an extension of Algorithm <ref>, adapted to the power aware objectives described in Subsections <ref> and <ref>.

The resulting formulation is summarized
as Algorithm <ref>. There, we integrate the power-aware gradient formulation (<ref>) with the projection steps (<ref>) and (<ref>) to realize pga.



Algortihm <ref> is designed to tackle the maximization problems outlined in (<ref>) and (<ref>) by proper settings of the projection operator and the regularization coefficient. For instance, by setting the projection to be given in (<ref>) and setting λ = 0, Algorithm <ref> tackles problem (<ref>). Similarly, by using λ > 0 and setting the projection to be (<ref>), it tackles problem (<ref>).

 







 §.§ Unfolded Power Aware PGA



In Subsection <ref>, we showed how data can be leveraged to tune the hyperparameters of Algorithm <ref> to enhance its performance within J iterations, by converting it into a machine learning architecture.

Here, we follow this methodology to convert the power-aware Algorithm <ref> into a trainable architecture. As opposed to the unconstrained case, where the learned hyperparameters were solely associated with the solver, i.e., with pga, in the following we also leverage data to tune parameters associated with the optimization objective <cit.>, and particularly the (typically elusive) regularization coefficient λ. 




  §.§.§ Architecture

Following the conversion of pga with momentum into a machine learning model  in Subsection <ref>, we view the iterative algorithm as a multi-layer architecture. This allows using the  data (<ref>) to learn  hyperparameters that yield desirable settings within a few iterations. 

 Algorithm <ref>  introduces another hyperparameter, λ, associated with optimization objective, and particularly the ℓ_1 norm reduction. Here, we increase the abstractness of the trainable architecture by allowing λ to differ from one iteration to another, i.e., have each iteration taking its gradient step over a different loss surface. Furthermore, we turn the scalar regularization coefficient into a matrix with different values for each entry in W. We add these learnable matrices {λ_j}_j=0^J-1 to the set of trainable parameters θ. Accordingly, we now have  3 · J · P · N · L trainable parameters, written as

    θ=  {α_j, β_j, λ_j}_j=0^J-1.


The power aware architecture is visualized in Fig. <ref> for a given learned hyperparameters θ. For clarity of presentation, in Fig. <ref> we separate the gradient step in Algorithm <ref> into two separate steps, each one dependent on a different hyperparameter.










  §.§.§ Projection

A key issue in deep unfolding is maintaining the  forward path differentiable. This allows the optimizer to be represented as a  trainable architecture that can be tuned with deep learning techniques based on  backpropagation <cit.>. Algorithm <ref> mainly differs from Algorithm <ref> in the projection step. The projection step in both (<ref>) and (<ref>) contains continuous-to-discrete mappings that nullify gradients. 


To overcome this, we distinguish between the projection step utilized when applying the optimizer, and a  differentiable surrogate utilized for evaluating gradients during training. Specifically, we use the following surrogate projections:

    
  * Approximating 𝒫_𝒲_ sph: The projection onto 𝒲_ sph can be viewed as a step function. Accordingly, we approximate it during training using a harp and shifted sigmoid function <cit.> denoted 𝒫̂_𝒲_ sph, given by
    
    [𝒫_𝒲_ sph]_i,j = |W_i,j| = σ (s_m (|W_i,j| - ζ_m)).

    In (<ref>) σ denoting the sigmoid function, s_m denotes a sharpness hyperparameter, and ζ_m stands for the shift, imitating the threshold ζ from (<ref>).

    
  * Approximating 𝒫_𝒲^Q_ qph: For phase approximation with Q  quantization levels, we note that the projection operator is a piecewise-constant monotonically increasing function. Following <cit.>, we  approximate the quantization levels with sum of Q differently shifted sigmoid functions. The resulting phase projection sets ϕ_i,k^⋆ in (<ref>) to be 

    ϕ̂_i,k^⋆= 2π/Q∑_q=1^Q σ(s_p(∡W_i,j - 2π (q-0.5)/Q + π)),

where s_p is a sharpness hyperparameter.



This above formulations  approximate the  projection operators in a differentiable manner, ensuring smooth gradients during training while maintaining the characteristics of the original projection step.
 The ability of (<ref>)-(<ref>)  to approximate their corresponding hard (non-differentiable) projection operators is illustrated in Fig. <ref>.















  §.§.§ Loss Function


We assume access to a dataset D, which comprises past channel realizations.
This data is leveraged to learn the hyperparameters for solving the following constrained optimization problem:


    θ^⋆ =  max_θ∑_{H[b]}∈D R(W_J(θ)A; {H[b]}) 
       subject to: W_J=  blkdiag{W_J_1,…,W_J_P}
       (i)     ||W_J||_0 ≤ S_max,   W_J_p∈W_ sph,   ∀ p∈{1,…,P}. 
       (ii)   W_J_p∈W_ qps,   ∀ p∈{1,…,P}.

 
The constraints on W are either  (i) or (ii), depending on the solved problem, (<ref>) or in (<ref>), respectively. 


The learning objective in (<ref>) under (i) encapsulates a key difference from  the unconstrained one  (<ref>).  Specifically, for sparse phase shifters, the learning objective stems from (<ref>), namely, maximizing R under the ℓ_0 norm constraint. When formulating the resulting pga iterations, we relaxed ℓ_0 norm to the ℓ_1 norm, using the coefficient λ to balance between rate maximization (R) and  sparsification. To translate this into a suitable loss for training that encourages a desired ℓ_1 norm, we introduce the following training loss 


    ℒ_𝒟(θ) =  -1/|D|∑_{H[b]}∈D   ∑_j=1^Jlog(1+j) R(W_j(θ)A;  {H[b]}) 
       - γ(W_J(θ)_1 - S_max)^2,

where γ≥ 0 is a regularization hyperparameter. The loss (<ref>)  is also used with quantized phase shifters, i.e., under (ii), by setting γ=0, as constraint (ii) is guaranteed to hold when using the projection operator 𝒫_𝒲^Q_ qph in modular beamforming. 

The updated loss function (<ref>) is an extension of the unconstrained loss function (<ref>), adding the squared error between the ℓ_1 norm of W_J and S_max. 
The rationale stems  from our desire to encourage W_J_1 (being a differentiable approximation of W_J_0) to approach S_max without having to tune different regularization coefficients for different S_max. This allows training via, e.g., mini-batch stochastic gradient descent, as in Algorithm <ref>.












 §.§ Discussion



The design of the constrained modular beamformer via Algorithm <ref> is based on the pga steps designed for the unconstrained setting in Algorithm <ref>, leveraging its interpretable operation by introducing dedicated projections. Despite the similarity in inference, the procedure involved in learning the hyperparameters from data substantially differs from the unconstrained case, necessitating differentiable approximations of the projection operators during learning, altering gradient computation, and a dedicated training loss. This allows learning to rapidly set constrained modular beamformers with only a minor degradation in the rate, as shown in Section <ref>.

Once the hyperparameters are learned through the unfolded learning procedure detailed in Subsection <ref>, the setting of the modular beamformer in Algorithm <ref> follows the unconstrained pga in Algorithm <ref>. The key differences are the additional phase term in the gradient computations via (<ref>), and the modification of the projection operators via (<ref>) or (<ref>). As none of these operations dominates the computational burden, the complexity of the power-aware design is similar to that of the unconstrained case, analyzed in Subsection <ref>. 

Our power-aware design boosts power efficiency by supporting deactivated and low-resolution phase shifters. The translation of these properties into  power savings is highly dependent on the hardware. We opt for a parametric representation of the desired sparsity level and phase shifter resolution, allowing the unfolded optimizer to be useful for different implementations and different power levels by setting of the corresponding parameters, e.g., S_max and Q. While our learning procedure utilizes the projection surrogates visualized in Fig. <ref>, one can consider alternative differentiable approximations of such continuous-to-discrete mappings, see, e.g., <cit.>. Moreover, additional methods to boost power efficiency by, e.g., deactivating panels rather than phase shifters, can be obtained as extensions to our framework, which are left for future study.




















§ EXPERIMENTAL STUDY



In this section, we numerically evaluate the proposed unfolded algorithm for rapid tuning of uplink modular beamforming[The source code and hyperparameters are available at <https://github.com/levyohad/Power-Aware-Deep-Unfolding-for-Beamforming>]. We describe the experimental setup in Subsection <ref>, while our results for unconstrained and power-aware settings are reported in Subsections <ref>-<ref>, respectively.



 §.§ Experimental Setup


We simulate a communication system, where the channel matrices {H[b]}_b=1^B are obtained from a Rayleigh fading distribution, as well as physically compliant realizations obtained from the quadriga model <cit.>. We treat the number of iterations J as a pre-defined requirement, and train our algorithm to minimize its loss within this number of iterations.

The snr is defined as  1/σ_w^2.


We compare our unfolded optimizer (U-PGA+M) with the following benchmarks: 
 
    
  * PGA + M: running pga with momentum (Algorithm <ref>) with fixed hyperparameters {μ,β}, ∀ j =1 … J, chosen from a discrete grid, to maximize the average R after J=500 iterations for the entire training set. 

    
  * Line Search:   pga where the hyperparameters {μ_j,β_j}_j=0^J-1 are selected for each iteration through grid search to maximize the achievable rate R at each step <cit.>. This method performs an exhaustive search to find the optimal step size for each iteration, independently for each data sample, at the cost of excessive additional processing latency. 
    
    
    
    
  * Manifold Optimization (MO): The optimizer proposed in <cit.> for centralized architectures, adapted to support   modular beamformers. It suggests a relaxed version of the maximization problem (<ref>), taking gradient steps of W toward the fully digital beamformer G_ opt, trying to minimize the error ||WA - G_ opt||^2, and projecting the resulted matrix W on the feasible set, using P_W(·). As in the Line Search benchmark, it performs an exhaustive line search to choose the best step size for each iteration. 
   
    
  * cnn: A cnn based on the architecture of <cit.>, trained to map channel realizations {H[b]}_b=1^B into the phases of W that maximize the channel-rate R. This approach is extremely parameterized. 
    For example, in a system with K=7 users, M=8 antennas, and B=4 frequency bins, the  cnn  consists of 274088 learned parameters, whereas U-PGA+M has only 960 parameters.

 



 §.§ Unconstrained Modular Beamformers


We start the experimental study by considering unconstrained modular beamformers, as described in Subsection <ref>. Our evaluation first focuses on small scale mimo settings, after which we assess transferability to larger systems.



  §.§.§ Small-Scale Hybrid Modular mimo

We first examine two scenarios representing relatively small-scale settings. In Scenario 1, we set (T=5, P=2, L=4, N=20, B=2, K=20), and the channel matrices obey Rayleigh fading. Scenario 2 uses quadriga channels with (T=5, P=4, L=2, N=3, B=4, K=5).  

We use these scenarios to evaluate the ability of our unfolded design to improve the sum-rate achieved within J=10 iterations. To that aim, we depict in Figs. <ref>-<ref> the average sum-rate vs. the iteration index j for Scenarios 1 and 2, respectively.





For fair comparison, we set all optimization-based algorithms to have J=10 iterations, with the cnn having 10 layers, and compare the sum-rate to that achieved by PGA+M  with J=500 iterations. We consistently observe that in the considered scenarios with a limited number of iterations, our unfolded pga outperforms all benchmarks, including the costly and greedy line search hyperparameter setting, and the highly parameterized cnn benchmark. The latter particularly struggles in Rayleigh channels, as  cnns tends to capture spatial relationships, and such relationships do not exist under Rayleigh fading. Similarly, the MO optimizer exhibits slow convergence due to the additional modular structure imposed in our setting of modular beamformers. 


The results after  J iterations illustrated in Figs. <ref>-<ref> are also reported in Table <ref>, along with  latency of the various methods. The latter is evaluated for all algorithms on the same platform (Google Colab premium). 

We notice in Table <ref> that the unfolded algorithm is 30-50 times faster than optimizers that employ line search or run a large number of iterations, while achieving a comparable sum-rate.  
The cnn benchmark has the least latency here, owing to the efficient implementation and natural support of such architectures in platforms such as Google Colab.
This efficiency persists across multiple scenarios, emphasizing our ability to provide reliable modular beamformers with minimal latency, that can be further enhanced by employing dedicated hardware accelerators.















Figs. <ref> and <ref> show the resulted sum-rate for each benchmark in a wide range of snr values, for Scenarios 1 and 2, respectively. 
There, we observe that the gains obtained by the unfolded algorithm while using the same number of iterations, are translated into snr gains of 1.25 - 1.5 dB compared to the pga + M algorithm with the same J iterations.




  §.§.§ Transferable Modular Structures Simulation Results


As explained in Subsection <ref>, one of the key advantages of our learn-to-optimize modular beamforming design is the transferability of learned parameters when scaling the system. Modular structures allow us to train the system with specific settings and then scale it up to a larger number of users or panels without needing to retrain the model. 

Scaling the number of users:
To demonstrate this, we first evaluate U-PGA+M that was initially trained with a small number of users, K=5, and then deployed in a system with a large number of users, K=50, using the same hyperparameters.  This example illustrates the transferability of the unfolded model.
For this evaluation, we introduce Scenario 3: (T=5, P=8, N=4, B=2, K=50) and Scenario 4: (T=5, P=8, N=4, B=2, K=5). In both scenarios the channel realizations are sampled from a Rayleigh fading distribution.
Fig. <ref>, presents the sum-rates achieved by the various benchmarks  on Scenario 3, along with the performance of the U-PGA+M algorithm, which was trained on Scenario 4. These results demonstrate the flexibility of the unfolded algorithm and underscore its advantages in adapting to rapidly changing environments. The average rates at snr of 0  are also reported in Table <ref>.




Scaling the number of panels:
Next, we provide an example of scaling up the number of panels. In this example, we use c = 8 as the scaling coefficient, and increase the number of panels from P to c· P, using {α_j, β_j } = {I_c ⊗α_j, I_c ⊗β_j}.
To this end, we introduce Scenario 5: (T=5, P=2, N=4, B=2. K=7) and Scenario 6: (T=5 · 8, P=2 · 8, N=4, B=2, K=7); both  obey Rayleigh fading.




Fig. <ref>
illustrates the results of various benchmarks trained on Scenario 6, alongside the performance of the U-PGA+M algorithm that was trained on Scenario 5. These results highlight the flexibility of our design, demonstrating how it can be initially trained on a smaller network, perform straightforward calculations, and then be effectively scaled to much larger and more complex systems.





 §.§ Power-Aware Modular Beamformers


We proceed to evaluating the ability of our unfolded optimizer to encourage power-efficient modular beamformers. We divide our evaluation to boosting efficiency via deactivating phase shifters, and via using low resolution phase shifters.



  §.§.§ Sparse Modular Beamformers

We evaluate the effect of designing modular beamformers with deactivated phase shifters by boosting sparsity in W. We focus on Scenario 2,  and apply U-PGA+M using Algorithm <ref>  utilizing the projection step  P_W(·) in  (<ref>). We set the parameter S_max to be 0.75 · N · L · P. 
As the benchmarks detailed in Subsection <ref> do not naturally account for such constraints, we compare here U-PGA+M to unconstrained settings that are based on pga, possibly with line search and/or large number of iterations. 





The resulting rates vs. iteration compared to those achieved without such power-aware design are reported in Fig. <ref>. We observe that by applying Algorithm <ref>, one achieves a 25 % reduction in active components while maintaining a comparable sum-rate and identical running time to unconstrained settings. 
The unfolded algorithm learns to rapidly compensates for the inactive components by tuning the active phase shifters.
Fig. <ref> further highlights the advantages of the unfolded power-aware algorithm by evaluating the resulting sum-rate over different snr. There, we observe that our power-aware design not only allows accurate operation with 25% of the phase shifters deactivated, but it also offers approximately 1 dB improvement in snr over the pga + M benchmark  with the same number of iterations.













  §.§.§ Low-Resolution Modular Beamformers

We conclude our numerical study by evaluating the effect of using quantized phase shifters, and the ability of our power-aware design to account for such constraint when tuning modular beamformers. 
We consider the power-aware  Algorithm <ref>, utilizing the projection step for P_W(·) as specified in (<ref>), when the phase shifters can be configured to merely 4 bits in phase, i.e.,  Q=16. 
The benchmarks detailed in Subsection <ref> are adapted to produce low-resolution phase shifters by altering the projection step (for optimization methods) or quantizing the produced phases (for the cnn).  

The resulting sum-rate vs. iteration and vs. snr for Scenario 1 are illustrated in Figs. <ref> and <ref>, respectively.
The results demonstrate that the proposed unfolded algorithm, which is particularly geared to accounting for low-resolution phase shifters, significantly outperforms other benchmarks, including both slow iterative methods that are not trained as machine learning models, and suffer more notable degradation due to quantization, as well as the  heavily parameterized cnn.

Specifically, while pga + M struggles to overcome the quantization constraints using gradient steps, and tends to converge to a local maximum, the unfolded algorithm effectively learns to traverse the loss surface and projection constraints, achieving a better sum-rate. As shown in Fig. <ref>, the unfolded algorithm consistently performs better across various snr values compared to all other benchmarks.






















§ CONCLUSIONS




In this work, we investigated rapid beamforming design for uplink modular mimo systems under three different sets of constraints: unconstrained phase shifters, sparse phase shifters, and quantized phase shifters. Our proposed algorithm, a momentum-aided gradient-based optimization of the sum-rate objective, boosts power efficient settings while operating  with a fixed and small number of iterations. By transforming the optimizer into a lightweight, trainable model, we harness data to fine-tune its hyperparameters, enhancing its adaptability to various system conditions. Our experimental results demonstrate a significant improvement in the achieved sum-rate compared to benchmarks operating within the same computational-time constraints, while also surpassing other benchmarks in hardware-aware and power-aware designs. 








lemmasubsection 
corollarysubsection 
remarksubsection 
equationsubsection	



 §.§ Proof of Theorem <ref>


For ease of presentation, we define the following auxiliary variables: 
G^†≜ (G^HG)^-1G^H,
P≜GG^†, 
Z[b] ≜PH[b]H^H[b],
and Q[b] ≜I_k + ρ_s/σ_w^2Z[b].
Using these symbols, the objective with respect to which the derivative is taken can be written as

  R(G; {H[b]}) =  1/B∑_b=1^B log|Q[b]|.    


Plugging-in auxiliary variables, we aim at proving Theorem <ref> by  showing that 


    ∇_W R(W,A) = 
           1/B∑_b=1^B
        (ρ_s/σ_w^2AG^†H[b]H[b]^H
        Q^-1
        (I-P)) ^H.

The representation in (<ref>) can be derived using the differential method for matrices as described in <cit.>.
In the following proof the sign : denotes the Frobenius inner product, and the derivations are done using chain rule. For brevity we include our derivation for a single frequency bin.


Since R =log|Q| when B=1, it holds that the differential can be expressed as

    dR =   (Q^-1)^H  :  dQ
         (a)=(Q^-1)^H  : ρ_s/σ_w^2 dZ
    (b)=   (ρ_s/σ_w^2Q^-1)^H  :  dZ
         (c)=(ρ_s/σ_w^2Q^-1)^H  :  dP HH^H
    
         (d)=   (ρ_s/σ_w^2 HH^HQ^-1)^H  : (I - P)   dG G^†

where (a) stems from d Q = 0d I_K + d ρ_s/σ_w^2Z = ρ_s/σ_w^2 d Z, 
(b) and (d) are based on the Hermitian property  inner product, and (c) stems from d Z = d PHH^H, as H is independent of W.

Next, we substitute  P, G^†, and G using chain rule:


    dR =    (ρ_s/σ_w^2  (HH^H)Q^-1)^H  : (I - P)   dG G^†.


Equation (<ref>) can be explained by recalling that P≜G (G^H G)^-1G^H, and thus 

    dP   = dG (G^H G)^-1G^H + G d(G^H G)^-1G^H  
                + G(G^H G)^-10dG^H
    
            = dG G^† - G (G^H G)^-1 (0dG^HG + G^H dG) G^†
       = dG G^† - G (G^H G)^-1G^H dG G^†
       = (I - P)   dG G^†.

In (<ref>), the term dG^H vanishes since G^H is independent of G, and thus independent of W.
Once (<ref>) is achieved, the rest of the proof becomes straight forward, using the chain rule until (<ref>) is reached, i.e., 

    dR =   (ρ_s/σ_w^2 G^†HH^HQ^-1 (I - P))^H  :   dG
     
         =   (ρ_s/σ_w^2 G^†HH^HQ^-1 (I - P))^H  :   dWA
     
        =   (ρ_s/σ_w^2 AG^†HH^HQ^-1 (I - P))^H  :   dW ,

which concludes that ∇_W R(W,A) = (ρ_s/σ_w^2AG^†HH^HQ^-1(I-P)) ^H, thus proving the theorem.


 



IEEEtran




 Paper Title: RF Power Transmission for Self-sustaining Miniaturized IoT Devices.












Journal of  Class Files, Vol. 14, No. 8, August 2015
Shell et al.: Bare Demo of IEEEtran.cls for IEEE Journals

RF Power Transmission for Self-sustaining Miniaturized IoT Devices
    

Lukas Schulthess1, 
Federico Villani1, 
Philipp Mayer1, 
Michele Magno1
1Dept. of Information Technology and Electrical Engineering, ETH Zürich, Switzerland 


    August 5, 2024
=====================================================================================================================================================================



§ ABSTRACT 
Radio Frequency (RF) wireless power transfer is a promising technology that has the potential to constantly power small Internet of Things (IoT) devices, enabling even battery-less systems and reducing their maintenance requirements. However, to achieve this ambitious goal, carefully designed RF energy harvesting (EH) systems are needed to minimize the conversion losses and the conversion efficiency of the limited power. For intelligent internet of things sensors and devices, which often have non-constant power requirements, an additional power management stage with energy storage is needed to temporarily provide a higher power output than the power being harvested. 
This paper proposes an RF wireless power energy conversion system for miniaturized IoT composed of an impedance matching network, a rectifier, and power management with energy storage. The proposed sub-system has been experimentally validated and achieved an overall power conversion efficiency (PCE) of over 30 for an input power of -10dBm and a peak efficiency of 57 at 3dBm.











Wireless energy transfer, RF energy harvesting, battery-less devices, self-sustaining sensors.




§ INTRODUCTION



Wireless power transfer (WPT) is an emerging technology set to supply wireless devices in the future generation of self-sustaining and battery-free Internet of Things (IoT) and wireless sensor networks (WSN) <cit.>. 
RF transmission is widely used for wireless data transfer and wireless communications. 
Additionally, RF waves can also carry energy together with information. 
Due to the recent improvement in power consumption of electronics, leading to miniaturization and reduction of power consumption of IoT devices, WPT is increasingly emerging as a potential and feasible solution as power supply technology for them <cit.>.

Wireless power transfer has several advantages over environmental energy harvesting for the supply of IoT devices <cit.>. 
The main advantage is the controllable nature of wireless power transfer versus the unpredictable nature of environmental energy harvesting <cit.>. 
In fact, instead of harvesting energy for environmental sources, WPT technology is designed to exploit a stable and controllable wireless power supply by deploying a dedicated power transmitter that periodically or continuously sends energy
<cit.>. 
Moreover, RF WPT allows long-distance energy delivery to a multitude of IoT nodes and enables a small form factor at the receiver with compact receiving antennas, especially when compared to other technologies such as inductive coupling or magnetic resonance.
For these reasons, the interest for RF WPT has increased in research and industry in recent years <cit.>.
The vision for this emerging technology is to permanently power low-power devices, eliminating the need for batteries or increasing their lifespans and reducing maintenance without loss of performance <cit.>. 
This can be especially attractive in situations where a periodical change of batteries is difficult or not desired. For these reasons, powering the sensors through wireless power sources provides an effective alternative to other energy harvesting solutions such as solar, thermal, or vibration <cit.>.


WPT is mainly divided into near- and far-field WPT depending on the spatial range of the transfer. 
Near-field inductive WPT is used to wirelessly power many commercially available near-field communication (NFC) devices such as credit cards and smartphones and is widely adopted <cit.>.
However, its range is limited to few a centimeters, due to the mainly inductive transfer of energy and data in the near-field. 
On  the contrary, far-field WPT is based on sending and receiving electromagnetic (EM) waves using dipoles, which allows for power transmission at longer ranges of over 1 meter <cit.>. 

This makes far-field WPT more flexible and particularly suitable for power multi-casting, allowing the transmitter/receiver to move around, even in Non-Line-of-Sight (NLoS) environments <cit.>. 
Support of longer ranges is crucial to supply energy to IoT devices in a wide range of applications <cit.>. 
As a result, there has been a fair amount of research dedicated to the design of dedicated WPT transmitters for these sensor systems <cit.>. 
Despite this, far-field RF-WPT over longer ranges is still very challenging. One of the major challenges is the efficient transformation of the input EM power levels smaller than -10dBm, to the voltage and current demands of the load <cit.>. This is exacerbated by the need to use a miniaturized antenna and compact designs for small and unobtrusive  IoT devices <cit.>.


RF WPT is also increasing its popularity due to the availability of novel commercial products. For instance, Powercast allows power transfer up to 200 using a single narrow-band sub-GHz channel over several meters. 
Having power available at this range, combined with the possibility to supply several receivers from a single transmitter, makes RF WPT a promising power source for low-power IoT devices <cit.>. 
However, it must be taken into account that the effective power available at the IoT nodes is strongly affected by the distance, antenna size, RF matching, and rectification efficiency, and will therefore be orders of magnitude below the transmitted power of 200. Therefore, to efficiently extract power from EM waves, a carefully designed RF EH system is of utmost importance.




This paper presents the electronic blocks of an energy-efficient RF WPT sub-system for miniaturized IoT and focuses on critical design considerations for each part. The design and evaluation of an optimized RF power rectifier and matching circuit for 915 for input power levels ranging from -12dBm to 9dBm is presented. 
Experimental results demonstrate that the optimized design increases the conversion efficiency by up to 20 compared to off-the-shelf circuitry. Finally, the proposed sub-system, but also other commercially available RF rectifiers and power management systems are exploited to maximize the PCE. 
Experimental results show that is possible to achieve an overall PCE of over 30 for an input power of -10dBm which is above the state of the art for such systems.










































































§ ARCHITECTURE
 


The goal of a WPT sub-system is to extract power from electromagnetic waves received by the antenna as efficiently as possible to achieve power transfer over longer distances. Eventually, the captured electromagnetic energy is converted into a usable direct current (DC) voltage.
To achieve this, each of the three main blocks of the EH system (ref. Fig. <ref> (a)), 
impedance matching network, rectifier, and power management with energy storage requires a carefully considered design.




 §.§ Impedance Matching Network

To achieve maximal power transfer between the source (antenna) and the load (rectifier), a carefully designed impedance matching network (IMN) is essential. By transforming the low impedance of the antenna to the system impedance, power reflections can be mitigated. Choosing discrete components with a high-quality factor minimizes power dissipation and further improves the IMN's conversion efficiency <cit.>. 


 §.§ Rectifier

When designing a high efficient RF-to-DC rectifier, selecting appropriate diodes is one of the most important factors since they are the main source of loss and their characteristic determines the overall performance of the rectifier.
Zero-bias Schottky diodes provide very low threshold voltage and junction capacitance in comparison to common diodes, making them well-suited for RF rectifiers and allowing efficient operation <cit.>.



 §.§ Power Management with Energy Storage

After rectification, the harvested RF power is available as DC power and can be already used to drive a load. However, if the load's power needs cannot be guaranteed at all times, which is especially true for miniaturized IoT devices, power management with energy storage is needed <cit.>. By applying strict power-saving techniques to the IoT devices, the system can still achieve self-sustainability <cit.>.


The proposed WPT sub-system, illustrated in Fig. <ref> (b), is optimized for a single narrow-band RF power transmitter with a center frequency of 915 and is realized using off-the-shelf components. 
High-Q capacitors and inductors of the series GJM1555 and LQW18AN keep the conversion losses at a minimum. 
C1 is a DC block to allow the rectifying circuits' DC bias to settle at mid-rail voltage. 
C2 and L1, together with the diode capacitances, build a single band matching network in π-configuration. The rectifier structure itself is based on a voltage doubler using BAT15-04W zero-bias RF Schottky diodes for efficient RF-to-DC conversion.

High-quality RF matching requires a well-defined printed circuit board (PCB) stackup. Thus, the proposed circuit has been realized on a 2-layer PCB using RO4003c as the core dielectric with a thickness of 0.813. RO4003C provides a low and controlled dielectric constant of 3.38 over a wide frequency range and a dissipation factor of 0.0027 at 10, making it well suited for power-sensitive high-frequency designs.

For power management and energy storage, the e-peas AEM30940 energy management circuit has been selected [ref. Fig.<ref> (a)]. It can start operating with empty storage elements from an input voltage as low as 380 and an input power of 3. Once active, power from input voltages as low as 50 can be converted and temporarily stored in a capacitor before generating a constant output voltage level with the internal buck converter and low drop out (LDO) regulator.











§ EXPERIMENTAL RESULTS


To evaluate the performance of the proposed sub-system it has been tuned for 915 which is a commonly used carrier frequency for WPT <cit.>. The individual functional blocks have been evaluated in terms of functionality and efficiency. Aside from the proposed narrow-band rectification circuit of section <ref>, measurements are conducted on the e-peas AEM30940 evaluation board to facilitate reproducibility.

The evaluation of the matching and rectifier characteristics, including an optimal maximal power point configuration analysis, is given in subsection <ref>. Subsection <ref> evaluates the end-to-end harvesting efficiency of the e-peas harvesting integrated circuits (IC) when combined with different RF frontends. Finally, subsection <ref> analyzes the cold-start sequence of the harvesting IC for low input power.



 §.§ Matching Circuit and Rectifier

Ideally, RF harvesting circuits comprise a broadband antenna and non-frequency selective matching circuits, allowing them to harvest from a wide range of carrier signals and input powers. If the application, the carrier frequency, and the typical input power is known, the RF path can be matched precisely to the input impedance of the rectifying circuit. 

<ref> visualizes the characterization of the matching and rectification circuitry. For the evaluation, the two selectable input frontends on the AEM30940 evaluation board, e-peas HP and e-peas LP, respectively optimized for power levels above -10dBm  and below -10dBm, are compared against the narrow band implantation of section <ref> "custom". The circuit matching is analyzed with an HP7853 network analyzer at a constant source power of 0dBm and a matched resistive load on the rectifier output replacing the subsequent circuitry. <ref> (a) shows the S11 parameter of the three circuits for frequencies of up to 2. The 
high-quality factors of the selected components allow a narrow matching to the 915 target frequency for the custom rectifier. In contrast, the e-peas HP circuit allows additional harvesting from the 868 band.

To precisely analyze the rectification efficiency of the previously shown matching and rectifying circuits, an R&S SML03 signal generator configured to 915MHz is connected to the circuit input. The peak output power and the corresponding maximal power point (MPP) configuration are measured with a source measurement unit (SMU) Keysight B2901A emulating an adaptive load. <ref> (b) shows the resulting RF path efficiency as a function of the input power for optimally matched loads. The results suggest that for input signals higher than -5dBm, the e-peas HP outperforms the e-peas LP configuration. In contrast, the custom implementation shows superior performance for input power levels between -12dBm and 9dBm with up to 20 efficiency gain as compared to the commercial circuit's efficiency. The corresponding MPP configuration is shown in <ref> (c). The e-peas LP configuration shows a linear shift with 0.4/dBm over the analyzed power range. The circuits designed for higher input powers are approximately stable until  4dBm, with a significantly rising  MPP ratio above. The inset visualizes the output power for a varying load at 0dBm.























 §.§ Wireless Power Conversion Efficiency

Correctly controlling the output voltage of the RF path is essential to efficiently supply subsequent circuitry. The circuit behavior, when combined with an epeas AEM30940 harvesting IC, is shown in <ref> (b). For the measurement, the previously tested RF frontends have been  attached to the IC's harvesting input. To allow a measurement of the end-to-end harvesting efficiency, shown in <ref> (a) an SMU configured to 3.5 has been connected in parallel to an intermittent energy storage capacitor C_boost. This allows monitoring of the harvested current at a stable and repeatable storage configuration. The embedded maximal power point tracking (MPPT) has been configured to 50 of the open circuit voltage. 
The e-peas LP frontend allows energy-positive operation of the harvesting circuitry down to -15dBm. Combined with the high power optimized frontends, the circuit reaches -16dBm and -15dBm for custom and e-peas HP, respectively. The custom circuit combined with the AEM30940 IC shows increased harvesting efficiency for input power levels between -11dBm and 5dBm with a peak efficiency of 57 at 3dBm. 




 §.§ Cold Start

A critical feature of intermittent operation devices is the capability to power up despite totally discharged energy storage. <ref> (b) shows this cold-start behavior for the AEM30940 IC connected to the RF frontend presented in section <ref>. The IC's overcharge protection has been configured to V_ovch=2.7 and the under-voltage lockout level to V_ovdis=2.2. The regulator output voltage is V_lv=1.2. 

At t=0, the harvesting circuit is supplied with a 915MHz RF input signal with a power of -15dBm. After 35, the initial wake-up phase completes as the intermittent storage capacitor C_boost charges to the programmed maximum. The activation of the output regulator V_lv and the corresponding inrush currents combined with the limited input power results in consecutive under-voltage lockouts until normal operation is reached after 56. Finally, after 93, C_boost is fully charged, and the circuit operates in overcharge protection mode. 







§ CONCLUSION

The paper proposed and evaluated a WPT sub-system, composed of an impedance matching network, rectifier, and power management with energy storage, optimized to a single narrow-band channel at 915. The experimental evaluation showed an overall PCE of over 30 at an input power of -10dBm and a peak efficiency of 57 at 3dBm. In comparison to commercial solutions, the proposed RF frontend showed an increased PCE for input power levels between -11dBm and 5dBm.
The commercial IC AEM30940 from epeas allows to temporarily provide more power than is being harvested, acting as a buffer and keeping the voltage supply constant, which is especially important for miniaturized IoT devices. Using the proposed RF frontend, the AEM30940 is able to exit cold start after 56 from an input power as low as -15dBm and switches to normal operation. 












 Paper Title: CrystalTac: 3D-Printed Vision-Based Tactile Sensor Family through Rapid Monolithic Manufacturing Technique.



: Augmented Masked Autoencoder Pretraining on Public Brain MRI Data for 3D-Native Segmentation
    Asbjørn Munk1,2* Jakob Ambsdorf1,2* Sebastian Llambias1,2 Mads Nielsen1,2

    August 5, 2024
==============================================================================================




§ ABSTRACT 





Recently, vision-based tactile sensors (VBTSs) have gained popularity in robotics systems. 
The sensing mechanisms of most VBTSs can be categorised based on the type of tactile features they capture.  Each category requires specific structural designs to convert physical contact into optical information. The complex architectures of VBTSs pose challenges for traditional manufacturing techniques in terms of design flexibility, cost-effectiveness, and quality stability. Previous research has shown that monolithic manufacturing using multi-material 3D printing technology can partially address these challenges. This study introduces the CrystalTac family, a series of VBTSs designed with a unique sensing mechanism and fabricated through rapid monolithic manufacturing. Case studies on CrystalTac-type sensors demonstrate their effective performance in tasks involving tactile perception, along with impressive cost-effectiveness and design flexibility. The CrystalTac family aims to highlight the potential of monolithic manufacturing in VBTS development and inspire further research in tactile sensing and manipulation.










§ INTRODUCTION



The integration of tactile sensors with robots has drawn considerable interest in fields ranging from soft robotics <cit.>, bionic robotics <cit.>, to human-robot interface areas <cit.>. Among the different types of tactile sensors <cit.>, vision-based tactile sensors (VBTSs) <cit.> use cameras to record the physical deformation when interacting with external objects. This method has gained increasing attention due to its superior spatial resolution on tactile sensing and relatively simple structure. The rapid adoption of VBTSs is largely due to advancements in computer vision. Vision sensors, such as RGB cameras, project a 3D scene into 2D frames, converting the colour, shape, and motion information of the external environment into a distribution of pixel values. In comparison, VBTSs operate similarly to reprographic devices, scanning and mapping 2.5D features <cit.> into 2D images. These features, such as the texture geometry of the contact surface or shear force around the touched area, are termed 2.5D because their perception depth is confined to a limited range, and cannot be extended arbitrarily along the direction of light projection as 3D visual features can. Due to this limitation, VBTSs need a specific medium to convert physical information into optical signals that the camera can detect. The realisation of such a medium generally involves four steps: 
(1) establishing the correlation between tactile and optical features, often known as the tactile sensing mechanism; (2) creating the sensor architecture to embody this tactile sensing mechanism; (3) choosing appropriate techniques to manufacture the sensor's sub-components; and (4) finalizing the entire hardware assembly process based on the produced sub-components. In conclusion, the initial two steps fall under the `design' phase of VBTSs, while the final two steps comprise the `creation' phase. The design phase frequently imposes difficulties on the creation phase. This is due to the conventional method's segregation of design, manufacturing, and assembly stages. Nevertheless, in real-world applications, these stages are interconnected and must be meticulously coordinated <cit.>.








Among the three primary modules in VBTSs <cit.>-illumination and vision modules-these are typically procured from commercial electronic component suppliers. In contrast, the contact module presents the most significant challenge <cit.> (the subsequent discussion of VBTS manufacturing in this paper primarily refers to the fabrication of contact module). A VBTS contact module normally consists of six main components: the base, lens, elastomer, marker, coating, and skin. Although different types of VBTS may share similar core components, they are often produced using diverse manufacturing techniques <cit.>. This multi-dimensional variation significantly influences the process from conceptual design to final assembly. As analysed in <cit.>, the main challenges in VBTS manufacturing include complexities in design, process, time, and quality. Here, we briefly discuss how these four challenges impact the design and creation of VBTSs:


i) Design: VBTSs' structural framework and detailed constructions vary significantly, depending on the tactile sensing mechanism employed. This diversity presents a bidirectional challenge: (1) alterations in the sensor design may have a direct impact on the subsequent manufacturing stages, and (2) the constraints of existing manufacturing techniques can limit the initial sensor design.



ii) Creation: The creation of VBTSs is generally complex and involves several sub-components. The increase of manufacturing procedures necessitates more equipment, leading to substantial process and time complexities. Additionally, reliance on manual fabrication and assembly means the quality of the sensors depends on the specialised skills of the workforce. This results in a high incidence of assembly errors and variability in sensor performance.




Previous research has focused on the design and fabrication of a specific type of VBTSs <cit.>.  In <cit.>, monolithic manufacturing technology was proposed to fabricate the C-Sight tactile sensor, which leverages advancements in multi-material 3D printing. By simplifying the VBTS manufacturing into a single printing sequence, this method accelerates production and ensures greater consistency and integration of the sub-components. It is expected to substantially improve the reliability, productivity, and affordability of the newly developed tactile sensor. However, previous work of C-Sight is limited to a single kind of sensing mechanism. To better explore the potential of this technology, we summarise the mainstream VBTSs into 5 categories based on their sensing principles and then propose to develop CrystalTac-type sensor following this categorisation, also known as CrystalTac family, aiming to demonstrate the versatility of monolithic manufacturing.

The name 'CrystalTac' draws on the properties of the crystal, a mineral known for its clarity and variability in colour and texture when combined with other minerals, just like the versatile structure of the different tactile sensing mechanism-based VBTSs. The monolithic manufacturing technology should be competent for the fabrication of the CrystalTac family by adapting to different design needs, including customised sensing principles, overall dimensions, and architectural details. 

The main contributions of this paper are listed as follows:



  * We summarised the design and creation methods of known VBTSs and proposed a new categorisation method to encapsulate their typical sensing mechanisms, including IMM, MDM, MFM, and multi-mechanism fusion.


  * We developed the CrystalTac family, a series of sensors produced through monolithic manufacturing. This family includes C-Tac, C-Sight, C-SighTac, Vi-C-Tac, and Vi-C-Sight, each design based on a unique tactile sensing mechanism.



  * We conducted functional experiments to evaluate the sensing performance, cost-effectiveness, and design flexibility of CrystalTac-type sensors. Also, an optimisation upon sub-component manufacturing and several novel marker designs are introduced.




















§ BACKGROUND


Based on the four key steps for realising the `design' and `creation' of VBTSs introduced in the previous section, we review most of the known VBTSs and summarise their sensing mechanisms, manufacturing processes, and assembly methods. 




 §.§ Typical Sensing Mechanism of Known VBTSs


As shown in Fig. <ref>, the sensing mechanisms of most VBTSs <cit.> can be categorised into several distinct methods: Intensity Mapping Method (IMM), Marker Displacement Method (MDM), Modality Fusion Method (MFM), and multi-mechanism fusion. 



  §.§.§ IMM



The Intensity Mapping Method uses the pixel intensity values from the imaging frames to indicate tactile features, as illustrated in Fig. <ref>(A).
This approach leverages the high resolution of the VBTS camera frame and the changes in each pixel value over a threshold interval to reconstruct continuous features, including fine geometric textures and dense motion distributions. This change in pixel value requires the VBTS to incorporate specific structures, such as a light-conductive plate or a reflective coating. For example, the earlier optical waveguide-type sensors <cit.> usually rely on the total internal reflective (TIR) <cit.> to map the contact pattern located on the light-conductive plate surface. GelSight-type sensors <cit.>, such as OmniTact <cit.>, Digit <cit.>, GelTip <cit.>, InSight <cit.>, and Tactile Fingertip <cit.>, employ a coating layer with uniform reflective properties applied to a silicone elastomer, enabling pixel-level tactile sensing. By incorporating RGB lighting, GelSight captures multiple images under different lighting conditions within a single frame, facilitating the estimation of the overconstrained gradient of the contact surface geometry through the photometric stereo algorithm <cit.>. In contrast to intensity mapping via a light-conductive plate or a reflective coating, DTac-type sensors <cit.>, such as 9DTac <cit.>, employ a combination of translucent gel and an opaque layer, where variations in pixel darkness can indicate intensity-to-depth regressions or even estimate force/torque. Furthermore, if a depth camera is used instead of a traditional RGB camera in VBTS, each pixel intensity value will also include real physical distance information in addition to colour information. <cit.> integrates a latex inflatable film with a depth camera, achieving precise deformation measurements through stereo vision.







  §.§.§ MDM

The Marker Displacement Method, depicted in Fig. <ref>(B), translates tactile information into displacement distributions of marker patterns either positioned on the surface or embedded within the elastomer. In <cit.>, MDM is classified into 2D-MDM, 2.5D-MDM, and 3D-MDM, based on the type of tactile features represented by the pattern changes. Both 2D-MDM and 2.5D-MDM employ a single camera to capture the two-dimensional movements of markers in horizontal and vertical directions. However, 2.5D-MDM can indirectly represent corresponding contact depth information by observing the deformable characteristics of the markers, such as changes in size or shape. In contrast, 3D-MDM precisely captures the depth of markers using stereo vision. GelForce <cit.> uses two layers of spherical marker arrays to measure 3D force distributions. Inspired by the dermal papillae in the human fingertip, TacTip <cit.> designs pin-shaped markers, which amplify contact information through leverage principles. Furthermore, Yang et al.<cit.> employ marker displacements to estimate the gradient of the contact surface through the integration method. ChromaTouch <cit.> introduces two superimposed colour filters as a composite marker pattern, where contact deformation alters the markers' hue, centroid, and apparent size. Similar designs are found in <cit.>. DelTact <cit.>, Viko <cit.>, and <cit.> all utilise a dense random colour pattern for precise tracking of contact deformations. Sferrazza et al.<cit.> randomly embed spherical fluorescent green markers within an elastomer to enhance the sensing range. A multi-colour continuous marker pattern (CMP) <cit.> has been developed to improve the representation and extraction of VBTS contact information. Tac3D <cit.> employs virtual binocular vision to precisely measure the three-dimensional shape and force distribution of the contact surface using CMP.





  §.§.§ MFM

The Modality Fusion Method integrates multi-modal features beyond sole tactile input, typically incorporating vision and proximity sensing. This integration effectively mitigates the inherent limitations of a single tactile modality, such as the inability to perceive features like colour and distance. However, MFM cannot operate independently in VBTSs and needs to be combined with IMM or MDM for tactile sensing, thereby included in the concept of multi-mechanism fusion.




  §.§.§ Multi-Mechanism Fusion

The sensing mechanisms for VBTS introduced above each offer distinct advantages. However, researchers have been proactively developing multi-mechanism fusion sensors to further improve the performance of VBTSs by obtaining comprehensive sensing information.





  * IMM + MDM: GelSight-type sensors <cit.> incorporate dot markers above the reflective coating layer, aiming for force and shear detection rather than solely capturing fine textures, as demonstrated by GelSlim <cit.>, shown in Fig. <ref>(C), GelSight Wedge <cit.>, and GelStereo <cit.>. DenseTact <cit.> employs a randomised dense pattern with a coating to extract continuous tactile features. The Soft-bubble sensor<cit.> utilises a stereo camera for the tracking of shear-induced displacement through custom marker patterns. <cit.> introduced Particle Image Velocimetry (PIV) to establish a linear relationship between force and velocity. UVtac <cit.> employs a switchable ultraviolet (UV) method to decouple marker patterns and reflective membrane images, facilitating both object localisation and force estimation, with each functionality remaining independent of the other.







  * MDM + MFM: Finger Vision <cit.>, as illustrated in Fig. <ref>(D), embeds black marker patterns on a transparent elastomer to achieve multi-modality sensing. The approach proposed by <cit.> increases force resolution and sensitivity of Finger Vision by introducing whiskers. ViTacTip <cit.> also employs a transparent skin to enhance TacTip <cit.> with vision-tactile fusion capability. Similar to UVtac, SpecTac <cit.> uses UV LEDs and randomly distributed UV fluorescent markers, allowing for a switch between visual and tactile sensing modes, controlled by the activation of the UV LEDs.








  * IMM + MFM: Shimonomura et al.<cit.> employ a light-conductive plate and compound-eye camera to capture both tactile and proximity modality information. The tactile feature is obtained as an infrared image through the TIR principle, while the proximity feature is detected through stereo matching of a pair of images obtained by two cameras. See-Through-your-Skin (STS) <cit.> utilises a translucent coating and adjustable lighting to enable the transition between tactile and visual sensing modalities. Compared to STS, StereoTac <cit.> and VisTac <cit.> incorporate binocular cameras, extending 2D visual sensing into 3D areas. As displayed in Fig. <ref>(E), TIRgel <cit.> also utilises TIR through a purely transparent elastomer to visualise tactile features, achieving conversion between visual and tactile modalities via focus adjustment.







 










 §.§ Typical Manufacturing Method of Known VBTSs



Based on Table <ref>, the manufacturing methods of known VBTSs's sub-components are discussed below. The different tactile sensing mechanisms lead to variations in their manufacturing processes, where multi-mechanism fusion further complicates the procedure.







  §.§.§ Base



The base, also known as the case, mount, frame, body, bracket, or housing, typically serves as the external structure of VBTSs. It acts as the connecting element between the contact module and the illumination/vision modules. To ensure a secure fit among these modules and to achieve the required durability and compactness of the sensors, the base must be designed with an appropriate shape and constructed from a material of sufficient rigidity. 3D printing technology <cit.> is the most common option for prototyping. While commercially large-scale productions usually rely on the mould-forming method <cit.>.






  §.§.§ Lens



The lens allows the camera to capture tactile information without obstruction. Additionally, it internally supports the elastomer during interactions. The common manufacturing method employs a laser cutter to shape the acrylic board <cit.>. However, lenses with complex curved surfaces, diverging from simple planar shapes, can only be produced through mould-forming methods <cit.> or procured from commercial suppliers <cit.>. Additionally, some sensors <cit.> do not incorporate lenses but rely on their inherent structure or an internal skeleton.







  §.§.§ Elastomer



The elastomer serves as one of the primary mediums for converting tactile information into visual data, with core properties including transparency, colour, and stiffness. Mould-formed silicone <cit.> is the material most preferred for creating these elastomers, as it offers adjustable properties to suit various applications <cit.>. The preparation process involves  A/B solutions mixing, mould casting, bubble removal, and heat curing. Translucent and coloured silicone may require dyeing with pigment <cit.>. The adjustable stiffness depends on the variation of the ratio of mixed solutions <cit.>. Some VBTSs use commercial products as alternatives, modified using DIY methods <cit.>, to simplify manufacturing. There are exceptions, such as Soft-bubble <cit.>, which features an air-filled membrane design, while TacTip <cit.>, BioTacTip <cit.> and <cit.> use mixed ultra-soft gel (TechsiL RTA27905 A/B) and coloured water respectively.





  §.§.§ Marker


For MDM-type VBTSs, markers play the role of mapping tactile deformation to visual pattern distribution. Inspired by <cit.>, we classify the marker manufacturing methods into three types:




  * Surface Fabrication: Markers are prepared on the surface of premade elastomers through direct or indirect methods. The former approach combines light etching, mask templates, and stamp plates with ink printing <cit.>, or utilises plastic beads <cit.>. Similarly, SpecTac <cit.> and <cit.> manually apply fluorescent markers using a brush and UV pen, respectively. In the latter case, the complete marker patterns are printed in advance using materials such as sticker film <cit.>, transfer paper film <cit.>, or adhesive-backed templates <cit.>, and then applied to the surface of the elastomer.



  * Embedded Fabrication: To capture the deformation field at different depths, rather than solely at the contact plane, markers are embedded within the elastomer. <cit.> incorporated fluorescent markers into the solutions during the preparation of the silicone. GelForce <cit.> and <cit.> placed two marker arrays of different colours in elastomer, prepared layer by layer.




  * Integral Fabrication: 

Markers are prepared integrally with the skin in a single piece. The pin-shaped markers with the skin of TacTip were initially produced by mould casting <cit.>. Subsequently, 3D printing has been introduced for the later versions of  MDM-type VBTSs <cit.>.









  §.§.§ Skin



The skin serves as the direct contact interface with the external environment. For VBTSs designed with silicone elastomer as the main body, some lack a skin layer <cit.>, while others employ a skin for protection, which is achieved by casting a thin layer of silicone through mould-forming <cit.> or spray painting <cit.>. To filter noise, DTac <cit.> manually applies black silicone on the top surface, similar to <cit.>. Several designs implement fabric films or adhesive tape films as a skin-like layer, as demonstrated in <cit.>. For example, Finger Vision <cit.> employs a thin transparent plastic film to protect the silicone body from contaminants. Some VBTSs rely on the skin as the primary structural component. TacTip <cit.> and ChromaTouch <cit.> build the skin as the main body, with the latter also requiring the casting of a white silicone layer to filter noise. Furthermore, soft bubbles <cit.> and <cit.> both use hand-cut latex film as an outer skin for an air-filled structure.








  §.§.§ Coating
 


Coating layers are widely used in IMM-VBTSs. Unlike the skin, which is primarily intended for protection and shading, coatings are designed to convert tactile features into visual imaging. These can be categorised into two broad types: reflective coatings and controllable coatings.




  * Reflective Coating: GelSight-type sensors <cit.> incorporate metal pigments into silicone, using bronze flakes for semi-specular coatings and fine aluminum powder for matte coatings. Various manufacturing methods are employed, such as brush painting <cit.>, airbrushing <cit.>, sputtering <cit.>, and the paint dipping technique <cit.>. Similarly, <cit.> uses metal foil to create a semi-mirror coating through the gilding process.



  * Controllable Coating: Some MFM-based VBTSs control the coating transparency by adjusting internal lighting conditions. STS <cit.>, StereoTac <cit.>, and VisTac <cit.> all apply 2-3 layers of `mirror spray' to achieve a translucent layer as the controllable coating.














 §.§ Typical Assembly Method of Known VBTSs



The VBTS assembly strategy is often overlooked in most of the current research. Here, the assembly process is analyzed across three dimensions: tools, workflows, and mechanisms.





  §.§.§ Assembly Tool
 

The assembly tool consists of manual assembly and machine assembly. Due to the widespread use of traditional manufacturing methods, such as mould-forming <cit.>, manual assembly remains the prevalent practice. Machine assembly, characterised by the use of specific devices such as multi-material 3D printers, is exemplified by TacTip <cit.> and ChromaTouch <cit.>. Generally, manual assembly may result in assembly errors, while machine assembly necessitates specific hardware equipment. Most VBTSs are assembled manually or in hybrid form, with a limited number being fully assembled by machines, such as MagicTac <cit.>.





  §.§.§ Assembly Workflow
 


The assembly workflow consists of serial assembly and parallel assembly. Serial assembly involves combining components in a specific sequential order, typically determined by structural design or manufacturing requirements. For example, 9DTac <cit.> consists of a transparent layer, a translucent layer, and a black layer, thereby asking for a fixed assembly order due to its stacked construction. In contrast, parallel assembly allows components to be assembled simultaneously, as seen in <cit.>, which achieves this due to its simple structure. Although parallel assembly is more efficient, it presents challenges in the management of sensors requiring complex processes and intricate structures. The majority of VBTSs are fabricated using serial assembly, while a minority employ a hybrid of serial and parallel assembly. Purely parallel assembly is rarely used.






  §.§.§ Assembly Mechanism
 


The assembly mechanism consists of physical assembly and chemical assembly. Physical assembly relies solely on the interaction of hardware structures. For instance, <cit.> involves casting silicone into the base mould, thereby directly assembling the elastomer with the base. Chemical assembly utilises the properties of chemical reagents, such as material compatibility and adhesion. For example, coating pigments are often mixed with silicone to enhance adhesion with the elastomer, and lenses are frequently attached to the base using adhesives. Although physical assembly offers the benefits of convenience and is well-suited for modular designs, chemical assembly excels in ensuring enhanced durability and stability. Most VBTSs are produced through a hybrid of physical and chemical assembly, with only a few being made entirely through physical or chemical assembly.





In this section, we review the commonly used methods in the design and creation of VBTS, whose representative attributes can be summarised in Fig. <ref>. Each VBTS has several attributes with distinct characteristics, such as size, colour, stiffness, and manufacturing methods. This complexity corresponds to a wide range of structural variations, placing high demands on subsequent manufacturing and assembly processes. The complexities in process, time, and quality are difficult to address due to the limitations of traditional manufacturing methods. In <cit.>, a rapid monolithic manufacturing technique has been proposed which has the potential to become a standard method to simplify the design and creation of VBTS. Its feasibility has been evaluated by C-Sight, which relies on IMM. In this work, the CrystalTac family is designed as a series of sensors with different sensing mechanisms to demonstrate the adaptability of monolithic manufacturing.









§ DESIGN AND CREATION



Based on the details of monolithic manufacturing elaborated in <cit.>, this work focuses on the design and creation of the CrystalTac family. The first step involves discussing the manufacturing feasibility of CrystalTac-type sensors based on different sensing mechanisms through monolithic manufacturing, followed by the overall design of the CrystalTac family.




 §.§ Manufacturing Feasibility of Different Sensing Mechanism-based CrystalTac


From Table <ref> and Fig. <ref>, it is evident that different sensing mechanism-based VBTSs exhibit certain tendencies in terms of sub-components, each associated with specific attributes and corresponding manufacturing methods. In detail, the choice of sensing mechanism directly determines the type of sub-component, while the specific sensor design based on that mechanism influences the attributes of each sub-component. The most striking issue is the significant variation in size, colour, and stiffness requirements for the various sub-components. Here, we discuss which sensing mechanism could be applied to CrystalTac-type sensors in terms of technological and practical feasibility.






  §.§.§ Technological Feasibility



As utilised in monolithic manufacturing, PolyJet Printing (PP) <cit.> functions similarly to inkjet printing by spraying thousands of photopolymer droplets rather than ink. This process employs ultraviolet (UV) light to cure and construct parts in a layer-by-layer fashion. The combination of inkjet and photopolymerization technologies provides PP with two significant advantages:








  * High Printing Quality: Due to the small size of the ejected resin droplets, impressive micron-level spatial resolution can be achieved in both the horizontal XY-direction and vertical Z-direction. This level of precision is essential for achieving high print quality, characterised by fine resolution and a superior surface finish.


  * Multi-material Printing: By integrating multiple print heads, pp easily achieves multi-material printing with consistent print quality, whose printing capabilities encompass a wide variety of material properties. In addition to multi-colour printing, it can also produce flexible materials with varying properties.






These two advantages have considerable significance in manufacturing different sensing mechanism-based CrystalTac with complicated structure, as shown in Fig. <ref>(B-D). Referring to the typical print materials listed in <cit.>, Vero series (VB/VW/VC)[https://www.stratasys.com/en/materials/materials-catalog/polyjet-materials/vero/], Agilus30 series (AB/AW/AC)[https://www.stratasys.com/en/materials/materials-catalog/polyjet-materials/agilus30/], and support materials[https://www.stratasys.com/en/materials/materials-catalog/polyjet-materials/polyjet-support-materials/], are suitable selections for the skin, marker, elastomer, lens, and base of CrystalTac. However, due to the lack of print materials containing metal powder, reflective or controllable coatings are currently unavailable, rendering some IMM-VBTSs based on these coatings, such as GelSight, not yet feasible for CrystalTac.












  §.§.§ Practical Feasibility


As displayed in Fig. <ref>(A), the practical feasibility depends on the suitable design of each sensor, as well as the combination and assignment setups of loaded print materials. 






  * Suitable Sensor Design: As illustrated in Fig. <ref>(A.a1), the structural design of CrystalTac must be based on a defined mechanism. For example, MDM-CrystalTac requires the marker pattern to be embedded in the correct position, MFM-CrystalTac often requires minimal occlusion within the imaging path, and IMM-CrystalTac may need a multi-layer stacked structure. An irrational structural design can fail to achieve the desired sensing mechanism. However, the efficiency and flexibility of monolithic manufacturing can help researchers rapidly implement hardware iterations of a prototype design by controlling variables.



  * Combination Setup of Print Material: It should be noted that the print materials loaded in the feed box are limited, and the number of print heads is also fixed by the printer version, as shown in Fig. <ref>(A.a2). Therefore, it is necessary to consider the actual material loading situation of the current printer, especially with multi-mechanism fusion designs which require a more complex range of print materials. In general, the minimum criteria to be fulfilled include at least three print materials, of which AC, VW/VB, and support materials must be loaded. If there are spare print heads available, priority should be given to loading AW/AB or VC. Such strategies can maximise the design options for CrystalTac with limited material combinations.






  * Assignment Setup of Print Material: After determining the sensor design, it is necessary to assign the appropriate material to each sub-component based on different attributes as shown in Fig. <ref>(A.a3), considering various factors such as colour, stiffness, and shape. For example, MFM-CrystalTac requires high transparency and low distortion levels in the skin, while MDM-CrystalTac requires effective shading. It should be emphasised that although the number of print materials loaded is limited, their properties can be adjusted through fusion with each other. For instance, by mixing flexible Agilus30 materials into the rigid Vero series, materials with intermediate stiffness can be obtained, while Vero Vivid[https://www.stratasys.com/en/materials/materials-catalog/polyjet-materials/verovivid/] can be combined with a wide range of materials to achieve rich colours. These characteristics can be adjusted by changing the mixing ratio, and the entire process is automated by the printer, greatly enhancing design flexibility and ensuring production quality.







The above analysis demonstrates that monolithic manufacturing can provide sufficient capability to create the CrystalTac family with different sensing mechanisms but given certain prerequisites. As seen in Supplementary <ref>, taking into account the optimisation for key sub-components of CrystalTac, a thickness range of 2-3 mm is optimal for printed lenses, offering a balance between high structural strength and satisfactory imaging performance. Similarly, the suitable height for the printed elastomer should range between 2 and 5 mm to balance image quality and softness.
















 §.§ Overal Design of CrystalTac Family



The CrystalTac family design is introduced here, whose family tree is illustrated in Fig. <ref>. According toFig. <ref>(A), the CrystalTac family is categorised into five branches, the C-Sight using IMM mechanism, the C-Tac using MDM mechanism, the C-SighTac using IMM + MDM mechanism, the Vi-C-Sight using IMM + MFM mechanism and the Vi-C-Tac using MDM + MFM mechanism. It should be emphasised that these five CrystalTac sensors are proposed not to provide a final design version, but rather to validate its potential to implement various tactile sensing mechanisms.








  §.§.§ Contact Module







  * C-Sight: To realise the IMM mechanism, C-Sight has been designed, as described in <cit.>, inspired by DTac <cit.>. The sectional diagram in Fig. <ref>(A) shows that within the gap between C-Sight's outer skin (a6) and the clear elastomer (a3), two additional components are present: a translucent layer (a5) composed of support material and a pure white layer (a4) made of Agilus30 White. The deformation resulting from external contact alters the distance between the black skin and the white layer. Where the distance is shortened, the pixel intensity appears darker, aiding in inferring the contact depth.





  * C-Tac: To achieve the MDM mechanism, C-Tac can be embedded with 2D and 2.5D markers through different material assignments as shown in Fig. <ref>(B), such as rigid Vero and flexible Agilus30 series. The distinction between them lies in the ability to provide pseudo-depth information, achieved either through shape deformation or variations in morphological structure. Monolithic manufacturing allows for customisation of both the stiffness properties and the morphological geometry of the markers. As shown in Supplementary <ref>, three kinds of marker patterns can be employed for C-Tac, including multi-layer markers, Voronoi markers and coordinate markers, each possessing distinct characteristics compared to dot markers.





  * C-SighTac: As illustrated in Fig. <ref>(C), C-SighTac is developed based on C-Sight and C-Tac to achieve the IMM+MDM mechanism. This approach involves the structural framework of C-Sight and marker designs from C-Tac into appropriate positions. Similar to marker-enhanced GelSight <cit.>, markers may enable C-SighTac more sensitive to dynamic features, such as force.




  * Vi-C-Tac: Inspired by Finger Vision <cit.> and ViTacTip <cit.>, Vi-C-Tac uses transparent skin to replace the opaque one of C-Tac as displayed in Fig. <ref>(d), aiming to realise MDM+MFM mechanism. The internally embedded markers provide greater dynamic sensitivity to physical interaction while retaining the multi-modality sensing capability through the clear skin.








  * Vi-C-Sight: Vi-C-Sight utilises IMM+MFM mechanism, inspired by <cit.>. Using TIR principle, variations in the gradient of the elastomer surface upon the contacted object alter the trajectory of internally reflected light beams. This causes the contact texture to be mapped onto the transparent elastomer surface, resulting in imaging brightness change while still retaining background visual information. As seen in Fig. <ref>(E), Vi-C-Sight uses pure Agilus30 Clear as the elastomer material to leverage this mechanism rather than a multi-layer grid structure.













  §.§.§ Vision and Illumination Modules



To demonstrate CrystalTac's customisation flexibility for prototype designs and its adaptability for modifying mature designs, two bases with different vision and illumination modules are applied to the five proposed sensors. The first customised base is designed for C-Sight and C-SighTac, characterised by a square body with six internal white illumination sources, as detailed in <cit.>. The other base is from the well-known Digit <cit.>, which features a small black curved housing with an RGB light source. It is a popular commercial VBTS product due to its suitability for mounting on robot hands and its quick-change contact module design. So we used it as the base for C-Tac, Vi-C-Tac, and Vi-C-Sight. A description of these two bases is provided in Supplementary <ref>.










§ PERFORMANCE EVALUATION


In this section, we designed several experiments to evaluate the CrystalTac family across a variety of task targets. Initially, the performance of CrystalTac with different sensing mechanisms was verified through three functional tasks. As C-Sight has been tested on tactile reconstruction task <cit.>, it, along with the similar C-SighTac, was not further evaluated. Instead, the focus was on C-Tac, Vi-C-Tac, and Vi-C-Sight, which were selected for object recognition, object and texture hybrid recognition, and see-through-skin exploration, respectively. Additionally, two further evaluations were conducted on the manufacturing cost and customisation flexibility of CrystalTac.







 §.§ Object Recognition



The C-Tac with single-layer dot markers was used to achieve the object recognition task. This sensor features a black skin to prevent external light interference and a white marker pattern that is sensitive to mapping skin deformation. As illustrated in Fig. <ref> (A.a), six print parts were selected, including a dot, ring, sphere, curve, waves and multiple dots. The difference between the reference image and the image after contact can indicate both the location and extent of deformation. For a marker that has been displaced or deformed, its original position is indicated in red, while the final position is indicated in blue; the absence of these colours suggests that the marker remained stationary. For instance, when the hollow ring is in contact, only the peripheral markers of C-Tac are displaced, whereas the centrally located markers are the most displaced when for the sphere.


Subsequently, a total of 1200 images were collected, with 200 images per class, to train the object recognition models. A Densenet121 <cit.> model was employed for data inference, and the evaluation results are summarised in Fig. <ref>(A.b). With a test accuracy of 98.3471%, the trained model correctly identified 119 out of 121 images in the test set, demonstrating C-Tac's capability to characterise tactile information from physical contact based on the MDM mechanism.









 §.§ Object and Texture Hybrid Recognition



To evaluate the MDM+MFM mechanism provided by Vi-C-Tac, a hybrid recognition experiment was designed, with the setup illustrated in Fig. <ref> (B.a). The experiment involved three different fabrics-blue cotton, glossy chemical fibre, and rough hemp-as well as three print parts: curve, waves, and dots. By wrapping the fabric around the objects, visual information alone was insufficient to distinguish the print objects, though it allowed for capturing the fine texture of the fabric. Thereby, the markers were utilised to capture tactile features, enabling the analysis of the shape of the underlying parts. A Vi-C-Tac system with double-layer dot markers was employed. A total of nine permutations between these fabrics and parts were tested, as shown in Fig. <ref>(B.b). The enlarged local details revealed distinct textures: cotton fabric displayed clear horizontal grain, chemical fibre exhibited point textures with strong reflective properties, while the features of hemp were less distinct due to its rough characteristics. The double-layer markers were discernible within the view, with non-overlapping parts of the markers differentiated by colour contrast.











For each permutation case, 200 images were collected, resulting in a total of 1800 images for model training. By adding two separate head structures following the output of Densenet121, two classifiers for object recognition and texture recognition were implemented in a decoupled manner. The test results, shown in Fig. <ref>(B.c), indicate identification accuracies of 100% (182/182) for object recognition and 99.45% (181/182) for texture recognition. Both visual and tactile features were successfully captured by Vi-C-Tac, thanks to the MDM+MFM mechanism.















 §.§ See-through-Skin Exploration



A Vi-C-Sight was employed to conduct see-through-skin exploration, utilising its IMM+MFM mechanism. The elastomer, made up entirely of Agilus30 Clear, functions as a light-conductive plate <cit.>, enabling the total internal reflection effect to achieve the fusion of proximity-tactile sensing. Deformed areas are differentiated in brightness from undeformed areas, while other regions remain highly transparent, thereby facilitating the see-through-skin capability. As shown in Fig. <ref>(A.a), fine textures on the objects' surfaces are clearly captured. For instance, fingerprints are visible without interference from other marker patterns, allowing Vi-C-Sight to produce such clear imaging that the entire outline of the finger is recognisable against a darker background. A distinct line of demarcation marks the contact areas where textures are mapped onto the elastomer surface, akin to glass. The fabric fibres can embed themselves into the elastomer during contact, enhancing the TIR effect. This effect can be observed in the geometry of criss-cross cotton, the fibre orientation of twill denim, and the level of embroidery deformation, which can all be mapped to variations in pixel intensity. Unlike Vi-C-Tac, the MFM mechanism of Vi-C-Sight inherently couples visual and tactile information through changes in the luminance of the pixels, similar to TIRgel <cit.>.








In Fig. <ref>(A.b), an embossed print base with a human figure was wrapped in a skeleton fabric. Vi-C-Sight was used to explore the surface of these combined parts and generate an exploration map. Due to the sparse nature of the fibres, the fabric was not as visually distinctive as the opaque printed adhesive pattern also present. However, the figure pattern of the embossed print base beneath, where the two overlap, is visible in the reddish colour of the embossed areas, indicating a higher degree of deformation and a more pronounced fabric texture in the immediate vicinity.














 §.§ Manufacturing Cost Evaluation of CrystalTac Family



A similar evaluation of manufacturing costs to <cit.> has been applied to the CrystalTac family, with the results summarised in Table <ref>. For C-Tac with single-layer dot markers, Vi-C-Tac with double-layer dot markers, and Vi-C-Sight, all of which utilise a Digit base, C-Tac is slightly larger in size and volume due to the additional layer of opaque skin. C-Sight and C-SighTac, which use a customised base, have similar volumes to the others due to their rectangular shape. This difference in shape is reflected in the unit print speed (T/V) and unit material consumption cost (C/V), as C-Tac/Vi-C-Tac/Vi-C-Sight possess complex curved surfaces with overhangs, necessitating additional support materials. This requirement reduces the unit print speed and increases the unit material consumption cost. For instance, the T/V and C/V for C-Tac/Vi-C-Tac/Vi-C-Sight are approximately 11.5 min/cm^3 and 0.73 £/cm^3, respectively, which is higher compared to C-Sight/C-SighTac, with around 10 min/cm^3 and 0.685 £/cm^3. Therefore, it can be concluded that the closer the shape of a CrystalTac component is to a rectangle, the more efficient it is to print through monolithic manufacturing, thereby lowering the manufacturing cost.













As shown in Fig. <ref>(B), further evaluations have been conducted concerning different sensor capacities within a single print batch. For all five types of CrystalTac sensors, the maximum number along the X direction on the print tray is 8, while it varies in the Y direction-6 for C-Tac/Vi-C-Tac/Vi-C-Sight and 8 for C-Sight/C-SighTac-resulting in maximum capacities of 48 and 64, respectively. As the print batch capacity approaches these maximum values, both the average printing time and cost decrease sharply until reaching a batch capacity of 8, after which they stabilise at a slightly lower level. This pattern aligns with the conclusions obtained in <cit.>. The underlying reason is that when the capacity exceeds 8, additional motion overheads are required in the Y direction for the extra columns. For example, in the case of C-Tac, when the batch capacity is 48, the average print time and cost are only 9.08 minutes and £2.43, representing decreases of 87.73% and 48.05% from 74 minutes and £4.678, respectively, when the batch capacity is 1.
















 §.§ Customised Flexibility Evaluation of CrystalTac Family



As discussed, customised flexibility is an essential metric for reducing design complexity. With monolithic manufacturing, as long as the CAD model of CrystalTac is designed, it can be manufactured directly without any additional processes. This capability allows for the realisation of many complex designs to meet the requirements of specific tasks.









  §.§.§ Customised Internal Structure
 As shown in Fig. <ref>(A), for designs that only require tactile information, light robustness is enhanced by using Agilus30 Black for the outer skin, which blocks external ambient light. Additionally, the 2.5D markers made of soft material can deform to provide richer tactile information. In contrast, markers made of rigid material maintain their structure while still moving with deformation, enabling the creation of more complex geometries. Further design examples of novel markers and complicated structures can be seen in Supplementary <ref> and Supplementary <ref>.





  §.§.§ Customised External Structure
 The external structure plays a crucial role in tactile sensing, particularly in dynamic interactions. As shown in Fig. <ref>(B), the anti-slip texture can increase friction between the sensor surface and external objects, effectively reducing undesirable sliding and providing a solid foundation for robust gripping tasks. This texture also protects the contact module body by being wear-resistant, thus extending the service life and reducing usage costs. Furthermore, whiskers can enhance the sensitivity of built-in markers. By leveraging the principle of micro-leverage effect, tactile information such as shear or pressure received at the external end is magnified at the internal end.












  §.§.§ Customised Shapes and Sizes
 The ability to modify the size and shape of the contact module is also a key feature. Compared to the traditional flat shape, spherical and cylindrical surfaces perform better in in-hand manipulation tasks since they can obtain more sensing area. However, these surfaces are more difficult to manufacture, especially internal lenses, which often require complicated processes that challenge traditional manufacturing methods. As shown in Fig. <ref>(C), a hemispherical sample and two additional samples with cone-shaped and cylindrical structures were manufactured. Their built-in lenses are consistent with the overall shapes, with elastomers all 2mm thick, matching the lens thickness.











§ CONCLUSIONS AND FUTURE WORK


In this work, we introduce a sensor family named CrystalTac, comprising five branches with unique designs. Building on the previous work of C-Sight, the monolithic manufacturing technique has demonstrated its capability to address the existing challenges in the traditional design and creation process, showing potential to become a universal manufacturing technology for VBTSs. Accordingly, we summarised the sensing mechanisms of currently known VBTSs, including IMM, MDM, MFM, and multi-mechanism fusion. Based on these typical mechanisms, five types of CrystalTac sensors were fabricated using monolithic manufacturing: C-Tac, C-Sight, C-SighTac, Vi-C-Tac, and Vi-C-Sight. Subsequent functional experiments demonstrated that the CrystalTac sensors perform well and meet their design targets with unique tactile sensing mechanisms. Additionally, monolithic manufacturing has led to significant improvements in manufacturing costs and design flexibility, often constrained by conventional methods of manufacturing and assembly.



The CrystalTac family can be regarded as an initial template, with no strict parameter limitations for each sensor detail, thereby encouraging the community to view it as a foundation for further development. The significance of this work lies in demonstrating the capability of monolithic manufacturing to produce VBTSs with various tactile sensing mechanisms, providing confidence and inspiration to other researchers in the tactile robotics field. In future work, we aim to advance the new CrystalTac series by enhancing the capabilities of monolithic manufacturing in terms of production quality and efficiency, as well as multi-material printing for VBTSs. These technologies can be seamlessly integrated with tactile sensory enhancements in dexterous hands to perform tasks such as human-computer interaction or dexterous manipulation.






§ ACKNOWLEDGMENT

The authors would like to thank Andrew Stinchcombe, Ugnius Bajarunas, and Tom Barnes for their assistance and valuable advice in technique development and hardware fabrication.






 §.§ Author Contributions
 
Conceptualisation, W.F. and H.L.; hardware, W.F. and H.L.; software, W.F.; writing—original draft preparation, W.F.; writing—review and editing, H.L. and D.Z.; visualisation, W.F. and H.L.; supervision, D.Z.; project administration, D.Z. All authors have read and agreed to the published version of the manuscript.




 §.§ Funding

Wen Fan and Haoran Li are partially funded by the CSC scholarship. The authors would like to acknowledge the Royal Society Research Grant (RGS/R1/221122).





 §.§ Conflicts of Interest


The author(s) declare(s) that there is no conflict of interest regarding the publication of this article.





 §.§ Data Availability


Please contact the authors to obtain data, including the design details of the CrystalTac family.











§ SUPPLEMENTARY MATERIALS





 §.§ Optimisation on Sub-components Manufacturing of CrystalTac




Several extended experiments have been conducted to explore the optimal range of sub-component attributes in CrystalTac, including the lens and elastomer.











  §.§.§ Optimisation for CrystalTac Lens Manufacturing
 


VeroClear is highly effective for manufacturing lens components, being a nearly colourless material that exhibits dimensional stability. Its properties are comparable to those of Polymethyl methacrylate (PMMA), commonly known as acrylic. Its upgraded version, VeroUltraClear, offers 95% light transmission and improves upon VeroClear with higher clarity, transparency, and a lower yellow index. We produced a batch of lens samples using VeroClear, which demonstrated good transparency. However, there were still strip-shaped textures present on the surface of the printed lenses, as depicted in Fig. <ref>(A).

The potential cause of these textures could be the alternate superimposition of the axial movements Px and Py of the print head in the X/Y direction. When two adjacent movements, Px1 and Px2, separated by ΔPy, produce a deviation ΔPz in the vertical height Pz possibly due to a mechanical error or the state of material polymerisation-an uneven texture is created between the Px1 and Px2 trajectories. These textures complicate the processes of refraction and reflection as incident light passes through the material. Assuming that the normal incidence angles of three parallel beams on the lens surface are N1, N2, and N3, with angles of incidence θ1, θ2, and θ3, respectively, the angles of reflection for their reflected light will be θr1, θr2, and θr3. It is evident that these reflected beams are no longer parallel. Similarly, for light refraction, which follows Snell's law as shown in Eq. <ref>, the refractive indices N_lens/N_air of the two media (lens and air) are greater than 1. Therefore, the refraction angles θl1, θl2, and θl3 will be slightly smaller than θ1, θ2, and θ3, and these refracted beams will also not be parallel to each other.



    sinθ/sinθ l = N_lens/N_air




When applied to CrystalTac, the aforementioned phenomenon of printed lenses leads to two problems. Firstly, internal illumination can cause irregular reflections on the lens surface. Secondly, images captured through the lens may exhibit optical distortion, as demonstrated in Fig. <ref>(B). Both issues significantly impair the quality of the tactile data acquired by CrystalTac. The first problem can be addressed by optimising the design of the illumination system. The second issue, however, differs from common optical lens distortions such as barrel distortion, pincushion distortion, and mustache distortion, which can be corrected using distortion parameters derived from chessboard pattern calibration. This optical distortion arises from the randomly shaped surface of the lens, leading to unpredictable outcomes that cannot be corrected through simple parameter adjustments.







This issue is evidently a consequence of the operating principles of pp, which, despite having a surface finish superior to most other 3D printing technology. To avoid redundant post-processes such as mechanical polishing, two methods have been tested to determine if the imaging quality of the printed lens surface can be improved:





  * Adjusting Printing Thickness: 


As shown in Fig. <ref>(A.a), four lenses with different thicknesses were printed: 1mm, 2mm, 3mm, and 5mm. All lenses have planar dimensions of 30mm x 30mm. For the control group, four additional acrylic lenses of the same size were fabricated using laser cutting. As indicated in Fig. <ref>(A.c), by placing a QR code pattern beneath the lenses, a camera positioned above can capture the pattern's imaging through the lenses. To quantitatively analyse the optical distortion level, the Structural Similarity Index Measure (SSIM) was introduced as the metric to evaluate image similarity. This approach assesses similarity by considering a combination of factors, including image luminance, contrast, and structure. To avoid the influence of surface texture direction, each printed lens was placed over the QR pattern along both the X and Y directions, capturing two image samples. From Fig. <ref>(A.d), the similarity between images taken through printed lenses and acrylic lenses is around 86% for thicknesses less than 3mm. However, as thickness increases to 5mm, similarity decreases to 83%. This decrease is primarily attributed to the lower light transmittance of VeroClear compared to acrylic, with excessive thickness resulting in darker images through the printed lenses. The highest similarity score, approximately 87.5%, is attained with lenses 1mm thick, suggesting that some distortion may be due to surface texture. Similarly, when comparing the similarity to the actual pattern, there is an overall decrease of about 7%, indicating that acrylic lenses also contribute to some degree of distortion.
 


  * Applying Acrylic Spray: 

As illustrated in Fig. <ref>(B), applying acrylic spray can fill the uneven texture on the lens surface with liquid droplets. Once these droplets are cured, which can enhance the overall finish of the lens, offering a more convenient and efficient alternative to physical polishing using sanding equipment. In our test, all four printed lenses were placed together and a canister acrylic spray was applied 1 m above them for around 2 s. After waiting for an hour under ventilated conditions, the liquid acrylic layer on their surface was fully cured and ready for testing, with all the settings being the same as in the previous. Also from Fig. <ref>(D), the similarity between sprayed lens imaging and acrylic lens imaging is down overall by 3-5% compared to printed lens imaging without post-process. However, the similarity to the real pattern has been improved, reaching a maximum of 86% with lenses 3mm thick. The improvement in the impact of surface texture due to the application of the spray is significant, as indicated by the reduced disparity in similarity values across the X and Y directions for lenses of all thicknesses.


As illustrated in Fig. <ref>(A.b), applying acrylic spray can fill the uneven texture on the lens surface with liquid droplets. Once cured, these droplets enhance the overall finish of the lens, offering a more convenient and efficient alternative to physical polishing with sanding equipment. In our test, all four printed lenses were placed together, and a canister of acrylic spray was applied from a distance of 1m for approximately 2 seconds. After curing for an hour in ventilated conditions, the liquid acrylic layer on the surface was ready for testing, with all settings identical to the previous tests. As shown in Fig. <ref>(A.d), the similarity between images taken through sprayed lenses and acrylic lenses decreased overall by 3-5% compared to images from printed lenses without post-processing. However, the similarity to the real pattern improved, reaching a maximum of 86% with 3mm thick lenses. The improvement in the impact of surface texture due to the spray application is significant, as indicated by the reduced disparity in similarity values across the X and Y directions for lenses of all thicknesses.






In summary, altering the printing thickness and applying acrylic spray both impact the imaging quality of printed lenses. Considering various factors, a thickness range of 2-3mm is optimal for printed lenses, providing a balance between structural strength and satisfactory imaging performance. Building on this foundation, the application of acrylic spray improves the surface finish of the printed lens.







  §.§.§ Optimisation for CrystalTac Elastomer Manufacturing
 


Most VBTSs require a transparent, silicone-like elastomer as the core material to facilitate technologies such as IMM, MDM, MFM, and multi-mechanism fusion. Agilus30 Clear, due to its transparent and flexible texture, is suitable for replacing silicone. However, the Shore hardness of pure Agilus30 is 30A, which is too hard for designs requiring a softer elastomer. For instance, GelSight typically uses a Shore hardness range of 5-20A <cit.>. By incorporating the multi-layer grid structure proposed in MagicTip <cit.>, the stiffness of the printed elastomer can be further reduced. To assess the impact of this structure on imaging quality, five test samples were manufactured. These samples shared the same X/Y dimensions of 30mm x 30mm for the elastomer, but varied in height: 2mm, 5mm, 10mm, 15mm, and 20mm, as shown in Fig. <ref>(B.a). The skin layer's thickness was set to 0.5mm to balance a soft texture with stable strength. The same QR code pattern and SSIM metric used for evaluating the imaging quality of printed lenses, as depicted in Fig. <ref>(A.b), were employed.



As displayed in Fig. <ref>(B.b), as the height of the printed elastomer increases, the imaging quality decreases in two ways. Firstly, the image tone gradually becomes darker and tends toward a darker yellow. This is primarily due to the support material filled in the internal core, which has lower light transmission than the Agilus30 material. Secondly, the imaging quality deteriorates, leading to a gradual loss of detail in the pattern beneath. This trend is confirmed by Fig. <ref>(B.c), where the similarity between the printed elastomer imaging and the real pattern is 76% at a height of 2mm but gradually drops to around 52% when the height exceeds 10mm. We also found that overall height affects stiffness to a certain extent, even with the same outer skin thickness, particularly when the height is less than 5mm. This is mainly because a reduction in elastomer height reduces the number of layers in the embedded grids, and too few layers limit the range of deformation that can occur in the support core.


For CrystalTac, to optimise the imaging quality and elasticity of the printed elastomer with the multi-layer grid structure, the suitable height ranges are 2mm to 5mm, where thinner layers lead to better transparency and thicker layers result in softer stiffness.





 §.§ Marker Design for CrystalTac Family












  * Dot Marker: 
Dot markers are among the most common patterns in VBTSs <cit.>. Regarding shape, dot markers can include both cylinders with slender thicknesses and spheres with symmetrical dimensions. The overall marker array distribution can be categorised into either uniform or random spatial arrangements. Furthermore, dot markers can be transformed into pin-shaped markers by integrating them with the skin, a feasibility demonstrated in <cit.>. As shown in Fig. <ref>(A), if a rigid material such as Vero is used for dot marker manufacturing, the printed pattern will exhibit properties of a 2D marker. However, if flexible materials like Agilus30 or Tango are used, the markers have the potential to deform into various shapes. This deformation is an additional feature, categorising them within the realm of 2.5D markers. The precise pattern of the markers is dictated by the design specifications established during CAD modelling, which involves considerations of both the number and density distribution of markers, allowing for the customisation of either a uniform or a random distribution.





  * Double-layer Marker: When the elastomer contacts the object, the extent of elastic deformation decreases along the longitudinal axis. The double marker approach utilises this inhomogeneity by capturing the difference in distribution between the upper and lower pattern layers to map tactile information <cit.>. As shown in Fig. <ref>(B), this contrasting disparity typically enhances specific dynamic tactile characteristics, such as shear motion, compared to a single-layer marker configuration. The fabrication process for double-layer markers is similar to that of dot markers, with the primary distinction being the inclusion of an additional marker layer in the CAD modelling. Moreover, the construction principles of spatial structures comprising more than two layers remain consistent. By increasing the number of layers, the extreme case is that the different layers of the marker transition from a discrete state of separation to a continuous state of interconnection, resembling a top-to-bottom cylinder. By controlling the gradient distribution of colours, it should be possible to provide a more fine-grained mapping of the tactile information. All of the above variants can be easily implemented through monolithic manufacturing.






  * Voronoi Marker:  Similar to the lattice pattern in <cit.>, the Voronoi marker is designed with a continuous pattern distribution in a square layout as shown in Fig. <ref>(C). Each edge of such a square marker can be viewed as a collection of dot markers, thus improving spatial resolution compared to the standard dot marker, as investigated in continuous marker pattern (CMP) <cit.>. Using monolithic manufacturing, Voronoi markers can be seamlessly integrated into the fabrication of the skin or embedded at any specific spatial layer within the elastomer, according to a customised design. However, the marker design must adhere to geometric principles, where each tessellation cell of the Voronoi pattern should be a polygon, such as a triangle, square, or hexagon.



  * Coordinate Marker: Most marker patterns share two characteristics: each marker typically has a regular spatial shape, such as a sphere or dot, and multiple markers are grouped into an array to map force, shear, or other contact information. These characteristics imply that a single marker plays a limited role in the overall array due to the relatively low marker density. Through monolithic manufacturing, we introduce the coordinate marker, as illustrated in Fig. <ref>(d). Each individual marker in the array can independently infer the 6D pose of its localised elastomer. This capability stems from the distinct design of each marker, akin to a three-dimensional pointer within a gyroscope, making it sensitive to both rotational and translational movements. Consequently, markers characterised by complex spatial structures, as opposed to regular shapes, can be classified as stereo markers(including proposed coordinate markers). To our knowledge, the design and creation of such stereo markers are achievable exclusively through monolithic manufacturing.













 §.§ Vision and Illumination Design for CrystalTac Family











 §.§ Printing Sample of Customised CraystalTac














 Paper Title: Physical Modelling and Cancellation of External Passive Intermodulation in FDD MIMO.



















































Physical Modelling and Cancellation of External Passive Intermodulation in FDD MIMO
The research was carried out at Skoltech and supported by the Russian Science Foundation (project no. 24-29-00189).
    
1st Stanislav Krikunov
Skoltech 

Moscow, Russia

Stanislav.Krikunov@skoltech.ru
2nd Viacheslav Zemlyakov
Southern Federal University

Rostov-on-Don, Russia

vvzemlyakov@sfedu.ru
3rd Andrey Ivanov
Skoltech 

Moscow, Russia

AN.Ivanov@skoltech.ru

    August 5, 2024
===========================================================================================================================================================================================================================================================





§ ABSTRACT 
In this paper, the physical approach to model external (air-induced) passive intermodulation (PIM) is presented in a frequency-division duplexing (FDD) multiple-input multiple-output (MIMO) system with an arbitrary number of transceiver chains. The external PIM is a special case of intermodulation distortion (IMD), mainly generated by metallic objects possessing nonlinear properties (“rusty bolt” effect). Typically, such sources are located in the near-field or transition region of the antenna array. PIM products may fall into the receiver band of the FDD system, negatively affecting the uplink signal. In contrast to other works, this one directly simulates the physical external PIM. The system includes models of a point-source external PIM, a finite-length dipole antenna, a MIMO antenna array, and a baseband multicarrier 5G NR OFDM signal. The Channel coefficients method for multi-PIM-source compensation is replicated to verify the proposed external PIM modelling approach. Simulation results of artificially generated PIM cancellation show similar performance as real-life experiments. Therefore, the proposed approach allows testing PIM compensation algorithms on large systems with many antennas and arbitrary array structures. This eliminates the need for experiments with real hardware at the development stage of the PIM cancellation algorithm.




Multiple Input Multiple Output (MIMO), Frequency Division Duplexing (FDD), Passive Intermodulation (PIM), Carrier Aggregation (CA).
















§ INTRODUCTION


Enhanced Mobile Broadband (EMBB) is a service defined by the 3rd Generation Partnership Project for 4G Long-Term Evolution (LTE) and 5G New Radio (NR) deployment to provide higher data rates for the end user <cit.>. To achieve this, EMBB utilizes such technologies as Multiple Input Multiple Output (MIMO) <cit.>, Orthogonal Frequency Division Multiplexing (OFDM), and Carrier Aggregation (CA) <cit.>. MIMO provides spatial signal diversity, OFDM provides frequency domain expansion, and CA allows flexible spectral resource allocation between different component carriers (CCs) of transmitted data <cit.>. In addition, LTE and NR specifications support the frequency division duplex (FDD) regime, where the transmitter (TX) and receiver (RX) operate simultaneously, occupying different frequency bands <cit.>. However, real base station (BS) hardware is non-ideal and has nonlinear properties <cit.>. This is especially noticeable when non-contiguously aggregated downlink (DL) signals pass through shared nonlinearities, the intermodulation products are generated <cit.>. Some of them may fall into the RX band of the FDD system. All FDD transceivers have a duplexer between TX and RX chains, which protects the RX chain from intermodulation at the same frequency as TX CC. However, IMD products of CC interaction may fall into the RX band. Additionally, these products and products at other frequencies may affect surrounding systems working in frequency/time division duplexing modes as external sources of interference.



Meanwhile, nonlinear devices can be either passive or active. IMD products of passive devices are called Passive Intermodulation (PIM). Among PIM sources are weak mechanical connections in the TX chain, kinks and sharp edges in conductors, duplexer filters or ferrite fillers, switches, metal oxide layers covering conductors and junctions, and dirt in connectors. Passive nonlinearity is mainly caused by nonlinear conductive and magnetic properties of devices inside the transceiver chain (internal PIM) or outside the system in the near-field or transition antenna array region (external PIM, air-induced PIM) due to metal fences or billboards near the antenna array <cit.>.

PIM represents one of the major interference problems in modern radio systems for service providers and equipment suppliers. PIM interference results in decreased coverage of BS cells, a decrease in the sensitivity of RX uplink (UL) signals, or possibly the complete inoperable transmission link.

Internal PIM can be eliminated by improving the production process of the equipment at the cost of manufacturing expenses. Unfortunately, external PIM cannot be eliminated in this way since it may be a part of a non-controllable built environment around the transceiver antenna system.





 §.§ Existing approaches


Despite the wide variety of PIM compensation methods <cit.>, modern approaches do not allow one to simulate external PIM in the MIMO system physically. All known papers use a compensation model approach. In this case, the compensation model design follows the physical mechanism of external PIM generation with significant simplifications. Therefore, these research results are limited by real data measurements. Such kinds of measurement are not available for many researchers. Also, none of these methods provides a comprehensive process by which artificial interference caused by an external PIM source could be simulated directly, especially in an arbitrary MIMO system.
The authors of <cit.> also mention that they are unaware of any works explicitly aiming to model and cancel air-induced PIM in FDD MIMO scenarios. 

It is also worth noting that in almost all recent articles, external PIM compensation methods intended for large-scale MIMO systems are tested only on base stations with 2 transceiver paths. Thus, works <cit.> limit their research by a dual TX/RX chain MIMO system and 3 external sources of PIM. This is not enough for a comprehensive MIMO system test. Some works <cit.> consider more complex systems with more transceiver chains. Unfortunately, the results of such works are difficult to reproduce since no data and simulation codes are available in open sources.

Consequently, developing a unified physical model that allows a realistic simulation of the external PIM phenomenon in FDD MIMO systems is an important direction from both theoretical and practical points of view.



 §.§ Contributions and Novelty


All the external PIM cancellation algorithms mentioned in the literature require real hardware-measured testing data. Accessing the real BS hardware or measured data is impossible in most cases. Additionally, setting up an experiment and environment with an artificial PIM source is a rather expensive procedure requiring many specialists and additional preparations.

This paper presents a new physical model of artificial external PIM generation based on electromagnetic theory. The model has the following features:

1) The ability to generate any number of external point PIM sources based on arbitrary Uniform Rectangular Array (URA) structure;

2) The ability of near- or far-field zone effects and polarization effects to be considered due to the near-field dipole antenna model;

3) The ability to test both UL and DL PIM compensation methods;

4) A relatively simple model that does not require large computational resources and provides acceptable modelling accuracy;





































































































§ SYSTEM MODEL


In this paper, we apply Standard equivalent complex baseband signal modelling <cit.>. The process of generating PIM consists of the following steps:


  * OFDM signal precoding via the Discrete Fourier Transform Type 1 codebook (designed for single-user MIMO mainly).

  * The frequency domain estimation of the electric field magnitude induced at an observation point by all antenna elements (external PIM source location).

  * A nonlinear element excitation and generation of passive intermodulation harmonics.

  * Backward propagation of intermodulation products at different frequencies compared to forward signal propagation.

  * Thermal noise distortion at the receiver chain. Duplexer filters are assumed to be ideal band-pass filters.

The PIM generation process is illustrated in PIMdrawio.



 §.§ Single antenna model


This work is based on a finite-length dipole antenna model of a zero-radius wire <cit.>. This allows for avoiding significant computational resource usage and provides good numerical agreement with a real antenna. Effects associated with the influence of magnetic fields are assumed to be negligible and are not considered in this article. For a dipole antenna oriented along the z-axis, the electric field at each frequency is given in cylindrical coordinates (<ref>). A sinusoidal current distribution is assumed.

    E_ρ^f = j  η I l/4 πρ[Δ z_- G_1 + Δ z_+ G_2 - 2z cos(k_f l/2) G_0 ] 
    
        E_ϕ^f = 0 
    
        E_z^f = -j η I l/4 π[G_1 + G_2 - 2 cos(k_f l/2) G_0 ]



    G_0 = e^-j k_f R_0/R_0, G_1 = e^-j k_f R_1/R_1, G_2 = e^-j k_f R_2/R_2     
 
    R_0 = √(ρ^2 + z^2), R_1 = √(ρ^2 + Δz_-^2), R_2 = √(ρ^2 + Δz_+^2)     
 
    ρ= x^2 + y^2, Δz_- = z - l/2, Δz_+ = z + l/2 ,

where x, y, z are the observation point coordinates in the cartesian coordinate system, E_ρ, E_ϕ, E_z are electric field components in the cylindrical coordinate system, r is observation point distance, η is free space impedance, I is the complex amplitude of the current feeding the antenna and l is antenna length. The average current feeding n-th antenna can be calculated using the Ohm law:

    I_n = √(P_n/Z_n),

where P_n is antenna element radiating power, and Z_n is antenna impedance. All antenna impedances are assumed to be the same, and antennas are considered not to affect each other. All antennas have the same radiating power.

Thus, the instantaneous value, if the input signal can be calculated as follows:


    I_n = I_n(t) = I_n U_n(t)


It is worth noting that the normalization to maintain power ratios is done as 𝔼[|U_n(t)|^2]=1 (for each TX chain separately).



 §.§ Antenna field coordinate transformation

The following approach with coordinate transformation has been applied to model a dipole oriented along other axes (crossed dipoles in the x-y plane). x,y,z are observation point coordinates associated with 𝐫⃗ vector, x_0^i,y_0^i,z_0^i are n-th antenna coordinates from the array associated with the 𝐫⃗_n vector. The initial dipole antenna is aligned along the z-axis. The electric field calculated in cylindrical coordinate system at the observation point 𝐫⃗_n(x-x_0^i,y-y_0^i,z-z_0^i) is 𝐄⃗(𝐫⃗-𝐫⃗_n)^cyl=𝐄⃗(Δ𝐫⃗_n)^cyl. Suppose that the dipole has been rotated in a local coordinate system under a linear transformation effect: Φ_coord=Φ_coord(Δ𝐫⃗_n) without changes in the observation point. In this case, the observation point field will be calculated as 𝐄⃗(Δ𝐫⃗_n^*)^cyl=𝐄⃗(Φ_coord(Δ𝐫⃗_n))^cyl. At the same time, this field can be transformed into Cartesian coordinate system using linear transformation for vectors Φ_vect=Φ_vect(𝐫⃗_n^*): 𝐄⃗(𝐫⃗_n^*)^cart = Φ_vect𝐄⃗(Δ𝐫⃗_n^*)^cyl.
Inverse coordinate transformation can be used to obtain the field in the original coordinate system: 𝐄⃗(𝐫⃗_n)^cart = Φ_coord^-1𝐄⃗(Δ𝐫⃗_n^*)^cart.





 §.§ Antenna array model


A typical MIMO antenna array consists of rows and columns of dual-polarized antenna elements radiating from the same point (crossed dipoles to consider polarisation properties). The whole array can be divided into subarrays. Each subarray is connected to two radio chains, normally one per polarization. Antennas within the subarray utilize tunable phase shifts to provide electric antenna directivity pattern tilt.

In practice, even if the antenna array consists of identical antenna elements, their current distributions may differ from the law of current distribution in a separate antenna due to the mutual coupling effect. In the current work, the mutual coupling effect is assumed to be negligible. Assume the current density distribution on the n-th antenna element as


    J⃗_⃗n⃗ = I_n J⃗ (r⃗-r⃗_n),

where I_n is the complex amplitude of the current applied to the n-th antenna element,  J⃗ is current density distribution in a local coordinate system of the antenna, r⃗ and r⃗_⃗n⃗ are observation point radius vectors and n-th antenna position in a global coordinate system, respectively. This current density distribution produces an electric field E⃗_n^f[J⃗_⃗n⃗] at the observation point at frequency f, which can be calculated using (<ref>).

Thus, the total electric field E⃗^⃗f⃗ produced by N_TX antenna elements at the observation point r⃗ can be represented as a superposition of the fields of individual antenna elements, taking into account the exciting currents:


    E⃗^f(r⃗) = ∑_n=0^N_TX-1 I_n^fE⃗_n^f[J⃗_⃗n⃗(r⃗-r⃗_n)]




 §.§ Point source model of external PIM

Assume that the electric field at the PIM source location (observation point) is E⃗^f(r⃗) is calculated using (<ref>). The total field exciting the PIM source can be represented as a product of PIM source orientation vector p⃗=p⃗(p_x,p_y,p_z), |p⃗|=1 and electric field vector at the given point:


    E_tot^f(r⃗) = E⃗^f(r⃗) ·p⃗


The induced voltage at the observation point is proportional to the electric field:


    U_eff^f∝ E_tot^f(r⃗)


The voltage after nonlinear distortion is proportional to a non-linear function of U_eff^t=IDFT(∑_f U_eff^f) taken in the time domain:


    U_PIM^f = DFT(ℱ_NL [U_eff^t]),

where DFT and IDFT are direct and inverse Discrete Fourier Transforms respectively. The radiation field in the reverse direction can be represented similarly to the forward path. In this case, the signal at n-th receiving antenna terminal will be defined as:


    U_n^f∝ U_PIM^f E⃗_n^f(r⃗)

Assume the level of PIM generated by a point source is specified by normalization relative to the level of thermal noise. This does not affect the amplitude and phase relationship between the generating signals and the generated PIM. This is because, physically, a point source emits a tiny amount of energy, which cannot be considered using the point source model.



 §.§ Nonlinear element model

Nonlinear elements, depending on their nature, may also have memory effects <cit.>. In this case, a nonlinear element producing IMD harmonics can be modelled in various ways. One common approach is Generalized Memory Polynomial (GMP) <cit.>. Such nonlinear function ℱ_NL=ℱ_NL [ℳ,𝒦,𝒫] has the following form of (<ref>).

    ℱ_NL = ∑_m∈ℳ∑_k ∈𝒦∑_p=1^𝒫 g_mkpu(t-m)|u(t-m-k)|^p-1,

where ℳ is the number of global precursor (ℳ<0) and postcursor (ℳ>0) taps, respectively, 𝒦 - the number of envelope lead (𝒦<0) and lag (𝒦>0) samples; 𝒫 is polynomial order, g is amplitude coefficient, u is input signal, t is time index.







§ SIMULATION RESULTS

The compensation approach for model testing is based on the Channel coefficients method <cit.>. Channel coefficients here are adaptive weights of different nonlinear terms taken into account. Upsampled low and high-frequency component carriers, referred to as basis functions (BFs) of each antenna signal, are weighted using carrier-wise adaptive channel coefficients. The original Levenberg–Marquardt optimization algorithm is replaced with Stochastic gradient descent (SGD) for faster implementation and better adaptation. The least squares (LS) coefficient estimation of BFs left unchanged. RX signal represents pure external PIM mixed with thermal noise. The algorithm tries to suppress external PIM at the level of thermal noise floor (NF).









 §.§ Simulation setup

The modelling was carried out for 5G NR band n3 shown in fig: PIM_generation. The component CCs of 5MHz bandwidths are located at the 1819 MHz and 1866.5 MHz frequencies respectively and their intermodulation products lie at the 1771.5 MHz central frequency (-71.25 MHz w.r.t. TX zero frequency). In baseband digital processing, where the PIM cancellation occurs, the TX and RX signals are oversampled by a factor of 16 for the BF generation. Artificially generated data is divided into 2 sets: a training set of 131072 samples and a testing set of 65536 samples. Signal simulation parameters are summarized in Table <ref>.

In this work, the nonlinearity for PIM generation follows the structure of (<ref>) with the following parameters for simplicity: 𝒫={3}, ℳ={0}, 𝒦={0}. Thus, a simple, memoryless, nonlinear model of IMD-3 is assumed. The nonlinear model of the compensation algorithm follows the same structure as well. However, more complex parameters are chosen under the assumption of unknown nonlinearity behaviour at the external PIM source: 𝒫={3,5}, ℳ={-2,-1,0,1,2}, 𝒦={-1,0,1}. The orientation of the PIM source p⃗ is random for each scenario. The total PIM level is artificially scaled to a reasonable value.

Each antenna in the simulations is assumed to be ideally matched to the transceiver path, has a 50 Ohm impedance, and has an average radiating power of 37 dBm. The conversion of energy into radiation occurs completely.

Two scenarios of external PIM source location are considered:

1) Scenario 1: A single external PIM source is located in front of the centre of the antenna array at a distance of 2.5 m relative to it. PIM cartesian coordinates r⃗ are: {0, 0, 2.5}.

2) Scenario 2: Three external PIM sources are located in front of the antenna with a 1 m separation along the x-axis and a 0.5 m separation along the y-axis. PIM cartesian coordinates r⃗ are: {-1, -0.5, 3}, {0, 0, 3}, {1, 0.5, 3}.



 §.§ External PIM compensation in 16T16R MIMO

The result for a large-scale MIMO system with 16 transmitting and receiving antennas is shown. Each transceiver chain in the system is represented by a set of half-wavelength dipoles grouped into unit cells of 4 antennas. The signal from each channel is divided equally among all antennas within a unit cell. The distance between the antennas is equal to half of the wavelength (w.r.t. the central frequency of the TX signal band). Even channels have vertical polarization; odd channels have horizontal polarization (crossed dipoles).  The antenna layout is shown in fig: ANTENNA_1PP_16T16R_5MHZ. A dashed line limits unit cells.



  §.§.§ Single point source compensation.

The compensation ability of the algorithm for scenario 1 is shown in fig: PSD_1PP_16T16R_5MHZ. External PIM is cancelled to the noise floor after full algorithm convergence. The algorithm convergence curve is shown in fig: ERR_1PP_16T16R_5MHZ.


  §.§.§ Multiple point source compensation.

The compensation ability of the algorithm for scenario 2 is shown in fig: PSD_3PP_16T16R_5MHZ. In this case, external PIM cannot be cancelled to the noise floor which can be caused when the optimization algorithm hits a local minimum.



 §.§ Observed effects




  §.§.§ Near-field PIM power variation


The location of the PIM source significantly affects the received power distribution between the channels. This is especially noticeable in the near-field zone of the antenna array since the wave here is spherical (fig: P_VAR_16T16R_D=01m) and the electric field varies a lot. As the source moves away from the antenna array, the wave becomes flat, and such a power variation is no longer noticeable (fig: P_VAR_16T16R_D=1m). Polarization effects can also be noticed since the signal power at even antennas differs from the signal of odd antennas.



  §.§.§ Identification of the number of PIM sources


The technique described in paper <cit.> can be used to calculate the number of independent PIM sources. The authors apply Eigenvalue Decomposition (EVD) to obtain eigenvalues of the RX signal covariance matrix. The number of eigenvalues exceeding the noise power level corresponds to the number of PIM sources spatially distinguishable from the receiver side. Generally, the number of eigenvalues exceeding the noise floor eigenvalues does not correspond to the real number of objects generating PIM. For example, 2 closely located sources may generate similar signals by level and structure. Therefore, in terms of eigenvalues, they can be considered the same source of external PIM.

Several cases of the number of external PIM sources are demonstrated in fig: RANK_NPP_1-4_16T16R: 1, 2 and 4 sources. The eigenvalues of the RX covariance matrix for each case are sorted. The external PIM average power is artificially set to the same value by scaling in each case.









§ CONCLUSION

This paper presents an approach to modelling external sources of passive intermodulation in 5G FDD MIMO systems. The approach allows for simulating PIM sources including the near-field zone of the MIMO antenna array and does not require computationally complex full-wave electromagnetic modelling. The external PIM effect can be reproduced with sufficient accuracy for PIM compensation algorithm testing on large-scale MIMO systems. The cancellation results of artificially generated PIM are aligned with experiments conducted on real data available from the literature. Thus, this approach can be used to test and debug algorithms to cancel PIM from external sources.

ieeetr




 Paper Title: Advanced ISAC Design: Movable Antennas and Accounting for Dynamic RCS.


Local analysis of a single impurity on a graphene Josephson Junction
    Elisabetta Paladino
    August 5, 2024
====================================================================

§ ABSTRACT 
We investigate resource allocation in integrated sensing and communication (ISAC) systems exploiting movable antennas (MAs) to enhance system performance. Unlike the existing ISAC literature, we account for dynamic radar cross-section (RCS) variations. Chance constraints are introduced and integrated into the sensing quality of service (QoS) framework to precisely control the impact of the resulting RCS uncertainties. Taking into account the dynamic nature of the RCS, we jointly optimize the MA positions and the communication and sensing beam design for minimization of the total transmit power at the base station (BS) while ensuring the individual communication and sensing task QoS requirements. To tackle the resulting non-convex mixed integer non-linear program (MINLP), we develop an iterative algorithm to obtain a high quality suboptimal solution. Our numerical results reveal that the proposed MA-enhanced ISAC system cannot only significantly reduce the BS transmit power compared to systems relying on fixed antenna positions and antenna selection but also demonstrates remarkable robustness to RCS fluctuations, underscoring the multifaceted benefits of exploiting MAs in ISAC systems.




§ INTRODUCTION


In the realm of sixth-generation (6G) wireless networks, the significance of multiple-input multiple-output (MIMO) systems for enhanced data rates, security, and integrated sensing and communication (ISAC) is undeniable<cit.>. However, conventional MIMO systems, which are based on multiple radio frequency (RF) chains, entail high hardware costs and complexity<cit.>. To address these issues, antenna selection (AS) has emerged as a practical solution, aiming to reduce the number of RF chains by selecting a subset of antennas with favorable channel characteristics<cit.>. Traditional MIMO systems, whether they employ AS or not, are constrained by the fixed positions of their antennas. This fixed positioning cannot fully leverage the channel variations across the spatially continuous transmitter area. Utilizing these variations could potentially enhance system performance by facilitating diversity and multiplexing gains. Holographic MIMO enables the exploitation of the channel variations across the transmitter area by employing densely packed, electronically controlled passive elements. However, the practical implementation of holographic MIMO is challenging because of the complexity and cost associated with managing a large array of narrowly-spaced antenna elements, which complicates channel estimation and data processing<cit.>.

Inspired by the spatial degrees of freedom (DoFs) offered by holographic MIMO surfaces, the novel MIMO concept of movable antennas (MAs) and fluid antennas has emerged as a bridge between holographic MIMO and traditional MIMO <cit.>. In MA-enabled systems, each antenna element is connected to an RF chain via a flexible cable, allowing physical adjustments within a designated spatial region using electromechanical devices like stepper motors <cit.>. This mobility allows the optimization of the antenna positions to achieve favorable spatial antenna correlations, significantly enhancing the MIMO system's capacity. Movable antennas (and similarly also fluid antennas) provide greater spatial flexibility compared to fixed-position antennas, resulting in enhanced system performance while requiring fewer antenna elements to exploit the available DoFs<cit.>.

To tap into the full potential of MA-enabled systems, initial works have explored joint beamforming and antenna positioning. In <cit.>, an alternating optimization (AO) algorithm was proposed for MA-enabled MIMO systems, where the base station (BS) and multiple users were equipped with MAs. Similarly the authors of <cit.> considered a multiuser MA-enabled uplink communication system with a fixed antenna array at the BS. Employing zero-forcing (ZF) or minimum mean square error (MMSE) combining, the MA positions were adjusted using a gradient descent (GD) method. However, both works optimistically assumed continuous adjustment of the MA positions within a given region, which may not be practical. Prototype designs in <cit.> and <cit.> employ discrete motion control of electromechanical devices with finite precision, leading to a quantized transmitter area and finite spatial resolution.

In recent years, ISAC has attracted significant attention as a promising technology to increase spectrum efficiency and enable the sharing of physical infrastructure for information transmission and environmental data collection<cit.>. In this regard, the authors of <cit.> studied transmit beamforming for MIMO ISAC systems, where a least-squares problem was formulated to obtain the ideal beampattern for sensing, while guaranteeing the quality of service (QoS) of the communication users. However, the achieved signal-to-noise ratio (SNR) of the received radar echoes was not considered, which may lead to unreliable sensing. Furthermore, although the dynamic nature of the radar cross-section (RCS) is well documented in the radar literature, and captured by the Swerling models <cit.>, it has been largely overlooked in the ISAC literature. In this paper, we bridge this gap and take into account the uncertainty caused by the dynamic nature of the RCS by modeling its impact on the sensing SNR.

Furthermore, we reveal that MAs offer a unique advantage for ISAC systems. The dynamic reconfigurability and sub-wavelength positioning of the antenna elements enhances both  data transmission and sensing accuracy by enabling precise beamforming and optimal beamwidth design, avoiding undesirable side lobes, and improving interference management. Crucially, the flexible sub-wavelength positioning of the MAs is vital for managing the dynamic RCS. 

This paper pioneers the exploitation of MAs with a spatially discrete transmitter area and the consideration of the uncertainty introduced by the dynamic RCS for enhancing ISAC system performance. The main contributions of this paper can be summarized as follows:


    
  * We investigate the benefits of MAs for ISAC systems, where we account for the discrete nature of the possible MA positions.
    
  * We introduce a new sensing performance metric leveraging chance constraint to address fluctuations in the SNR caused by the dynamic RCS of the sensing targets. 
    
  * We jointly optimize the MA positions and the downlink information and sensing beamformers for minimization of the transmit power of the BS, which results in a challenging non-convex mixed integer non-linear program (MINLP).
    
  * Our simulations demonstrate that strategic MA positioning can significantly reduce power consumption in ISAC systems compared to baseline methods. Furthermore, expanding the transmitter area enhances performance, albeit at the expense of increased computational complexity.
    

Notation: 
In this paper, matrices and vectors are denoted by
boldface capital letters 𝐀 and lower case letters 𝐚, respectively. 𝐀^T, 𝐀^*, 𝐀^H, Rank(𝐀), and Tr(𝐀) are the transpose, conjugate, Hermitian, rank, and trace of matrix 𝐀, respectively. 
𝐀≽0 denotes a positive semidefinite matrix. 𝐈_N is the N-by-N identity matrix. ℝ^N× M and ℂ^N× M represent the space of N× M real-valued and complex-valued matrices, respectively. |·| and ||·||_2 stand for the absolute value of a complex scalar and the l_2-norm of a vector, respectively. 0_L and 1_L represent the all-zeros and all-ones vectors of length L, respectively. 𝔼[·] refers to statistical expectation.



§ SYSTEM MODEL



We consider a dual function radar and communication (DFRC)-BS that is equipped with N MA elements, capable of sub-wavelength positioning to enhance spatial resolution. The DFRC-BS  serves K single-antenna communication users and concurrently senses E potential targets with high precision. The MA elements can be simultaneously adjusted within a designated two-dimensional transmitter area. We assume that perfect channel state information (CSI) is available at the DFRC-BS in order to reveal the maximum performance of MA-assisted ISAC systems<cit.>. 



 §.§ Transmitter Model

The transmitter area of the MA-enabled communication and sensing system is quantized<cit.>.

We collect the M possible discrete positions of the MAs in set 𝒫={𝐩_1,⋯, 𝐩_M}, where the distance between the neighboring positions is equal to d in horizontal and vertical direction, as shown in Fig. <ref>. Here, 𝐩_m=[x_m,y_m] represents the m-th candidate position with horizontal coordinate x_m and vertical coordinate y_m. 
In other words, the feasible set of the position of the n-th MA element, 𝐭_n, is given by 𝒫, i.e., 𝐭_n∈𝒫. For notational simplicity, we define sets 𝒦={1,⋯,K}, 𝒩={1,⋯,N}, and ℳ={1,⋯,M} to collect the indices of the users, MA elements, and candidate positions of the MA elements, respectively. Furthermore, we define the binary position selection vector of the n-th MA element as 𝐛_n=[b_n[1],⋯,b_n[M]]^T,∀ m, where b_n[m]∈{0,1} and ∑_m=1^Mb_n[m]=1, ∀ n. Note that b_n[m]=1 if and only if the m-th discrete position in 𝒫 is selected for the n-th MA element <cit.>. 

As antenna elements cannot be infinitely small, two MA elements cannot be positioned arbitrarily close to each other. Therefore, the center-to-center distance between any two MA elements must exceed a certain minimum distance, D_min. We define a distance matrix 𝐃∈ℂ^M× M, whose entry D_m,m' represents the distance between the m-th and m'-th candidate positions in 𝒫. Thus, the minimum distance constraint between any pair of MA elements can be expressed as follows:

    𝐛_n^T𝐃𝐛_n'≥ D_min, n≠ n', ∀ n, n'∈𝒩.





 §.§ Signal Model

     The DFRC-BS transmits simultaneously information symbols c_k∼𝒞𝒩(0,1), k ∈{1,...,K}, to K communication users. In addition, for sensing, dedicated radar signal 𝐬_0∈𝒞^N × 1 with covariance matrix 𝐑 = 𝔼[𝐬_0𝐬_0^H] ≽0 is also concurrently transmitted. Here,  communication and radar signals are assumed to be statistically independent such that 𝔼[c_k^*𝐬_0]=0. The baseband transmit signal of the DFRC-BS can be expressed as follows
     


    𝐱= ∑_k=1^K 𝐰_k c_k+𝐬_0,

where 𝐰_k ∈ℂ^N × 1 denotes the transmit beamforming vector for user k. To facilitate the sensing of multiple targets, we employ a multi-beam signal. Thus, 𝐬_0 is given by
𝐬_0 = ∑_n=1^N𝐯_n s_n,
where s_n are independent and identically distributed pseudo-random symbols, having zero mean and unit variance, and 𝐯_n is the n-th sensing beamforming vector. Accordingly, the sensing beamformers {𝐯_n}_n=1^N can be obtained from 𝐑 via eigenvalue decomposition (EVD).  Based on (<ref>), the covariance matrix of the transmit signal is given by 𝐑_𝐱=𝔼[𝐱𝐱^H]=∑_k=1^K 𝐰_k 𝐰^H_k+𝐑. 



 §.§ Communication Channel and Metric


In the considered MA-enabled MIMO system, the physical channel can be reconfigured by adjusting the positions of the MA elements. The channel vector between the n-th MA element and the K users is denoted by 𝐡_n(𝐭_n)=[h_n,1(𝐭_n),⋯,h_n,K(𝐭_n)]^T and depends on the position of the n-th MA element, 𝐭_n, where h_n,k(𝐭_n)∈ℂ denotes the channel coefficient between the n-th MA element and the k-th user. The channel between the m-th candidate position of the n-th MA element, 𝐩_m, and the k-th user is given by h_n,k(𝐩_m), which is modelled as 
h_n,k(𝐩_m)=1_L_p^TΣ_k𝐠_k(𝐩_m),
where 1_L_p denotes the uniform field response vector (FRV) at the k-th user, which has a single, non-adjustable antenna <cit.>. The diagonal matrix Σ_k=diag[σ_1,k,⋯,σ_L_p,k]^T contains on its main diagonal the path weights of the L_p paths that extend from the transmitter's location to the k-th user. The path weights σ_l_p,k, l_p∈{1,...,L_p}, are independently distributed, and follow complex Gaussian distribution 𝒞𝒩(0,L_0D_l_p,k^-α). Here, L_0 represents the reference large-scale fading at a distance of d_0=1 m, D_l_p,k is the distance between the BS and the k-th user via the l_p-th scatterer, and α denotes the path loss exponent. Furthermore, 𝐠_k(𝐩_m) denotes the transmit FRV linking the k-th user to the m-th MA position, 𝐩_m, and is given by 
𝐠_k(𝐩_m)=[e^jρ_k,1(𝐩_m),⋯,e^jρ_k,L_p(𝐩_m)]^T,
where ρ_k,l_p(𝐩_m)=2π/λ((x_m-x_1)cosθ_k,l_psinϕ_k,l_p+
(y_m-y_1)sinθ_k,l_p) represents the phase difference between 𝐩_m and the first position 𝐩_1 for the l_p-th channel path, and λ is the carrier wavelength <cit.>. For the k-th user, θ_k,l_p and ϕ_k,l_p represent the elevation and azimuth angles of departure (AoD), respectively, for the l_p-th channel path. The distribution of the angles is assumed to be given by f_AoD(θ_k,l_p, ϕ_k,l_p)=cosθ_k,l_p/2π, where both θ_k,l_p and ϕ_k,l_p are in the range of [- π/2, π/2] <cit.>.  Next, we define matrix 𝐇̂_n=[𝐡_n(𝐩_1),⋯,𝐡_n(𝐩_M)]∈ℂ^K× M collecting the channel vectors from the n-th MA element to all K users for all M feasible discrete MA locations. Then, 𝐡_n(𝐭_n) can be expressed as 𝐡_n(𝐭_n)=𝐇̂_n𝐛_n.
For the considered MA-enabled multiuser MISO system, the channel matrix between the BS and the K users, 𝐇=[𝐡_1(𝐭_1),⋯,𝐡_N(𝐭_N)]∈ℂ^K× N, is then given by
  𝐇=𝐇̂𝐁,
where matrices 𝐇̂∈ℂ^K× MN and 𝐁∈ℂ^MN× N are defined as follows, respectively,


    𝐇̂   =   
    [𝐇̂_1,⋯,𝐇̂_N],
    𝐁   =   [ 𝐛_1 0_M 0_M   ⋯ 0_M; 0_M 𝐛_2 0_M   ⋯ 0_M;   …   …   …   …   …; 0_M 0_M 0_M   ⋯ 𝐛_N ].


Next, we define 𝐡̂_k∈ℂ^1× MN as the k-th row of 𝐇̂. Then, the received signal of the k-th user is given by
y_k=𝐡̂_k𝐁∑_l∈𝒦𝐰_lc_l+𝐡̂_k𝐁∑_n∈𝒩𝐯_ns_n+n_k,
where n_k∈ℂ denotes the additive white Gaussian noise (AWGN) at the k-th user with zero mean and variance σ_k^2. Thus, the signal-to-interference-plus-noise ratio (SINR) of the k-th user is given by 


    γ_k=|𝐡̂_k𝐁𝐰_k|^2/∑_i∈𝒦∖{k}|𝐡̂_k𝐁𝐰_i|^2+𝐡̂_k𝐁𝐑𝐁^T𝐡̂^H_k+σ_k^2.



 §.§ Sensing Channel and Metrics

For target e, the FRV of the n-th MA element for all M feasible discrete MA positions is given by
𝐚_n(θ_e,ϕ_e)=[e^j ρ_e(𝐩_1) ,⋯,e^jρ_e(𝐩_M) ]^T,
where ρ_e(𝐩_m)=2π/λ((x_m-x_1)cosθ_esinϕ_e+(y_m-y_1)sinθ_e), and θ_e and ϕ_e are the elevation and azimuth AoDs corresponding to target e. Next, we concatenate the individual FRVs of all MA antenna elements, which leads to 
â(θ_e, ϕ_e) =[ 
𝐚^T_1( θ_e, ϕ_e), 
⋯, 𝐚^T_N(θ_e, ϕ_e)]^T.
The steering vector 𝐚(θ_e, ϕ_e)=[e^j ρ_e(𝐭_1) ,⋯,e^jρ_e(𝐭_N) ]^T of the N MA elements can thus be expressed as 𝐚(θ_e, ϕ_e)=𝐁^Tâ(θ_e, ϕ_e).


  §.§.§ Beam Pattern Matching Design

To ensure high-quality sensing, the desired target locations have to be illuminated by an energy-focusing beam with low side lobe leakage such that the desired echoes can be easily distinguished from clutter. To this end, we discretize the elevation angle domain [-π/2, π/2] to L directions and the azimuth domain [-π/2, π/2] to Q directions 
 
  and specify the ideal beam pattern {𝒟(θ_l,ϕ_q)}_q=1,l=1^Q,L,  where  𝒟(θ_l,ϕ_q) is given by



    𝒟(θ_l,ϕ_q)=
    
    1,    θ_e-Δ≤θ_l≤θ_e+Δ
       and  ϕ_e-δ≤ϕ_q≤ϕ_e+δ, e=1,…,E   
    
      0,    otherwise,

where 2Δ and 2δ are the beamwidths for each target with respect to the elevation and azimuth angles, respectively <cit.>. Consequently, to quantify the beam pattern matching accuracy, we adopt the mean square error (MSE) between the ideal beam pattern and the actual beam pattern as a performance metric for sensing <cit.>, which is given by

    0.951/Q1/L∑_q=1^Q∑_l=1^L |ρ_0𝒟(θ_l,ϕ_q) -â^H(θ_l,ϕ_q) 𝐁𝐑_𝐱𝐁^Tâ(θ_l,ϕ_q)|^2,

 where ρ_0 is a scaling factor.
 



  §.§.§ Received Echo Signal
The
communication waveform is completely known at the DFRC-BS and thus its reflected signal can
also be exploited for target detection. Thus, the communication signal is not treated as interference at the radar receiver. Under the assumption that the transmit waveform is narrow-band and the sensing channel is line of sight (LoS) <cit.>, the received echo signal at the DFRC-BS is given by
𝐫_e= 𝐇_e𝐱+ 𝐳,
   where  𝐇_e=ϵ_eL_0/2Ψ_e𝐚(θ_e,ϕ_e)𝐚^H(θ_e,ϕ_e) is the round-trip channel matrix of the e-th target and 𝐳∼𝒞𝒩(0,σ^2_e𝐈_N) is the received AWGN at the BS.
Here, Ψ_e denotes the distance between the DFRC-BS and target e, ϵ_e= √(Ω_e/4πΨ^2_e) is the reflection coefficient of target e, and Ω_e is the radar cross-section of target e. In contrast to the existing ISAC literature, where the RCS is assumed to be constant and known, we account for the dynamic nature of the RCS, characterized by the Swerling model in the radar literature<cit.>. In particular, Ω_e is modeled as exponentially distributed with probability density function (PDF)
p(Ω_e)=1/Ω_avexp(-Ω_e/Ω_av),
where Ω_av is the average RCS <cit.>. To achieve satisfactory sensing performance, we require the sensing SNR of target e to be higher than a pre-defined minimum threshold. This requirement is mathematically expressed as
Γ_e≜Ω_eL_0^2â^H(θ_e,ϕ_e)𝐁𝐑_𝐱𝐁^Tâ(θ_e,ϕ_e)/16πΨ^4_eσ^2_e>Γ^th_e,
where Γ^th_e is the minimum SNR required at the BS for sensing target e.
Given the dynamic nature of the RCS fluctuations, uncertainties for system design arise. In response to these uncertainties, we adopt a chance constraint to be able to provide performance guarantees. To this end, we impose the following probabilistic constraint to maintain the QoS for the sensing target



    Pr{Γ_e≤Γ^th_e}≤ν,
 where ν, 0<ν<1, denotes the maximum tolerable probability of failure. By enforcing (<ref>), we ensure that despite the uncertainty imposed by the dynamic RCS, the desired sensing SNR is achieved at least with probability 1-ν. 
 
 

    



§ PROBLEM FORMULATION AND PROPOSED SOLUTION



 §.§ Problem Formulation

In this paper, we aim to minimize the power consumption of the considered system while guaranteeing the QoS for each communication user and sensing target. 
 
 The resulting resource allocation problem is formulated as follows:


    𝒫_0:𝐁,𝐰_k,𝐑,ρ_0ℱ≜∑_k∈𝒦𝐰_k_2^2+Tr(𝐑)
            γ_k≥γ^th_k,∀ k∈𝒦,
    
       
       ∑_m=1^Mb_n[m]=1,  b_n[m]∈{0,1},0.90∀ n∈𝒩, ∀ m∈ℳ,
        𝐛_n^T𝐃𝐛_n'≥ D_min, n≠ n', ∀ n, n'∈𝒩,
        1/Q1/L∑_q=1^Q∑_l=1^L |ρ_0𝒟(θ_l,ϕ_q) -â^H(θ_l,ϕ_q) 𝐁𝐑_𝐱
       𝐁^Tâ(θ_l,ϕ_q)|^2≤δ_d,  
        Pr{Γ_e≤Γ^th_e}≤ν,

    where C1 guarantees that the SINR of user k does not fall bellow the minimum required threshold γ^th_k. C2 indicates that, for each MA element, only one position can be selected. C3 enforces that the position selection indicator for MA is an integer variable. C4 guarantees that the minimum distance between any pair of MA elements exceeds D_min. C5 ensures that the MSE between the desired radar beam pattern and
the actual beam pattern of the transmitted signal does not
exceed a predefined threshold δ_d. Finally, C6 ensures that the sensing SNR for target e at the DFRC-BS falls below threshold, Γ^th_e, with a probability of not more than ν. 


	

 §.§ Solution of the Optimization Problem

Optimization problem 𝒫_0 is non-convex due to the coupling between the optimization variables, binary constraint C3, binary quadratic constraint C4, and non-convex constraints C1 and C6. In general, it is very challenging to find a globally optimal solution for non-convex optimization problems. Here, we propose a low-complexity AO based iterative algorithm, which yields a suboptimal solution of 𝒫_0. In particular, we first optimize the beamforming for the communication users and sensing targets, and then we optimize the position of the MA elements. 



 


 
 

 
  



  §.§.§ Beamforming Optimization for Sensing and Communication

First, the positions of the MA elements are fixed, i.e., 𝐁=𝐁^(t), where t denotes the iteration index of the AO algorithm, and the beamforming for both communication and sensing are optimized. To do so, we employ semidefinite programming (SDP) and define 𝐖_k=𝐰_k𝐰^H_k and 𝐇_k=𝐡̂^H_k𝐡̂_k. For notational simplicity, we further define 𝐅_k≜𝐁𝐖_k𝐁^T and 𝐘≜𝐁𝐑𝐁^T. As a result, the SINR constraint can be restated as follows:



    :Tr(𝐇_k𝐅_k)/∑_i≠ kTr(𝐇_k𝐅_i)+Tr(𝐇_k𝐘)+σ^2_k≥γ^th_k.

Considering C5, we initially rewrite the beam pattern as 
 𝒟(𝐩_m,θ_l,ϕ_q,𝐅_k,𝐘)≜â^H(θ_l,ϕ_q) (∑_k∈𝒦𝐅_k+𝐘)â(θ_l,ϕ_q). Subsequently, the beam pattern MSE constraint is reformulated as follows:
 


    C5:1/Q1/L∑_q=1^Q∑_l=1^L |ρ_0𝒟(θ_l,ϕ_q) -𝒟(𝐩_m,θ_l,ϕ_q,𝐅_k,𝐘)|^2≤δ_d.

Next, we tackle constraint C6. We first recast this constraint as follows:




    C6:Pr{Ω_e≤16πΨ^4_eσ^2_eΓ^th_e/L^2_0𝒟(𝐩_m,θ_e,ϕ_e,𝐅_k,𝐘)}≤ν,
 
To calculate the probability in (<ref>), we recall that Ω_e is exponentially distributed. Therefore, this probability can be calculated as 
1-exp(-16πΨ^4_eσ^2_eΓ^th_e/L^2_0𝒟(𝐩_m,θ_e,ϕ_e,𝐅_k,𝐘)×1/Ω_av).
Consequently, we can restate (<ref>) as follows:

    C6:𝒟(𝐩_m,θ_e,ϕ_e,𝐅_k,𝐘)≥ -16πΨ^4_eσ^2_eΓ^th_e/ln (1-ν)Ω_avL_0^2.

  Finally, the resulting optimization problem for the communication and sensing beams can be rewritten as follows

    𝒫_1:𝐖_k,𝐑,ρ_0∑_k∈𝒦Tr(𝐖_k)+Tr(𝐑)
         :Rank(𝐖_k)≤ 1, ,,C6.

Now, by removing rank-one constraint C7 and employing SDP relaxation, problem 𝒫_1 becomes a convex optimization problem, which can be effectively solved using CVX. The tightness of the SDP relaxation can be confirmed using a similar approach as in<cit.>. However, due to space limitations, the proof is omitted here.


  §.§.§ Optimization of Position of MA Elements

In this subsection, we optimize the positions of the MA elements for given 𝐖_k=𝐖^(t)_k and 𝐑=𝐑^(t).  We start by reformulating the quadratic inequality constraint C4 into three linear inequality constraints using the following lemma<cit.>.

    Inequality constraint C4 can be reformulated as a set of linear inequality constraints using binary auxiliary variables ϕ_n,n',i,j

    0.9C4a:∑_i∈ℳ∑_j∈ℳD_i,jϕ_n,n',i,j≥ D_min,n≠ n',∀ n,n'∈𝒩,
       0.9C4b:ϕ_n,n',i,j≤min{b_n[i],b_n[j]},n≠ n', ∀ n,n'∈𝒩,∀ i,j∈ℳ,
       0.9C4c:ϕ_n,n',i,j≥ b_n[i]+b_n[j]-1,n≠ n', ∀ n,n'∈𝒩,∀ i,j∈ℳ.
 

To simplify notation, we introduce binary vector ϕ=[ϕ_1,2,1,1,⋯,ϕ_n,n',i,j,⋯,ϕ_N-1,N,M,M], n≠ n', ∀ n,n'∈𝒩, and ∀ i,j∈ℳ collecting all binary auxiliary variables.
Next, we relax the integer variables into continuous ones and we rewrite them as C8a: 0≤ b_n[m]≤ 1, C8b: ∑_m=1^M∑_n=1^Nb_n[m]-b^2_n[m]≤ 0, and  C9a: 0≤ϕ_n,n',i,j≤ 1, C9b: ∑_i∈ℳ∑_j∈ℳ∑_n∈𝒩∑_n'∈𝒩ϕ_n,n',i,j-ϕ^2_n,n',i,j≤ 0. Then, we linearized C8b and C9b via Taylor approximation<cit.>. To facilitate the solution design, we add C10:𝐅_k=𝐁𝐖_k𝐁^T and C11:𝐘=𝐁𝐑𝐁^T as two additional constraints in the optimization problem. Next, in the following lemma, we transform equality constraints  and  into equivalent inequality constraints <cit.>.

Equality constraints  and  become equivalent to the following inequality constraints by introducing auxiliary optimization variables 𝐒, 𝐓, 𝐔, and 𝐕 and applying Schur's complement:


    [        𝐒      𝐅_k     𝐁𝐖_k;    𝐅_k^H        𝐓        𝐁; 𝐖_k^H𝐁^T      𝐁^T      𝐈_N ]   ≽0,
       Tr(𝐒-𝐁𝐖_k𝐖_k^H𝐁^T)   ≤0,
       [      𝐔      𝐘     𝐁𝐑;    𝐘^H      𝐕    𝐁^T; 𝐑^H𝐁^T      𝐁    𝐈_N ]   ≽0,
       Tr(𝐔-𝐁𝐑𝐑^H𝐁^T)   ≤0.



We note that constraints  and  are  linear matrix inequality (LMI) constraints, whereas  and  present a challenge due to their difference of convex (DC) function structure. To address these new non-linearities, we employ Taylor approximation for the DC components in  and , transforming them into affine constraints as :f_1(𝐒)-g_1,k(𝐁)≤ 0 and :f_2(𝐔)-g_2,k(𝐁)≤ 0 where f_1(𝐒), g_1,k(𝐁), f_2(𝐔), and g_2,k(𝐁) are defined as follows


    f_1(𝐒)≜Tr(𝐒), f_2(𝐔)≜Tr(𝐔), 
       g_1,k(𝐁)≜Tr(𝐁^(t)𝐖_k𝐖_k^H𝐁^(t)^T)-
       2Re[Tr((𝐖_k𝐖_k^H𝐁^(t)^T)(𝐁-𝐁^(t)))],
       g_2,k(𝐁)≜Tr(𝐁^(t)𝐑𝐑^H𝐁^(t)^T)-
       2Re[Tr((𝐑𝐑^H𝐁^(t)^T)(𝐁-𝐁^(t)))],

where 𝐁^(t) is the solution in the t-th iteration. Finally, we introduce penalty factors τ_i, ∀ i∈{1,2,3,4}, to incorporate  and , , and  into the objective function. In this case, the optimization problem at hand can be written as follows:



    𝒫_2:𝐁,𝐅_k,𝐘,𝐒,𝐓,𝐔,𝐕,ϕ∑_k∈𝒦Tr(𝐖_k)+Tr(𝐑)+
       τ_1(f_1(𝐒)-∑_k∈𝒦g_1,k(𝐁))+τ_2(f_2(𝐔)-∑_k∈𝒦g_2,k(𝐁))+
       τ_3∑_m=1^M∑_n=1^N(b_n[m]-b^(t)_n[m](2b_n[m]-b^(t)_n[m]))+τ_4
       ∑_i∈ℳ∑_j∈ℳ∑_n∈𝒩∑_n'∈𝒩(ϕ_n,n',i,j-ϕ_n,n',i,j^(t)(2ϕ_n,n',i,j-ϕ_n,n',i,j^(t)))
        ,,,,,C5,C6,,,,.

In each iteration t, we update the solution set and efficiently solve 𝒫_2 by CVX. The proposed solution based on AO is summarized in Algorithm <ref>. Note that for sufficiently large penalty factors, τ_i,∀ i∈{1,2,3,4}, in 𝒫_2, the objective function of 𝒫_0 is non-increasing in each iteration of Algorithm 1 and converges to a high-quality suboptimal solution with polynomial time computational complexity <cit.>. The computational complexity of Algorithm 1 can be shown to be given by 𝒪(log(1/ε_AO)((2K+E+1)N^3+(2K+E+1)^2N^2+(K+2N+10MN+KMN+E+1)N^3+(K+2N+10MN+KMN+E+1)N^2), where 𝒪 ( · ) is the big-O notation and ε_AO is the convergence tolerance of Algorithm 1. 





§ SIMULATION RESULTS



In this section, we evaluate the performance of MA-enabled ISAC systems and study the impact of RCS fluctuations via numerical simulations. The transmitter area is modeled as a square with dimensions aλ× aλ, where a represents the normalized size of the transmitter area at the BS. The default parameters are detailed in Table <ref>. The users are randomly distributed, with the distance from the BS uniformly ranging from 10 m to 50 m, while two sensing targets are positioned at (θ_1=0^∘,ϕ_1=0^∘) and (θ_2=30^∘,ϕ_2=30^∘), with their distances from the BS varying between 10 m and 25 m.  
We compare our proposed approach against two baseline schemes. In baseline scheme 1, the positions of the antenna elements are fixed. The beamforming vectors are determined by solving the beamforming problem for a fixed antenna array. 
Baseline scheme 2 utilizes an AS strategy, where the BS is equipped with a 2× N uniform planar array (UPA) with fixed-position antennas, spaced λ/2 apart. This setup ensures statistically independent channels across different antennas. The beamforming optimization is conducted for all possible subsets of N antenna elements, and the subset that achieves the lowest BS transmit power is selected.



Fig. 2 shows that the average transmit power of the DFRC-BS monotonically increases with the sensing SNR requirement for all considered schemes, reflecting the higher power needed to meet stricter sensing requirements. Moreover, Fig. 2 reveals the effectiveness of MA positioning in reducing the required transmit power, especially when compared to the fixed antenna configuration of baseline scheme 1, which leads to suboptimal spatial correlations of the transmit signal. In contrast, the proposed approach selects the optimal MA positions for the given channel conditions. Although baseline scheme 2, through AS, improves the spatial DoFs at the DFRC-BS, its limited choices for the antenna positions, spaced at λ/2, still restrict the spatial resolution. These findings highlight that MAs with their sub-wavelength precision can significantly enhance the system's ability to shape its radiation pattern benefiting both communications and sensing, thereby boosting the power efficiency of ISAC systems. Furthermore, by varying the maximum tolerable probability of failure, ν, we observe that meeting stricter sensing QoS requirements, e.g., ν=0.01, requires a higher transmit power compared to more relaxed sensing QoS requirements, e.g., ν=0.1. Notably, the proposed scheme with optimized MA positions consumes less power for the stricter sensing QoS requirement of ν=0.01 than baseline scheme 1 with fixed antenna positioning for the more relaxed QoS sensing requirement of ν=0.1. This not only demonstrates the efficiency of the proposed scheme but also highlights its ability to mitigate the challenges caused by the dynamic RCS. 









Fig. 3 depicts the average transmit power versus the normalized transmitter size. For the proposed scheme, as the normalized transmitter size increases, the average transmit power required decreases due to the increased flexibility in MA positioning. Furthermore, for relaxed sensing QoS requirements (i.e., ν=0.1), the proposed scheme continues to benefit from increasing the transmitter size also for a>3, whereas performance saturates for a>3 for stricter sensing QoS requirements (i.e., ν=0.01). In other words, the range of transmitter areas in which the proposed scheme can benefit from the additional DoFs offered by flexible MA positioning is larger for less stringent sensing QoS requirements. On the other hand, baseline schemes 1 and 2 cannot benefit from an expanded transmitter area, as their performance is limited by their fixed antenna spacing of half a wavelength.








 



§ CONCLUSION



In this paper, we investigated for the first time MA-assisted multi-user multi-target ISAC systems with spatially discrete antenna positions. We introduced a new performance metric for evaluating the sensing QoS, employing a chance constraint to manage the impact of the uncertainty introduced by dynamic RCS variations. Subsequently, we optimized the MA positions and the communication and sensing beams for minimization of the total transmit power at the BS while ensuring the individual communication and sensing
task QoS requirements. The resulting non-convex problem was solved via AO where a high-quality suboptimal solution was obtained. Simulation results revealed that the proposed approach enables substantial power savings in ISAC systems, primarily due to the optimization of the MA positions, compared to baseline schemes employing fixed antenna positions and AS. Our results further unveiled that the sub-wavelength positioning of MA elements enhances the robustness of ISAC systems against RCS fluctuations, particularly under stringent sensing QoS constraints.





IEEEtran




 Paper Title: Air-to-Ground Cooperative OAM Communications.






















IEEE WIRELESS COMMUNICATIONS LETTERS, VOL. 13, NO. 4, APRIL 2024
Shell et al.: Bare Demo of IEEEtran.cls for IEEE Journals

Air-to-Ground Cooperative OAM Communications
    Ruirui Chen, Yu Ding, Beibei Zhang, Song Li, and Liping Liang
This work is supported by the Fundamental Research Funds for the Central Universities under Grant 2019QNB01. (Corresponding author: Liping Liang.)

Ruirui Chen, Yu Ding, Beibei Zhang, and Song Li are with School of Information and Control Engineering, China University of Mining and Technology, Xuzhou, 221116, China (emails: rrchen, yding, lb15060029, lisong@cumt.edu.cn).
Liping Liang is with State Key Laboratory of Integrated Services Networks, Xidian University, Xi'an, 710071, China (e-mail: liangliping@xidian.edu.cn). Beibei Zhang is also with Jiangsu Automation Research Institute, Lianyungang, 222061, China.




    August 5, 2024
================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================



§ ABSTRACT 
For users in hotspot region, orbital angular momentum (OAM) can realize multifold increase of spectrum efficiency (SE), and the flying base station (FBS) can rapidly support the real-time communication demand. However, the hollow divergence and alignment requirement impose crucial challenges for users to achieve air-to-ground OAM communications, where there exists the line-of-sight path. Therefore, we propose the air-to-ground cooperative OAM communication (ACOC) scheme, which can realize OAM communications for users with size-limited devices. The waist radius is adjusted to guarantee the maximum intensity at the cooperative users (CUs). We derive the closed-form expression of the optimal FBS position, which satisfies the antenna alignment for two cooperative user groups (CUGs). Furthermore, the selection constraint is given to choose two CUGs composed of four CUs. Simulation results are provided to validate the optimal FBS position and the SE superiority of the proposed ACOC scheme.



Air-to-ground, orbital angular momentum (OAM), cooperative, spectrum efficiency (SE), antenna alignment.






§ INTRODUCTION


Orbital angular momentum (OAM) has attracted a lot of interest in wireless communications for high spectrum efficiency (SE) transmission <cit.>. As a new orthogonal resource, OAM mode produces the helical phase front, which can be utilized for multiplexing communications <cit.>. The number of OAM mode is theoretically infinite, thus providing vortex wave communications with huge potential for significant SE improvement <cit.>.

By using the reflector antenna, the authors of <cit.> successfully separated two vortex wave signals at the same time and same frequency for the first time in 2011. To complete the reception of the vortex wave signal, the general receiving scheme utilized the reflector antenna and centralized antenna array <cit.>. For  ideal OAM communications, the authors of <cit.> proposed the mode combination method, which can achieve orthogonal OAM sub-channels at the same time and frequency. To improve information transmission, the authors in <cit.> combined OAM with index modulation by using different OAM modes. The authors of <cit.> and <cit.> enhanced the channel capacity by integrating OAM with non-orthogonal multiple access (NOMA). Note that OAM communications require the line-of-sight (LoS) path, which can not be guaranteed in terrestrial wireless channel that mainly consists of non-LoS (NLoS) paths <cit.>. The air-to-ground link, which is LoS-dominated, can be rapidly established by the flying base station (FBS) in hotspot region <cit.>. Thus, we concentrate on OAM communications in the air-to-ground scenario.


Since the wavelength at radio frequency is much larger than that at light frequency, the hollow divergence imposes the critical but challenging problem for air-to-ground OAM communications <cit.>.
The energy ring of the OAM beam expands as the communication distance increases, thus making the size of the receiving antenna big at the long distance <cit.>.
In general, the user has size-limited device and only one antenna, thus hardly receiving the vortex wave signal for high SE transmission <cit.>.
As the easily realized receiving scheme, the cooperative OAM communication scheme was proposed for users in hotspot region to achieve 2 OAM mode multiplexing <cit.>.
The other challenge for air-to-ground OAM communications is the antenna misalignment, which seriously limits the application of the OAM mode multiplexing.
If the transmitting and receiving antennas are not aligned, it is difficult to decode the vortex wave signals with different OAM modes <cit.>.
Due to the random distribution of users in hotspot region, the proposed cooperative OAM communication scheme in <cit.> still has the antenna misalignment scenario, thus requiring beam steering for performance compensation of OAM communications.







To break through the hollow divergence and antenna alignment limitations, we propose the air-to-ground cooperative OAM communication (ACOC) scheme to realize OAM mode multiplexing communications for users with size-limited devices in hotspot region.
The main contributions of this letter can be summarized as follows: 1) We adjust the waist radius to guarantee the maximum intensity at the cooperative users (CUs) according to the distance between the CUs; 2) The optimal FBS position is derived in closed-form solution, which satisfies the alignment requirement of air-to-ground OAM communications; 3) We give the selection constraint to choose two cooperative user groups (CUGs) composed of four CUs.



§ SYSTEM MODEL

In this letter, there are U users and a FBS in hotspot region due to the concert, stadium game, etc. We denote the user set by 𝒰={1,2⋯,U}. The users all have only one antenna and are randomly distributed in hotspot region.
The FBS has the uniform circular array (UCA), which consists of N antenna elements represented by the set 𝒩={1,2⋯,N}.

Moreover, the equipped UCA has the radius R and central angle φ=2π/N. The channel is assumed as the LoS-dominated and free-space path loss model. The location information of users is available to the FBS.


In the 3D cylindrical coordinate system, the u-th (u∈𝒰) user is located at (r_u, θ_u, 0) and the FBS flies at the height H.
The users follow the uniform distribution on the ground (z=0) in hotspot region.

The OAM beam is the Laguerre-Gaussian (LG) beam with the radial index p=0, the intensity distribution of which can be expressed as follows:

    I_ℓ(r,z)=2/π w^2(z)|ℓ |! [√(2)r/w(z)]^2|ℓ |exp [-2r^2/w^2(z)],

where w(z) = w_0√(1 + (z/z_R)^2) is the radius of the beam at z and the azimuthal index ℓ denotes the OAM mode. w_0 represents the waist radius at z=0. z_R=π w_0^2/λ is the Rayleigh range, where λ denotes the wavelength.

Due to the strict convexity of I_ℓ(r,z) with respect to r, the optimal radial radius r, which corresponds to the maximum I_ℓ(r,z), can be derived as

    r_(z)=√(|ℓ |/2)w(z).

Based on Eq. (<ref>) and size limitation of the FBS, we select the smaller waist radius w_0, i.e.,

    w_0=√(r_^2(z)/|ℓ |- √(r_^4(z)/|ℓ |^2-z^2λ^2/π^2)).

The following condition of the feasible radial radius should be guaranteed for the existence of the waist radius w_0.

    r_(z) ≥ r_(z) ≜√(zλ |ℓ |/π)


In general, the user is equipped with size-limited device and only one antenna, thus hardly receiving the vortex wave signal. To achieve L OAM mode multiplexing communications, L antennas should be deployed uniformly alongside the 2π/a arc and different OAM modes l_m  (m=1, 2, 3...M) should satisfy the equation l_m-l_0=ma, where l_0 and m are respectively the constant integer and arbitrary integer <cit.>. We can see that L CUs can cooperate with each other to achieve L OAM mode multiplexing communications, which requires the CUs to satisfy the uniform distribution alongside the circle. It is hard to realize OAM communications with the large number of OAM modes. Therefore, we choose two CUs as one CUG to achieve air-to-ground cooperative OAM communications.



§ AIR-TO-GROUND COOPERATIVE OAM COMMUNICATION SCHEME


We define two CUs as the u_1-th and u_2-th users, which can be considered as CUG 1. The distance between two CUs is denoted by d_u_1,u_2. Similarly, CUG 2 consists of the u_3-th and u_4-th users, the distance between which can be represented by d_u_3,u_4.

According to Eq. (<ref>), two CUs should be close to the radial radius with maximum intensity. To this end, two CUs should have same distance to the FBS and be at the diameter of the circle, which is shown in Fig. <ref>. For CUG 1, two CUs should be located at the blue circle with radius r_(z_1), r_u_1 =r_u_2 =r_1, and d_u_1,u_2=2r_(z_1), where z_1 represents the distance from the UCA center of the FBS to the midpoint of the connection between two CUs, which is written as

    z_1 = √(r_1^2 - d_u_1, u_2^2/4 + H^2).

The FBS with the height H sends vortex wave signals to CUG 1 through the direction of z_1, which is marked by the blue solid line in Fig. <ref>. Note that z_1 can be considered as the transmission distance of the vortex wave signal. Based on Eq. (<ref>), the transmission distance of the vortex wave signal will increase with the increase of the FBS height H. Thus, the FBS should fly at the allowable minimum flying height H_.

For the CUs, we define the maximum communication distance as d_.

The waist radius w_0 is adjusted by the FBS to guarantee that two CUs are at the diameter of the circle with radius r_(z_1). That is to say, we change the waist radius w_0 according to the radial radius r_(z_1)=d_u_1, u_2/2, and the direction of the OAM beam is from the UCA center of the FBS to the midpoint of the connection between two CUs. According to Eq. (<ref>), the condition d_u_1,u_2≥ 2r_(z_1) should be guaranteed to make the waist radius exist.
Similarly, CUG 2 has the propagation direction z_2, and should be located at the green circle with radius r_(z_2), r_u_3 =r_u_4 =r_2, and d_u_3,u_4=2r_(z_2).

This letter aims to realize aligned air-to-ground OAM communications for two CUGs by utilizing the mobility of the FBS. To this end, we derive the closed-form expression of the optimal FBS position, which realizes aligned air-to-ground cooperative OAM communications. Furthermore, the selection constraint is given to choose two CUGs composed of four CUs.



 §.§ Optimal FBS Position


Considering the FBS mobility, the optimal position of the FBS is the midpoint of the connection between two CUs, which can guarantee the minimum transmission distance for one CUG. It is easy for one CUG to obtain the maximum SE by adjusting the position of the FBS and the waist radius of the OAM beam. Therefore, we study the optimal FBS position to realize the air-to-ground cooperative OAM communications for two CUGs composed of four CUs. To guarantee the antenna alignment, the propagation direction for the CUG should be the perpendicular bisector of the connection between two CUs. In theory, the antenna alignment can be guaranteed if the FBS is at the perpendicular bisection plane of the connection between two CUs. To guarantee antenna alignment of two CUGs, the optimal FBS position should be at the height H_ and the intersection line of the two perpendicular bisection planes of two CUGs.



We give Fig. <ref> to demonstrate the optimal position of the FBS, which satisfies the alignment requirement of air-to-ground OAM communications. As shown in Fig. <ref>, we find the perpendicular bisection plane of the connection between u_1-th and u_2-th CUs, which can be denoted by PB1. Similarly, the perpendicular bisection plane of the connection between u_3-th and u_4-th CUs can be represented by PB2. Then, we can find the intersection line of the perpendicular bisection planes PB1 and PB2, which is marked by the red line in Fig. <ref>. Therefore, the optimal position of the FBS is at the height H_ and the intersection line of the two perpendicular bisection planes PB1 and PB2, which can be derived in the following.

The location of the u-th  (u ∈ U) user can also be denoted by (x_u,y_u,0), where x_u = r_ucosθ _u and  y_u = r_usinθ _u. Similarly, CUs u_1, u_2, u_3, and u_4 are located at (x_u_1,y_u_1, 0), (x_u_2,y_u_2, 0), (x_u_3,y_u_3, 0), and (x_u_4,y_u_4, 0), respectively. In Fig. <ref>, the midpoint of the connection between CUs u_1 and u_2 can be represented by M_1, the location of which is at (a_1,b_1, 0). Similarly, the midpoint of the connection between CUs u_3 and u_4 can be denoted by M_2, the location of which is at (a_2,b_2, 0). The intersection of the perpendicular bisectors can be denoted by (x_,y_, 0), which satisfies the following equations

    {[  y_ - b_1/x_ - a_1 =  - 1/k_1; y_ - b_2/x_ - a_2 =  - 1/k_2, ].

where k_1=y_u_2-y_u_1/x_u_2-x_u_1 and k_2=y_u_4-y_u_3/x_u_4-x_u_3.
By solving the equations, we can derive

    {[ x_ = (b_2 - b_1)k_1k_2 + a_2k_1 - a_1k_2/k_1 - k_2;        y_ = b_1k_1 - b_2k_2 + a_1 - a_2/k_1 - k_2. ].

Based on Eq. (<ref>), we can obtain the optimal FBS position as the closed-form expression (x_,y_, H_), which is at the height H_ and the intersection line of the two perpendicular bisection planes PB1 and PB2.



 §.§ CU Selection




There exists the optimal position of the FBS if four CUs u_1, u_2, u_3, and u_4 are not at one straight line. Note that it is a very small probability that four CUs are at one straight line. Therefore, four CUs can almost always be selected to form two CUGs in general. In Fig. <ref>, we give the planar demonstration of two CUGs, which is a quadrilateral with vertices u_1, u_2, u_3, and u_4. The diagonals of the quadrilateral can be represented by d_u_1, u_3 and d_u_2, u_4, which should satisfy the following condition

    {d_u_1, u_3, d_u_2, u_4}≤ 2r_sr,

where r_sr is the FBS service radius. Eq. (<ref>) can guarantee that four CUs can be served by the FBS. The distances of two CUGs should satisfy the following conditions

    2r_fea(z_1) ≤ d_u_1, u_2≤ d_mc

and

    2r_fea(z_2) ≤ d_u_3, u_4≤ d_mc,

which can guarantee the efficient information transmission for two CUGs. Based on the above analysis, two CUGs composed of four CUs, which satisfy Eqs. (<ref>), (<ref>), and (<ref>), can be chosen to complete aligned air-to-groud cooperative OAM communications. Therefore, Eqs. (<ref>), (<ref>), and (<ref>) can be considered as the selection constraint to choose two CUGs composed of four CUs.


The inner angle set of the quadrilateral composed by two CUGs can be expressed as

    Ψ={ψ_1, ψ_2, ψ_3, ψ_4}.
For the inner angle set Ψ, the angle square difference can be given in the following.

    Ψ=∑^4_i=1(ψ_i-π/2)^2

To further improve the SE, we should choose two CUGs composed of four CUs, which have the minimum Ψ and satisfy Eqs. (<ref>), (<ref>), and (<ref>).


 §.§ Scheme Description

For CU selection, the first CU u_1 can be the boundary user that has the maximum r_u. The CUs u_2, u_3, and u_4 can be chosen as the users closest, second closest and third closest to CU u_1. We define the selected CU set as 𝒰_c.
Then, we propose the ACOC scheme, which can be summarized as follows.

[0.25]9cm0.5pt
Air-to-Ground Cooperative OAM Communication Scheme
Initialization:
 1) Given user set 𝒰, user location information (r_u, θ_u, 0)
     and arbitrarily small value ϵ;
 2) Set Ψ=+∞, 𝒰_c=∅ and choose boundary user as CU
     u_1 in 𝒰;
Iteration:
 1) while (Ψ>ϵ)&(|𝒰|>3) do
 2)   Find CUs u_2, u_3, u_4 and obtain d_u_1, u_2, d_u_3, u_4;
 3)     if (2r_fea≤ d_u_1, u_2≤ d_mc)&(2r_fea≤ d_u_3, u_4≤ d_mc)
 4)        Calculate d_u_1, u_3 and d_u_2, u_4;
 5)        if ({d_u_1, u_3, d_u_2, u_4}≤ 2r_sr)
 6)           Obtain Ψ_in based on Eq. (<ref>);
 7)           if (Ψ>Ψ_in)
 8)              Set Ψ=Ψ_in and 𝒰_c={ u_1, u_2, u_3, u_4 };
 9)           end if
10)        end if
11)     end if
12)   Set 𝒰=𝒰∖ u_1 and u_1=u_2;
13) end while
14) According to locations of selected four CUs, obtain
     optimal FBS position (x_,y_, H_) based on Eq. (<ref>).
[0.25]9cm1pt

In initialization steps, we give the arbitrarily small value ϵ, and choose boundary user as CU u_1 in 𝒰. For each iteration, we choose CUs u_2, u_3, and u_4, which are the users closest, second closest and third closest to CU u_1 (step 2 of iteration). If the distances d_u_1, u_2 and d_u_3, u_4 both belong to [2r_fea, d_mc], we calculate d_u_1, u_3 and d_u_2, u_4 (steps 3-4 of iteration). Ψ_in is calculated based on Eq. (<ref>) when the four CUs all can be served by the FBS (steps 5-6 of iteration). If Ψ_in is smaller than the current angle square difference, we update the four CUs and set Ψ_in as the current angle square difference (steps 7-8 of iteration). Then, we remove the CU u_1 from 𝒰, and set u_2 as new u_1 in the next iteration (step 12 of iteration). When the iteration terminates, we can obtain four CUs in 𝒰_c. The optimal FBS position is at the height H_ and the intersection line of the two perpendicular bisection planes of the two CUGs (step 14 of iteration).



§ SIMULATION RESULTS


In this section, simulation results are provided to evaluate the performance of our proposed ACOC scheme. The users follow the uniform distribution in hotspot region that is the square area. For performance comparison, we give the existing cooperative OAM wireless (COW) communication scheme, where the CUs are served by the ground BS. Note that our proposed ACOC scheme can obtain the optimal FBS position to realize the antenna alignment. The simulation parameter is shown in Table I.






Figure <ref> presents the variation of the SE with the FBS position. The FBS flies at the allowable minimum flying height H_. The optimal FBS position (x_,y_, H_) is marked by 73. We can see that the SE achieves the largest value at the optimal FBS position (x_,y_, H_), which can validate the closed-form expression of the optimal FBS position in Eq. (<ref>).



In Fig. <ref>, we compare the proposed ACOC shceme with the suboptimal scheme and random scheme. The FBS position of the suboptimal scheme only guarantees the antenna alignment of one CUG. The random scheme sets the random FBS position. The FBS height and user number are 50 m and 4000 in Fig. <ref>(a) and Fig. <ref>(b), respectively. For the three schemes, the SE monotonically increases/decreases as the user number/FBS height increases. The proposed ACOC scheme obtains higher SE than the suboptimal scheme and random scheme. This is due to the fact that the optimal FBS position, which satisfies alignment requirement, can achieve aligned air-to-ground OAM communications for two CUGs.


Figure <ref> presents the SE performance under various heights. From Fig. <ref>, we can see that the larger the user number is, the larger the SE is for the two schemes. Furthermore, the SE is a monotonic decreasing function of the FBS height. This is due to the fact that as the FBS height increases, the transmission distance of the vortex wave signal will increase, thus making the minimum communication distance between two CUs larger. With less user number, the proposed ACOC scheme still can obtain higher SE than that of the COW communication scheme.



In Fig. <ref>, we demonstrate the SE performance versus user number. For the two schemes, it can be observed that the SE monotonically increases as the user number increases. This is because as the user number increases, the two schemes both obtain the large probability to realize long distance information transmission. Compared with the COW communication scheme, the proposed ACOC scheme achieves better SE performance even with higher position.



§ CONCLUSIONS


To overcome the hollow divergence and antenna alignment limitations, this letter studied the ACOC scheme to realize air-to-ground OAM communications for users with size-limited devices in hotspot region. The waist radius was adjusted to make the CUs at the maximum intensity, which can guarantee the reception of vortex wave signal. The optimal FBS position was derived in closed-form solution to achieve aligned air-to-ground cooperative OAM communications. Furthermore, we give the selection constraint to choose two CUGs composed of four CUs. Simulation results showed that the proposed ACOC scheme can achieve high SE transmission at the long distance.

99
1
W. Cheng, W. Zhang, H. Jing, et al., “Orbital angular momentum for wireless communications," IEEE Wireless Communications, vol. 26, no. 1, pp. 100-107, Feb. 2019.
2
X. Ge, R. Zi, X. Xiong, et al., “Millimeter wave communications with OAM-SM scheme for future mobile networks," IEEE Journal on Selected Areas in Communications, vol. 35, no. 9, pp. 2163-2177, Sept. 2017.
3
W. Yu, B. Zhou, Z. Bu, et al., “UCA based OAM beam steering with high mode isolation," IEEE Wireless Communications Letters, vol. 11, no. 5, pp. 977-981, May 2022.
4
F. Tamburini, E. Mari, A. Sponselli, et al., “Encoding many channels on the same frequency through radio vorticity: first experimental test," New Journal Of Physics, vol. 14, no. 3, Art. 2012.
5
R. Chen, H. Zhou, M. Moretti, et al., “Orbital angular momentum waves: generation, detection, and emerging applications," IEEE Communications Surveys and Tutorials, vol. 22, no. 2, pp. 840-868, 2020.
6
K. A. Opare, Y. Kuang and J. J. Kponyo, “Mode combination in an ideal wireless OAM-MIMO multiplexing system," IEEE Wireless Communications Letters, vol. 4, no. 4, pp. 449-452, Aug. 2015.
7
E. Basar, “Orbital angular momentum with index modulation," IEEE Transactions on Wireless Communications, vol. 17, no. 3, pp. 2029-2037, Mar. 2018.
8
A. A. Amin and S. Y. Shin, “Capacity analysis of cooperative NOMA-OAM-MIMO based full-duplex relaying for 6G," IEEE Wireless Communications Letters, vol. 10, no. 7, pp. 1395-1399, Jul. 2021.
9
A. A. Amin and S. Y. Shin, “Channel capacity analysis of non-orthogonal multiple access with OAM-MIMO system," IEEE Wireless Communications Letters, vol. 9, no. 9, pp. 1481-1485, Sept. 2020.
10
X. Pang, M. Liu, N. Zhao, et al., “Secrecy analysis of UAV-based mmWave relaying networks," IEEE Transactions on Wireless Communications, vol. 20, no. 8, pp. 4990-5002, Aug. 2021.
11
R. Chen, Y. Sun, L. Liang, et al., “Joint power allocation and placement scheme for UAV-assisted IoT with QoS guarantee," IEEE Transactions on Vehicular Technology, vol. 71, no. 1, pp. 1066-1071, Jan. 2022.
12
R. Chen, J. Zhou, W. Long, et al., “Hybrid circular array and luneberg lens for long-distance OAM wireless communications," IEEE Transactions on Communications, vol. 71, no. 1, pp. 485-497, Jan. 2023.
13
K. Liu, H. Liu, Y. Qin, et al., “Generation of OAM beams using phased array in the microwave band," IEEE Transactions on Antennas and Propagation, vol. 64, no. 9, pp. 3850-3857, Sept. 2016.
14
R. Chen, W. Cheng, Y. Ding, et al., “QoS-guaranteed multi-UAV coverage scheme for IoT communications with interference management," IEEE Internet of Things Journal, 2023. Available:https://ieeexplore.ieee.org/document/10201847.
15
R. Chen, W. Cheng, J. Lin, et al., “Cooperative orbital angular momentum wireless communications," IEEE Transactions on Vehicular Technology, 2023. Available:https://ieeexplore.ieee.org/document/10232879.
16
X. Xiong, H. Lou and X. Ge, “Modeling and optimization of OAM-MIMO communication systems with unaligned antennas," IEEE Transactions on Communications, vol. 70, no. 6, pp. 3682-3694, Jun. 2022.
17
W. Zhang, S. Zheng, Y. Chen, et al., “Orbital angular momentum-based communications with partial arc sampling receiving," IEEE Communications Letters, vol. 20, no. 7, pp. 1381-1384, Jul. 2016.



 Paper Title: Shadow Area and Degrees-of-Freedom for Free-Space Communication.




Data-driven Modeling for Grid Edge IBRs: A Digital Twin Perspective of User-Defined Models 





    Kaveri Mahapatra*, Bhaskar Mitra, and Soumya Kundu
Pacific Northwest National Laboratory, United States 



{kaveri.mahapatra, bhaskar.mitra, soumya.kundu}@pnnl.gov































    
=======================================================================================================================================================================================================

§ ABSTRACT 
The number of degrees-of-freedom (NDoF) in a communication system is limited by the number of antenna ports, element shapes, positions, and the propagation environment. As the number of antenna elements increases within a given region, the NDoF eventually saturates due to correlation of the radiated fields. The maximal NDoF can be determined numerically for communication between two regions using singular value decomposition of a channel model representing wave propagation between densely sampled sources at the transmitter and fields at the receiver. This paper provides a straightforward analytical estimate of the NDoF for arbitrarily shaped transmitter and receiver regions. The analysis show that the NDoF for electrically large regions is approximated by the mutual shadow area of the regions, measured in wavelengths. Several setups illustrate the results, which are then compared with numerical evaluations of the singular values of the propagation channel. These new analytical expressions also simplify to previously established results based on Weyl's law and the paraxial approximation.



Degrees of freedom, capacity, antenna theory, electromagnetic theory, modes





§ INTRODUCTION

Electromagnetic degrees-of-freedom (DoF) are critical parameters in communication systems, providing a fundamental estimate of the system's potential performance <cit.>. The number of degrees-of-freedom (NDoF) is particularly significant, as the capacity of a communication system tends to increase approximately linearly with the NDoF. This relationship underscores the importance of accurately determining and maximizing the NDoF in the design and analysis of advanced communication systems.

In the context of wave-based communication, DoF encompasses various components, including temporal, spatial, and polarization aspects. Each of these components contributes to the overall capacity and efficiency of the communication system <cit.>. 
The concept of NDoF extends beyond communication systems and plays a crucial role in fields such as imaging, radar, and antennas. Early investigations by Di Francia in the mid-20th century highlighted the significance of NDoF in these areas <cit.>. For instance, in imaging, NDoF determines the resolution and quality of the reconstructed images <cit.>. 

Wave propagation in canonical geometries, such as spherical symmetric setups or rectangular waveguides, has traditionally been used to determine the NDoF for both electromagnetic and scalar fields <cit.>. These findings can be interpreted as specific manifestations of Weyl's law <cit.>, formulated in the early twentieth century by Hermann Weyl. Weyl's law describes the asymptotic eigenvalue distributions of Laplace and Helmholtz equations in bounded regions, demonstrating that the number of propagating waves, and hence NDoF, is proportional to the region's size, measured in wavelengths <cit.>.

These estimates are instrumental in determining the NDoF in arbitrarily shaped waveguides and for radiation emanating from convex-shaped regions. Additionally, Landau's eigenvalue problem offers a framework for analyzing NDoF <cit.>. Numerical techniques, such as singular value decompositions (SVD) combined with free-space propagation models utilizing Green's function (dyadic), further enable the determination of NDoF for regions of arbitrary shapes <cit.>. In recent years, extensive research has focused on future communication systems, necessitating a deeper understanding of wave propagation and NDoF in complex environments <cit.>. 

This paper aims to bridge analytical and numerical approaches to provide insights into the NDoF between arbitrarily shaped transmitter and receiver regions, 
NDoF between arbitrarily shaped transmitter and receiver regions, 
thereby providing a comprehensive framework for assessing and enhancing the performance of modern communication and sensing systems. This paper provides an analytical estimate of the NDoF for free-space communication between arbitrarily shaped transmitter and receiver regions. The results suggest that the mutual shadow area of these regions, measured in wavelengths, accurately estimates the NDoF for sufficient short wavelengths. The mutual shadow area is easily evaluated for various configurations, offering valuable physical insight and understanding. These new analytical expressions also include previously established results based on Weyl's law, shadow area <cit.>, and the paraxial approximation <cit.>. 

Numerical evaluations of the singular values of the propagation channel for several setups are used to illustrate the results. It is demonstrated that the singular values primarily depend on the NDoF, and plotting normalized singular values for different setups with the same NDoF reveals a tendency of the singular values to cluster for large NDoF. To address the computational challenge of evaluating singular values of the channel matrix as the electrical size increases, a randomized SVD algorithm exploiting the low-rank structure is employed <cit.>.

The setup and free-space communication channel are presented in Sec. <ref>. The NDoF and shadow area for a circumscribing receiver structure are discussed in Sec. <ref>, followed by a proposed generalization to a partly circumscribing structure in Sec. <ref>. Communication between volumes and mutual shadow area is introduced in Sec. <ref>. Secs. <ref> and <ref> present numerical results in 2D and 3D, respectively. The paper is concluded in Sec. <ref>.






§ FREE-SPACE COMMUNICATION CHANNEL

Communication between antenna systems confined to a transmitting region  and a receiving region  can be modeled by the radiation from current densities J_T(r) in  to fields E_R() in , as illustrated in Fig. <ref>. For simplicity, we consider transmitting electric currents J_T and receiving electric fields E_R, but the results and model are also applicable to scenarios involving both electric and magnetic currents, as well as electric and magnetic fields.









The transfer of information from  to  depends on the number of independent temporal and spatial channels that can be formed between these regions. The number of temporal channels is proportional to the available bandwidth <cit.> and is not strongly linked to the geometrical configuration of the regions for electrically large structures. Here, we focus on the spectral efficiency (capacity per unit bandwidth) <cit.>. Spectral efficiency depends on the spatial channels between the regions and the signal-to-noise ratio (SNR).

The spectral efficiency, or capacity, is determined by Shannon's formula for a channel formed by antenna systems in  and  at a given SNR <cit.>. This capacity increases with an improved SNR and by including more (ideal) antenna elements in the regions. The increase in capacity is nearly linear up to a certain number of elements and saturates beyond this point <cit.>. Increasing the number of elements after saturation requires a significantly higher SNR to further enhance the capacity. This threshold number of antenna elements (ports) is referred to as the number of degrees of freedom (NDoF) <cit.>. The NDoF depends on the geometrical configuration of  and  and the frequency f or wavelength λ = c/f, where c denotes the speed of light.

The NDoF between two regions is often determined from the channel formed by a transmitting array in _T and a receiving array in _R. These arrays are created by discretizing the regions into sub-wavelength spaced point or dipole elements <cit.>. This approach is analogous to decomposing the current density into a set of basis functions, where each basis function corresponds to an array element <cit.>. Using N_T transmitting elements and N_R receiving elements defines a channel matrix H with dimensions N_R× N_T.




Forming an SVD of the channel matrix H reveals a threshold number, after which the singular values decrease rapidly. The slope of this decay increases with increasing frequency (decreasing wavelength). The effective NDoF is a way to approximate this threshold and the NDoF as <cit.> 

    N_e 
    	=
    H_F^4/HH^_F^2 
    	= (∑_nσ_n)^2/∑_nσ_n^2,

where σ_n denotes the squared singular values of H, eigenvalues of the correlation matrix 

    σ_n=(HH^),

the superscript ^ denotes Hermitian transpose, and ·_F is the Frobenius norm <cit.>. 
These approaches and approximations have been thoroughly investigated in the literature <cit.>. The effective NDoF is easy to evaluate and useful for arbitrary configurations and wavelengths.

The effective NDoF and distribution of the singular values are independent of the normalization of the channel matrix H. For the presentation in this paper, it is convenient to normalize the channel matrix with its Frobenius H_F, H= H/H_F.  This normalization is equivalent to normalizing the eigenvalues σ_n as 

    ζ_n
        =σ_n/∑_nσ_n

with ζ_n denoting the eigenvalues of the normalized correlation matrix, ζ_n=(HH^). These normalized eigenvalues satisfy

    ∑ζ_n=1
          and ∑ζ_n^2 = N_e^-1.


In this paper, we use simple geometrical properties of the regions  and  to estimate a NDoF, , in the asymptotic limit λ→ 0. This estimate is compared with an ideal uniform channel having  non-zero eigenvalues, each with an amplitude of ^-1, as illustrated in Fig. <ref>. The ideal uniform channel has a sharp bend, often referred to as a corner or knee, at the point (1,1), when the eigenvalue index n and eigenvalues are normalized with .

The eigenvalues of a non-ideal channel decay rapidly after the NDoF estimated by , as shown for the channel between two square plates in Fig. <ref> using ∈{500, 1000}, corresponding to ℓ/λ∈{28, 40} wavelengths per side. This channel forms a corner with a finite curvature. The evaluation of  semi-analytically from the shadow of the geometrical structure and ζ_n numerically from the channel, as shown in Fig. <ref>, is discussed in the remainder of this paper.






§ NDOF ESTIMATES AND SHADOW AREA

Analytic solutions of the wave equation or Maxwell's equations can be used to estimate the NDoF for canonical geometries, such as regions with spherical or cylindrical symmetries, by expanding the field in spherical and cylindrical waves, respectively <cit.>.
The NDoF for a spherical region with radius a surrounded by a larger spherical region, see Fig. <ref>a, has been investigated using various techniques <cit.>. These studies have found that electrically small regions, with ka ≪ 1, exhibit six (polarization) DoF, while regions with ka ≫ 1 have (ka)^2 DoF per polarization, where k=2π/λ denotes the wavenumber. In the corresponding 2D case with circular (cylindrical) symmetry, the expansion in cylindrical waves yields approximately 2ka DoF for ka≫ 1. When expressed in terms of surface area A and circumference L. The NDoF for cylindrical and spherical regions become 2L/λ and π A/λ^2, respectively.





The spherical and circular geometries can be considered special cases of Weyl's law <cit.>. Weyl's law describes the asymptotic distributions of eigenvalues for the Laplace and Helmholtz operators in a region with Dirichlet or Neumann boundary conditions <cit.>. For arbitrarily shaped waveguides, the number of propagating (scalar) modes is asymptotically given by Weyl's law <cit.>

    N_W≈
     2L/λ  and 
    	N_W≈
     π A/λ^2

for a line with length L and an aperture surface with area A, respectively. 

Weyl's law estimates the NDoF for arbitrary convex-shaped regions  circumscribed by a large spherical (cylindrical) receiving region  <cit.>. The NDoF, as quantified by area and length in Weyl's law, can be interpreted in terms of sampling. Specifically, the density of independent transmitters on a line is λ/2, consistent with Nyquist sampling and array theory, while it is λ/√(π)≈λ/1.8 for a surface <cit.>. However, the surface area A and length L in Weyl's law do not account for blockage for non-convex shapes.

The NDoF for non-convex transmitting regions , see Fig. <ref>b, is determined by investigating the ability of the transmitter to generate waves in directions k or, by reciprocity, to receiving plane waves propagating in the opposite direction <cit.>. This concept leads to the notion of shadow area in the asymptotic limit λ→ 0 <cit.>. The shadow area of a region  for a direction k quantifies how transmitters confined to the region can direct radiation in that direction. For convex shapes, this reduces to the classical cross-sectional area, commonly used to estimate the effective area for aperture antennas <cit.>. 

An object circumscribed by a (convex) receiving region  can interact with plane waves propagating in any direction, necessitating a summation over all directions. The total shadow area is formed by integrating the shadow area over a sphere (or circle for a line). The resulting NDoF for an arbitrarily shaped object surrounded by a large spherical region is approximately <cit.>

    N_A = L_s/λ  and 
        N_A = /λ^2,

where  denotes the total shadow area (L_s for length) of the object and λ is much smaller than the object's dimensions. These results reduce to Weyl's law for convex shapes, as the average shadow area is a quarter of the surface area = A/4 <cit.>, giving the total shadow area =4π=π A, consistent with Weyl'law (<ref>). Similarly, in 2D L_s=2πL_s=2L.

The total shadow area estimates the NDoF that can be transmitted from a region  to the far field <cit.>. This provides a limit that is independent of the receiver location and configuration. For line-of-sight communication between well separated planar regions the paraxial approximation can be used <cit.>. This asymptotic solution for narrow angular propagation provide closed form expressions for the NDoF for planar regions with areas A_T and A_R separated a distance d, see Fig. <ref>c, as <cit.>

    N_P = L_TL_R/dλ  and 
        N_P = A_TA_R/d^2λ^2

(per polarization) with d much larger than the object sizes. The paraxial approximation can also be related to the shadow area (length), specifically the shadow area (length) of the transmitter cast on the receiver. For large distances and an infinitesimally small transmitter region, this length is given by L_TL_R/d and the area by A_TA_R/d^2. Summing over the transmitter yields (<ref>).




§ PARTLY CIRCUMSCRIBING STRUCTURE



NDoF for communication between transmitters in the region  to receivers in the far field partly circumscribing the transmitter, or similarly, on a large partly circumscribing sphere , as shown in Fig. <ref>a is investigated first. It reduces to the case with circumscribing receivers (<ref>) in one limit and the paraxial approximation (<ref>) in the other. Current density J_T() in the transmitting region  constitutes sources for the radiated field and are expanded in sufficiently large number of basis functions, with expansion coefficients collected in a column matrix I <cit.>. It is assumed that the transmitting region  is constructed of some material with minimal material losses described by a loss matrix R_ρ <cit.>.
Similarly, the radiated field over the receiver region is obtained by integrating the field over . This integration is power normalized and involves a quadrature scheme with quadrature weights Λ_p^2 for quadrature points (directions) k_p and polarizations e_p with p∈[1,N_R]. Here, we for simplicity assume that the quadrature weights are positive, negative weights such as in Lebedev quadrature can also be considered <cit.>. We use a spherical (or cylindrical) coordinate system with directions k_p described by polar θ_p and azimuthal ϕ_p coordinates and polarizations in the θ_p and ϕ_p directions <cit.>. 
The quadrature weights can also be interpreted as an SNR weight for a communication system, which also include point measures for receivers in discrete directions.

Collect the received field in a column matrix f=ΛFI, where F is a matrix describing the linear map from the currents to the received field in the quadrature points k_p and polarizations e_p <cit.> including the square root of the quadrature weights collected in the diagonal matrix Λ. The capacity (spectral efficiency) of this communication system f=ΛFI+n constrained by (R_xP)=1 is determined by <cit.>

    log_2((+γΛFPF^Λ)) 
             (R_xP)=1 
             P≽0,

where P=ℰ{II^} denotes the covariance matrix of the currents,  γ quantifies the SNR from the additive noise n, and  denotes the identity matrix. In (<ref>), the current is without loss of generality normalized to a fixed value. The power constraint (R_xP)=1 with R_x a symmetric positive definite matrix is used to model different communication situations. Considering the dissipated power 1/2IRI in radiation and material losses uses the method of moments (MoM) resistance matrix <cit.> R_x=R.
The resistance matrix R can be decomposed into its radiation R_0 and material part R_ρ <cit.>, where 1/2I^R_0I and 1/2I^R_ρI model radiated power and power dissipated in ohmic or dielectric losses. Constraining only material losses uses R_x=R_ρ  in (<ref>). These material losses can also be interpreted as a constraint on the amplitude of the current density for a region with homogeneous losses modelled by a resistivity ρ_r and the loss matrix  R_ρ=ρ_rΨ with the Gram matrix <cit.> Ψ and least-squared norm of the current density I^ΨI, see App. <ref>. Alternative formulations of (<ref>) include constraints on efficiency <cit.> or bandwidth from the reactive energy around the antenna <cit.>. 

The capacity (<ref>) is determined by the water-filling algorithm <cit.> for which it is convenient to rewrite (<ref>) by a change of variable P=GPG^, where R_x=G^G is a factorization of a symmetric positive definite matrix <cit.>. Substituting P into (<ref>) results in

    log_2((+γHPH^))          
             (P)=1          
             P≽0

with the channel matrix H=ΛFG^-1. This optimization problem is diagonalized by an SVD of the channel matrix H, square roots of the eigenvalues of H^H = G^-F^Λ^2FG^-1, which can be written G^-F^Λ^2FG^-1V_n = ν_nV_n. Simplifying by multiplication with G^ and setting I_n=G^-1V_n results in the generalized eigenvalue problem

    F^Λ^2FI_n = ν_nR_xI_n.

This eigenvalue problem reduces to radiation mode efficiency for a receiving structure circumscribing the transmitter <cit.>, and here we for simplicity refer to the modes I_n  in (<ref>) as generalized radiation modes. 
These modal currents are orthogonal over the resistance matrix R_x and region 

    I_m^R_xI_n=δ_mn  and f_m^f_n=I_m^F^Λ^2FI_n=ν_nδ_mn.

Generalized radiation modes (<ref>) maximize the (Rayleigh) quotient between the radiated power over the directions  and dissipated power quantified by R_x 

    radiated power in /dissipated power
    	=
    	I_m^F^Λ^2FI_n/I_m^R_xI_n
    	=δ_mnν_n.

Using R_x=R interprets the eigenvalues ν_n∈[0,1] as an efficiency for orthogonal currents in  where only radiation in the region  is accounted for. 

Geometry and material properties of  and configuration of  are contained in the generalized radiation modes (<ref>).
The generalized radiating modes diagonalizes (<ref>) which solved using waterfilling <cit.> over radiation mode efficiencies ν_n results in

    max_∑P̃_n=1∑_n=1^Nlog_2(1+γν_nP̃_n),
  
with a finite number of non-zero power levels P̃_n associated with radiation modes of sufficiently high efficiency. 
Water-filling algorithm solves (<ref>) and distributes the power P̃_n over the modes <cit.> analogous to filling a bucket with heights proportional to ν_n^-1 with water.

The reciprocal of the normalized eigenvalues (ζ_n)^-1 are plotted in Fig. <ref> for the channel between the two rectangles shown in Fig. <ref>. 
For a given SNR (power level), illustrated by the dashed line in Fig. <ref> the allocated power is proportional to the distance down to (ζ_n)^-1. This illustrates the high cost in SNR to use more than  modes. The eigenvalues (ζ_n)^-1, related to the problem in (<ref>) reduces to (<ref>) for a matrix R_x=R_ρ that is proportional to the identity matrix. This occurs, in the case of homogeneous losses and a current density expanded in identical and orthogonal basis functions, or when uniformly sampled with point sources, see App. <ref>.   




Determination of the NDoF from the number of generalized radiation modes (<ref>) is easily achieved numerically for arbitrary shaped objects, material parameters, and receiver configurations. Analytical solutions in the asymptotic limit λ→ 0 complement the numerical results and provide physical insight and understanding. 

An analytical solution is found by relating the sum of the eigenvalues ν_n to geometrical properties of  and  obtained by the maximal effective area. The sum of the eigenvalues ν_n is given by the trace <cit.> 

    ∑_n=1^N ν_n=(G^-F^Λ^2FG^-1)
        =(R_x^-1F^Λ^2F) 
    
        =(FR_x^-1F^Λ^2)
        =∑_p=1^N_pΛ_p^2F_pR_x^-1F_p^,

where cyclic permutations of the trace operator  are used and F_p denotes row p of F. 

The identity (<ref>) is valid for arbitrary constraints R_x in (<ref>). Here, we consider R_x=R to constrain the dissipated power (radiation and losses) and R_x=R_ρ to constrain material losses (or equivalently the norm of the current density). For the case with R_x=R, the final terms in (<ref>) F_pR^-1F_p^ are recognized from the maximal partial gains <cit.> or maximal extinction-cross section <cit.> in directions k_p and polarizations e_p corresponding to row p of F. 

The partial effective area A_eff=λ^2 G/(4π), with the partial gain G in a direction k and polarization e is given by the partial radiation intensity normalized by the dissipated power and 4π <cit.>. Expressing the gain and effective area in the current matrix I, resistance matrix R, and radiation matrix F_p yields a framework useful for optimization over the currents. The maximal partial effective area is determined from the solution of the optimization problem <cit.>

    A_eff,p=λ^2I^F_p^F_pI
             I^RI=1,

where F_pI is proportional to the e-component of the far field in the direction k, and I^F_p^F_pI to the corresponding partial radiation intensity <cit.>. The solution of (<ref>) is

    max A_eff_p = λ^2F_p^R^-1F_p,

where the final terms in (<ref>) are recognized.
Inserting (<ref>) into (<ref>) produces the estimate

    ∑_n=1^N ν_n 
    =1/λ^2∑_p=1^N_pΛ_p^2
    max(k_p,e_p),

where by refining the quadrature rule over spatial points the sum approaches an integral

    ∑_n=1^N ν_n 
    = 1/λ^2∑_e={θ,ϕ}∫_ A_eff(k,e)Ω_k,

which can also be expressed in the maximum gain. This expression resembles the NDoF for a circumscribing sphere <cit.>, setting =4π.

The generalized radiation modes ν_n are related to the geometrical structure by observing that the maximal effective area in a direction k approaches the geometrical cross section, (k), in the electrically large limit <cit.> max(k) ≈(k). This connects the generalized radiation modes and geometrical properties of the region  
with the maximal effective area (<ref>) expressed solely in geometrical parameters producing the asymptotic NDoF

    ∑_n=1^Nν_n ≈2/λ^2∫_(k)Ω_k
        =2 A_TR/λ^2,

where the factor of two stems from the two polarizations {θ,ϕ}. The identity (<ref>) relates the efficiency of the generalized radiation modes with the shadow area A_TR. 

To estimate the NDoF, we assume that the efficiency of the generalized radiation modes separates into efficient modes and inefficient modes. This is similar to the decomposition of the channel eigenvalues ζ_n in Fig. <ref> which has approximately  modes ζ_n≈ 1 and thereafter rapidly decreasing eigenvalues. The normalized modes ζ_n are independent of the material loss (or similarly sampling density) but the channel eigenvalues increase as the sampling increases.  
The efficiency ν_n of the generalized radiation modes diminish rapidly, similar to ordinary radiation modes <cit.>. 
Assuming that the efficient generalized radiation modes have ν_n≈ 1 and inefficient modes have ν_n≈ 0 according to the ideal channel, see Fig. <ref>, resulting in efficiencies ν_n according to

    ν_n ≈
        1     n < 
    
        0     n > ,

where  denotes the NDoF for the given shape and frequency. The transition from large to small modal efficiencies decreases with increasing electrical size, see Fig. <ref>. Assuming this ideal channel, inserted into (<ref>) estimates the NDoF as

    =2A_TR/λ^2  as λ→ 0,

where the factor of two stems from the two polarizations of the EM field. The corresponding NDoF for 2D cases is

    =L_TR/λ  as λ→ 0,

where one polarization degree-of-freedom is assumed,  (<ref>).

These generalized radiation modes are slightly different from the modes induced by the decomposition of the channel matrix H in (<ref>) but is assumed to have similar NDoF. The mutual shadow length L_TR estimates the NDoF (<ref>) as the shadow length measured in wavelength. This estimate is compared with numerical evaluation of the channel between _T and _R. This channel can be modelled in many ways, points sources or expansion of a current density in basis functions. There is also a choice of using electric and/or magnetic current densities (single or double potentials). The presented results are for simplicity compared with the common model based on uniformly sampled (=λ/5) point sources <cit.>.



The mutual shadow length L_TR estimates the NDoF (<ref>) as the shadow length measured in wavelength. 
 This estimate is compared with numerical evaluation of the channel between _T and _R. To compare different configurations of transmitting and receiving regions, the singular values are evaluated for a fixed  by using (<ref>) to set the wavelengths to

    λ = L_TR/.


The shadow length for cylindrical structures (circle) with radius a is L_TR=2a· 2π=4π a which estimates the NDoF (<ref>) to 4π a/λ=2ka which is consistent with the number of cylindrical waves using a cutoff ka. For a partly surrounding receiver, the shadow length is proportional to the angular coverage, the π/2 coverage depicted in Fig. <ref> has shadow length π a. Normalized singular values ζ_n for 2π coverage and three cases of π/2 coverage are shown in Fig. <ref> using wavelengths (<ref>) corresponding to NDoF ∈{100,400}. 

The transmitter is modelled as a solid circle (or cylinder) with radius a. Using (<ref>) the NDoF ∈{100,400} corresponds to a/λ= a/L_TR∈{8,32} for the circumscribed case and a/λ∈{32,127} for the case with π/2 coverage. Here, we note that all cases show a rapid transition from nearly constant eigenvalues ζ_n≈ 1 for n< to rapidly decaying eigenvalues for n>. Moreover, the two =100 and =400 cases appear to separate, particularly the =400 traces almost overlap for n>. 

The presented results are evaluated for receivers in the far field. The same setup is evaluated for a receiver region at a finite distance, a receiver at radius 5a, but the results are indistinguishable. This suggests that the same approach using rays (plane wave) can be used for object at finite distances. 




§ COMMUNICATION BETWEEN VOLUMES AND MUTUAL SHADOW AREA

To determine the asymptotic NDoF for communication between two arbitrary shaped regions, we take a similar approach and focus on the propagation of plane waves between the regions. 
This effectively reduce the question of NDoF to the question about the number of waves that can be supported between the objects. These waves are constructed by a plane-wave expansion of the current density over all directions. This procedure is depicted in Fig. <ref>, where plane waves (rays) in a direction k are illustrated. To communicate from _T to _R with a ray in the direction k, the ray needs to intersect both regions. The amount of interaction is proportional to the mutual area of this interaction. This is analogues to the shadow area for the far-field or circumscribing case in (<ref>) and (<ref>) which can be interpreted as a total interception of plane waves (rays) from directions described by . 





A simple interpretation of this area comes from the mutual shadow area of the two objects, see Fig. <ref>. In this picture, the transmitting region _T illuminated by a plane wave in the direction k cast a shadow on the receiving region _R. This area can also be seen as the mutual shadow area of the two shadows. This intersection area integrated for all directions is here shown to be proportional to the NDoF between the two regions in the asymptotic limit λ→ 0.    






The total mutual shadow area is determined by integrating the mutual shadow area for all directions k

    L_TR = ∫_0^2π
        |(ϕ)|ϕ

in ^2 and

    A_TR = ∫_4π |(k)|Ω_k

in ^3, where the shadow region is determined from _T to _R, the shadow from  onto  is not included. In (<ref>) and (<ref>) || denotes the mutual shadow area (length) for a given direction of illumination. 
Here, we also note that the mutual shadow area can alternatively be determined from the difference between the sum of the shadow areas of _T and _T minus the shadow area of the composed object _TR. The NDoF for communication between systems confined to those regions are approximately

    = 
        
            L_TR/λ   in ^2
    
            A_TR/λ^2    in ^3

in agreement with (<ref>) and (<ref>).
These estimates (<ref>) of the asymptotic number of propagating modes for (scalar) Helmholtz equation are doubled for Maxwell's equations when two orthogonal polarizations contribute. These NDoF reduces to many previously known cases for those specific geometries, the case with a receiving spherical region circumscribing the transmitter (<ref>) and paraxial approximation (<ref>).  





Evaluation of shadows is well-developed in computer graphics <cit.>. Shadows for canonical shapes can be evaluated semi-analytically, while shadows for objects described by a mesh can be determined numerically by combining the shadows of each mesh element.

To illustrate shadow and mutual shadow areas, two spherical regions with radii a_T≥ a_R separated by a distance h as depicted in Fig. <ref> are considered. The spheres are, without loss of generality, placed on the z-axis, producing a body of revolution (BoR) object with the incident illumination parametrized by the polar angle θ. The shadow areas are determined on a plane perpendicular to the direction of illumination.



The shadow of each sphere is a circular disc with the same radius as the sphere. The two shadows can overlap, partly overlap, or being separated depending on the illumination angle θ related to the distance d=hsinθ between the disc centers. The shadow  is inside the shadow of  for sufficiently small polar angles θ<θ_1, see App. <ref> for details. This gives a mutual shadow area π a_R^2. For larger polar angles θ_1<θ<θ_2, the two shadows partly overlap, with a mutual shadow area given in App. <ref>. As the polar angle increases θ>θ_2, the two shadows separate, and the mutual shadow area vanish. For non-intersecting shadows, it is not possible to communicate between the regions using rays with this propagation direction.   

As the polar angle θ approaches π, there is a corresponding mutual shadow area from the receiving region  onto the transmitting region . This mutual shadow area has the same size but is not accounted for in the shadow area from  to  in (<ref>). 

The mutual shadow areas |(θ)| have simple analytical expressions and the total mutual shadow area A_TR is determined by integration of |(θ)| over [0,θ_2] weighted by 2πsinθ from integration over the azimuthal angle ϕ of the spherical coordinate system. 

Shadow lengths are also straight forward to evaluate in 2D. Starting with two line objects as depicted in Fig. <ref> and detailed in App. <ref>. The shadow from a line object is an interval with endpoints determined from the projection of the endpoints on a line p perpendicular to the direction of the illumination, p·k=0. The intersection length is finally determined from the intersection of the shadow intervals, see App. <ref> for details.     






§ NDOF AND SHADOW LENGTH IN 2D

To illustrate NDoF and shadow length, consider first two parallel lines with lengths ℓ_1 and ℓ_2 separated by a distance d as shown in the inset of Fig. <ref>. The mutual shadow length is determined by integrating |(ϕ)| over all illumination directions ϕ, with the transmitter casting a shadow on the receiver  

    2∫_0^ϕ_1ℓ_2cosϕϕ
        +2∫_ϕ_1^ϕ_2ℓ_1+ℓ_2/2cosϕ-dsinϕϕ

with tanϕ_1=δ and tanϕ_2=β, where δ=|ℓ_1-ℓ_2|/(2d) and β=(ℓ_1+ℓ_2)/(2d). Integration over ϕ leads to the total shadow length

    L_TR=2d(√(1+β^2) - √(1+δ^2))
        ≈
        2min{ℓ_1,ℓ_2}, d→ 0 
    ℓ_1ℓ_2/d,  d→∞,

where the paraxial result (<ref>) is recovered as d→∞ and Weyl's law for d≪ℓ, with a factor 1/2 due to radiation in one direction.

Different configurations of transmitting and receiving regions are compared by evaluating the eigenvalues of the channel correlation matrix for a fixed  by setting the wavelength to λ=L_TR/ and using uniformly sampled point sources with =λ/5 <cit.>.

The results in Fig. <ref> show the normalized eigenvalues ζ_n in (<ref>) of the resulting channel matrix for ∈{5,10,50,100} and distances d/ℓ∈{0.1,0.5,1,5}. 
The NDoF  is evaluated using (<ref>) as depicted in the inset of Fig. <ref>. The considered cases d/ℓ∈{0.1,0.5,1,5} are indicated by markers. Small distances have mutual shadow lengths L_TR≈ℓ corresponding to twice the shorter strip length (<ref>). The shadow length decreases with the distances and approximates the paraxial approximation as d/ℓ→∞. For d/ℓ = 5, the shadow length L_TR≈ℓ/10 implies a tenfold decrease in the NDoF for a fixed wavelength compared to the d≪ℓ case. The electrical sizes varies between ℓ/λ≈ 5 for =5 and d/ℓ=0.1 to ℓ/λ≈ 1000 for =100 and d/ℓ=5. The singular value index n on the horizontal axis is scaled with the NDoF . 


The normalized eigenvalues ζ_n are approximately equal to 1/ for n<. A 'corner' in the singular values is observed around n≈. For indices n>, the singular values ζ_n decay rapidly, particularly for large . 
This sharp transition from an almost constant ζ_n≈ 1 to a rapidly decaying ζ_n is used to define a NDoF, here denoted . This implies that there are  modes with approximately the same performance, and the performance decreases rapidly if additional modes are used, indicating a very high cost to use more than  modes. 
The results show that this 'corner' becomes more pronounced as the electrical size increases, or more precisely, as the NDoF increases. 


The results in Fig. <ref> indicate a strong correlation between the NDoF estimates from the shadow area  and from the corner of the eigenvalue distribution . They also show that the distribution of the eigenvalues ζ_n primarily depends on . The curves for the same  but different distances d/ℓ overlap, except for the short distance d=0.1ℓ and low ∈{5,10}. These cases correspond to a separation between the transmitting and receiving regions  d/λ= d/ℓ∈{0.5,1}, which is on the order of a wavelength. The results coincide for larger  corresponding to shorter wavelengths, supporting  as the NDoF as λ→ 0.  For sub-wavelength distances, the NDoF can be higher due to strong reactive coupling <cit.>.


The broadside configuration maximizes the mutual shadow length. Rotating one of the lines reduces this length. Consider a setup with two lines, one of length ℓ and the other ℓ/2, separated by a distance ℓ/2 and rotated by an angle ϕ, as shown in the inset of Fig. <ref>. The mutual shadow length is evaluated as described in App. <ref>. The mutual shadow length vanishes for the endfire configuration (ϕ = π/2), at which the estimate (<ref>) becomes invalid. The mutual shadow lengths L_TR are plotted in Fig. <ref> for ϕ∈{0,π/9,2π/9}. Additionally, a setup with rectangular regions is included in Fig. <ref>.




Normalized singular values ζ_n are depicted in Fig. <ref> for ∈{5,10,50,100} and the nine configurations shown in the inset and in Fig. <ref>. Normalizing by  groups the curves for the different configurations, indicating that the mutual shadow length is a fundamental parameter for the NDoF and the distribution of the eigenvalues. The spread within each group decreases as  increases, and the different configurations for the =100 case overlap for n>.




The results in Fig. <ref> depend slightly on the specific configuration and the point source model used for the channel matrix. This dependence can be attributed to the symmetry of the point source model, where the radiated field from a point source is isotropic. Consequently, the radiated field from a line region is symmetric in directions away from the line, reducing the NDoF. However, this symmetry is broken for regions with an inner region, such as an area in 2D or a volume in 3D. The symmetry is also broken by using both electric and magnetic current sources for electromagnetic fields <cit.>, similar to single and double layer potentials for scalar fields.




§ NDOF AND SHADOW AREA IN 3D

Shadow areas in 3D are also easily evaluated for arbitrary shaped regions using numerical techniques. Consider for simplicity a geometry composed by N convex shapes, n=1,...,N, each described by a set of points r_np, p=1,...,N_p. This can be a surface mesh with node points. Using spherical coordinates with the illuminating direction defined by the radial direction, , we project the shadow on the orthogonal plane spanned by the unit vectors θ and ϕ, see Fig. <ref>.
Using standard algorithms to find the convex hull of this set of points, convhull in matlab. This produces polygon-like shapes in 2D, for which it is easy to construct unions and intersections. 
Note that shadow areas can be determined in many ways, and that determining shadows is a standard procedure in computer vision <cit.>.

Mutual shadow area for three configurations using square plates with side lengths ℓ are depicted in Fig. <ref>. Parallel plates separated a distance d, similar to the setup in Fig. <ref>, are shown with a solid line. The mutual shadow area approaches A_TR≈πℓ^2 for d≪ℓ according to a half of Weyl's law (<ref>). For larger distances, it approaches the paraxial approximation A_TR≈ℓ^4/d^2.

Combining the vertical separation with a transverse shift is illustrated by the dashed curve. The effect of the transverse shift is negligible compared with the vertical shift for d≪ℓ, but its impact increases for d> ℓ. This is partly explained by the reduced shadow due to the projection of the planar regions along a 45^∘ angle and the increased distance which together reduces the mutual shadow area approximately a factor of 4 for d≫ℓ. The dotted line illustrates the case with one rectangle rotated 90^∘, demonstrating an end-fire configuration. This reduces the mutual shadow area, both for small and large distances. Note that the mutual shadow area vanishes for an end-fire setup of two plates in the same plane, producing a different scaling.  








Normalized eigenvalues ζ_n for the setups in Fig. <ref> using d/ℓ∈{0.5,1,2} and ∈{50,100,200} are shown in Fig. <ref>. The different setups are separated by solid, dashed, and dotted line styles as in Fig. <ref> and line widths for distances. The results show that ζ_n≈ 1 for n< and that the curves cluster for different  for n>. The slopes of the eigenvalues are less steep than for the corresponding 2D cases in Fig. <ref>. This is partly explained by the distribution of the DoF over two orthogonal directions. 

Evaluation of the channel for fixed NDoF  case corresponds to evaluation of the channel for different wavelengths. The top left case in Fig. <ref> has a mutual shadow area of A_TR≈ 1.3ℓ^2 which for the =200 case imply ℓ^2/λ^2=ℓ/A_TR≈ 154. Using 5 sample points per wavelength produces a channel matrix with dimension N_T=N_R≈ 4 10^3. This is easily handled on a standard PC. The corresponding bottom right case has a much smaller mutual shadow area A_TR≈ 0.0114ℓ^2 which produces a channel matrix of dimension N_T=N_R≈ 4 10^5. 

For these large dimensions, storage of H and evaluation of singular values or eigenvalues are challenging. However, using the low-rank property of the channel matrix H, rank(H)≈≪min{N_T,N_R} is much smaller than the dimension of H opens for evaluations based on randomized SVD <cit.>.   


Randomized SVD is an effective approach to evaluate the dominant singular values of low-rank matrices <cit.> and can be applied to evaluate (H), and hence ζ_n. Moreover, the randomized SVD is matrix-free, meaning that the full channel matrix is not needed for the evaluation of the singular values. This makes it suitable for cases with very large dimensions of H, when λ→ 0. The matrix-free randomized SVD can be interpreted as evaluating the received signals for a set of random excitations.

The rank of the channel matrix can be approximated by . By constructing a N_T× P random matrix A with, for example, P = 3, the received signals Y = HA can be evaluated, where Y has dimensions N_T× P, without evaluation of H. An orthonormal basis W of Y is then retransmitted through the adjoint channel, which is evaluated by exciting the channel from the receiver side using W^∗, B = W^H = (H^W^∗)^. This final matrix B is of size P × P and is therefore easy to evaluate using an ordinary SVD, (B). Its singular values approximate the dominant singular values of (H) <cit.> and hence square roots the eigenvalues σ_n in (<ref>).



§ CONCLUSIONS

This study demonstrates that the asymptotic NDoF for communication between regions is approximated by the mutual shadow area of the regions when measured in wavelengths. This analytical estimate provides valuable physical insights and complements numerical evaluations. The numerical results reveal that the eigenvalues of the channel are approximately constant up to the NDoF, after which they decay rapidly. Additionally, it is observed that the normalized eigenvalues of the channel correlation matrix tend to cluster for different setups with the same NDoF. The use of randomized singular value decomposition enables the evaluation of eigenvalues for the correlation matrix, facilitating simulations for setups with small wavelengths. The presented theory and results enhance the understanding of communication system performance in various setup of transmitter and receiver geometries in a free-space propagation environment.

The results presented here primarily focus on simple setups involving lines and plates. However, the approach is versatile and can be applied to arbitrarily shaped regions, including non-convex and non-connected geometries. In setups with symmetries, it may be necessary to consider both electric and magnetic current densities. The generalized radiation modes, derived from the radiated field over a finite far-field region, are also directly applicable to the study of characteristic modes. This framework facilitates the examination of the NDoF for fixed structures, offering valuable insights into their performance and behavior.





§ CURRENT DENSITY

The current density J() in  is expanded in a set of basis functions ψ_n() <cit.>

    J() = ∑_n=1^N_T I_nψ_n().

These basis functions can be local, such as divergence conforming or point sources, or global such as spherical waves. The least-squares, L^2, norm of the current density is

    ∫ |J|^2
        =∑_mn I_m^∗I_n∫ψ_m()·ψ_n()
        =I^ΨI,

where Ψ denotes the Gram matrix <cit.>. The L^2 norm is not defined for point sources, but for sub-wavelength sampling, it is possible to consider the point sources as an approximation of pulse basis functions, constant basis functions with unit volume (area). For the case with uniform sampling using point sources, the Gram matrix is proportional to the identity matrix. Homogeneous material losses modeled with a resistivity ρ_r are described by a material loss matrix R_ρ=ρ_rΨ meaning that current norm (<ref>) and material losses are related.   

The radiated field from the current density is determined by a convolution with a Green's function. 
The used scalar Green's function in ^2 and ^3 are

    G_2(r-r') = /4H_0(k|r-r'|)
          and 
        G_3 = ^- k|r-r'|/4π|r-r'|,

respectively, and the Green's dyadic G=(+k^-2∇∇)G_3.





§ MUTUAL SHADOW LENGTH

The mutual shadow length from a set of lines described by their end points _mn, with m∈{T,R} is determined for an illumination in the k-direction, using a plane wave E_z()=^- kk· polarized in the z-direction, see Fig. <ref>. The shadow is projected on a line perpendicular to k=xcos(ϕ)+ysin(ϕ), which using polar coordinates with azimuth angle ϕ results in

    p = -xsin(ϕ)+ycos(ϕ).

Projecting the node points on the p-line produces a set of numbers p_mn=p·r_mn and the lines a set of intervals I_m=[min_n p_mn,max_n p_mn]. The mutual shadow is formed by I_1∩ I_2. The intersection is empty if max_n p_mn≤min_n p_qn for some m≠ q, 
    max_n{p_1n}≤min_n{p_2n}  or max_n{p_2n}≤min_n{p_1n}.

The length of the interval is finally

    || = max_n{p_1n} - min_n{p_2n}

or

    || = max_n{p_2n} - min_n{p_1n}.

The total mutual shadow length is determined by integration (<ref>).




§ NDOF AND SHADOW AREA FOR TWO SPHERES

Shadow area and mutual shadow area are illustrated for two spheres having radii a_n, n=1,2 and separated a distance h, see Fig. <ref>. The spheres are without loss of generality placed on the z-axis producing a BoR object with an incident illumination parametrized by the polar angle θ. 


The shadow of each sphere is a circular disc with the same radius as the sphere. The two shadows can overlap, partly overlap, or being separated depending on the illumination angle θ through the distance d=hsinθ. For 0≤θ≤θ_1 with

    sinθ_1=|a_1-a_2|/h

the two shadows overlap, as illustrated in the bottom part of Fig. <ref>. For θ_1<θ<θ_2 with

    sinθ_2=(a_1+a_2)/h

the two shadows partly overlap. And for θ_2≤θ≤π/2 the two shadows are separated. This behavior is repeated symmetrically for π/2≤θ≤π. 

The total shadow area is determined by integrating over all incident angles, which reduces to a line integral over [0,π/2] for this specific case.  For the mutual shadow area, it is sufficient to integrate over [0,θ_2]. The mutual area for θ<θ_1 is π a_1^2 giving 

    A = 2π^2min{a_n^2}(1-cosθ_1)
        +2π∫_θ_1^θ_2 A(θ)sinθθ.

The partly overlapping shadow area is divided into two parts  A=A_1+A_2 determined as

    A_n = a_n^2((d_n) - d_n√(1-d_n^2))

with

    d_n = (d^2+2a_n^2-a_1^2-a_2^2)/(2da_n)

The integral simplify for two equal spheres a_1=a_2 and can be expressed in elliptic integrals, however it is also trivially solved by numerical quadrature, see Fig. <ref>.  
The mutual shadow area is normalized by the paraxial approximation

    A_p = A_1 A_2/h^2 = π^2 a_1^2 a_2^2/h^2,

which approximated the mutual shadow area very well for distances h>2(a_1+a_2), see Fig. <ref>.








10

url@samestyle








Pizzo+etal2022
A. Pizzo, A. de Jesus Torres, L. Sanguinetti, and T. L. Marzetta, “Nyquist
  sampling and degrees of freedom of electromagnetic fields,” IEEE
  Trans. Signal Process., vol. 70, pp. 3935–3947, 2022.

Hu+etal2018
S. Hu, F. Rusek, and O. Edfors, “Beyond massive MIMO: The potential of data
  transmission with large intelligent surfaces,” IEEE Trans. Signal
  Process., vol. 66, no. 10, pp. 2746–2758, 2018.

Migliore2006a
M. D. Migliore, “On the role of the number of degrees of freedom of the field
  in MIMO channels,” IEEE Trans. Antennas Propag., vol. 54, no. 2,
  pp. 620–628, Feb 2006.

Migliore2008
M. Migliore, “On electromagnetics and information theory,” IEEE Trans.
  Antennas Propag., vol. 56, no. 10, pp. 3188–3200, Oct. 2008.

Franceschetti2017
M. Franceschetti, Wave theory of information.1em plus 0.5em minus
  0.4emCambridge University Press, 2017.

Bjornson+etal2024
E. Björnson, C.-B. Chae, R. W. Heath Jr, T. L. Marzetta, A. Mezghani,
  L. Sanguinetti, F. Rusek, M. R. Castellanos, D. Jun, and Ö. T. Demir,
  “Towards 6G MIMO: Massive spatial multiplexing, dense arrays, and
  interplay between electromagnetics and processing,” arXiv preprint
  arXiv:2401.02844, 2024.

Poon+etal2005
A. S. Poon, R. W. Brodersen, and D. N. Tse, “Degrees of freedom in
  multiple-antenna channels: A signal space approach,” IEEE Trans.
  Information Theory, vol. 51, no. 2, pp. 523–536, 2005.

Franceschetti+etal2009
M. Franceschetti, M. D. Migliore, and P. Minero, “The capacity of wireless
  networks: Information-theoretic and physical limits,” vol. 55, no. 8, pp.
  3413–3424, Aug. 2009.

DiFrancia1955
G. T. Di Francia, “Resolving power and information,” JOSA, vol. 45,
  no. 7, pp. 497–501, 1955.

DiFrancia1956
——, “Directivity, super-gain and information,” IRE Trans. Antennas
  and Propag., vol. 4, no. 3, pp. 473–478, 1956.

Bucci+Franceschetti1989
O. M. Bucci and G. Franceschetti, “On the degrees of freedom of scattered
  fields,” IEEE Trans. Antennas Propag., vol. 37, no. 7, pp. 918–926,
  1989.

Bucci+Isernia1997
O. Bucci and T. Isernia, “Electromagnetic inverse scattering: Retrievable
  information and measurement strategies,” Radio science, vol. 32,
  no. 6, pp. 2123–2137, 1997.

Kildal+etal2017
P.-S. Kildal, E. Martini, and S. Maci, “Degrees of freedom and maximum
  directivity of antennas: A bound on maximum directivity of nonsuperreactive
  antennas,” IEEE Antennas Propag. Mag., vol. 59, no. 4, pp. 16–25,
  2017.

Hill1994
D. A. Hill, “Electronic mode stirring for reverberation chambers,” IEEE
  Trans. Electromagn. Compat., vol. 36, no. 4, pp. 294–299, 1994.

Janaswamy2011
R. Janaswamy, “On the EM degrees of freedom in scattering environments,”
  IEEE Trans. Antennas Propag., vol. 59, no. 10, pp. 3872–3881, 2011.

Weyl1911
H. Weyl, “Über die asymptotische verteilung der eigenwerte,”
  Nachrichten von der Gesellschaft der Wissenschaften zu Göttingen,
  Mathematisch-Physikalische Klasse, vol. 1911, pp. 110–117, 1911.

Arendt+etal2009
W. Arendt, R. Nittka, W. Peter, and F. Steiner, “Weyl's law: Spectral
  properties of the Laplacian in mathematics and physics,”
  Mathematical analysis of evolution, information, and complexity, pp.
  1–71, 2009.

Gustafsson2024a
M. Gustafsson, “Degrees of freedom for radiating systems,” arXiv
  preprint arXiv:2404.08976, 2024.

Piestun+Miller2000
R. Piestun and D. A. Miller, “Electromagnetic degrees of freedom of an optical
  system,” JOSA A, vol. 17, no. 5, pp. 892–902, 2000.

Miller2019
D. A. Miller, “Waves, modes, communications, and optics: a tutorial,”
  Advances in Optics and Photonics, vol. 11, no. 3, pp. 679–825, 2019.

Jensen+Wallace2008
M. A. Jensen and J. W. Wallace, “Capacity of the continuous-space
  electromagnetic channel,” IEEE Trans. Antennas Propag., vol. 56,
  no. 2, pp. 524–531, 2008.

Nordebo+etal2006
S. Nordebo, M. Gustafsson, and G. Kristensson, “On the capacity of the free
  space antenna channel,” in IEEE Antennas and Propagation Society
  International Symposium 2006,.1em plus 0.5em minus 0.4emIEEE
  Press, 2006, pp. 3105–3108.

Ji+etal2023
R. Ji, S. Chen, C. Huang, J. Yang, E. Wei, Z. Zhang, C. Yuen, and M. Debbah,
  “Extra DoF of near-field holographic MIMO communications leveraging
  evanescent waves,” IEEE Wirel. Commun. Le., vol. 12, no. 4, pp.
  580–584, 2023.

Yuan+etal2024
S. S. Yuan, J. Wu, H. Xu, T. Wang, D. Li, X. Chen, C. Huang, S. Sun, S. Zheng,
  X. Zhang et al., “Breaking the degrees-of-freedom limit of
  holographic MIMO communications: A 3-D antenna array topology,”
  IEEE Trans. Veh. Technol., 2024.

Yuan+etal2022
S. S. Yuan, Z. He, X. Chen, C. Huang, and E. Wei, “Electromagnetic effective
  degree of freedom of an MIMO system in free space,” IEEE Antennas
  Wirel. Propag. Lett., vol. 21, no. 3, pp. 446–450, 2022.

Migliore2019
M. D. Migliore, “Horse (electromagnetics) is more important than horseman
  (information) for wireless transmission,” IEEE Trans. Antennas
  Propag., vol. 67, no. 4, pp. 2046–2055, 2019.

Ruiz-Sicilia2023
J. C. Ruiz-Sicilia, M. Di Renzo, M. D. Migliore, M. Debbah, and H. V. Poor,
  “On the degrees of freedom and eigenfunctions of line-of-sight holographic
  MIMO communications,” arXiv preprint arXiv:2308.08009, 2023.

Halko+etal2011
N. Halko, P.-G. Martinsson, and J. A. Tropp, “Finding structure with
  randomness: Probabilistic algorithms for constructing approximate matrix
  decompositions,” SIAM review, vol. 53, no. 2, pp. 217–288, 2011.

Paulraj+etal2003
A. Paulraj, R. Nabar, and D. Gore, Introduction to Space-Time Wireless
  Communications.1em plus 0.5em minus 0.4emCambridge: Cambridge
  University Press, 2003.

Molisch2011
A. F. Molisch, Wireless Communications, 2nd ed.1em plus 0.5em
  minus 0.4emNew York, NY: John Wiley & Sons, 2011.

Harrington1968
R. F. Harrington, Field Computation by Moment Methods.1em plus
  0.5em minus 0.4emNew York, NY: Macmillan, 1968.

Shiu+etal2000
D.-S. Shiu, G. J. Foschini, M. J. Gans, and J. M. Kahn, “Fading correlation
  and its effect on the capacity of multielement antenna systems,” IEEE
  Trans. on Communication, vol. 48, no. 3, pp. 502–513, Mar. 2000.

Horn+Johnson1991
R. A. Horn and C. R. Johnson, Topics in Matrix Analysis.1em plus
  0.5em minus 0.4emCambridge University Press, 1991.

Stratton1941
J. A. Stratton, Electromagnetic Theory.1em plus 0.5em minus
  0.4emNew York, NY: McGraw-Hill, 1941.

Kristensson2016
G. Kristensson, Scattering of Electromagnetic Waves by Obstacles.1em plus 0.5em minus 0.4emEdison, NJ: SciTech Publishing, an imprint
  of the IET, 2016.

Gustafsson+Nordebo2007a
M. Gustafsson and S. Nordebo, “On the spectral efficiency of a sphere,”
  Prog. Electromagn. Res., vol. 67, pp. 275–296, 2007.

Gustafsson+Lundgren2024
M. Gustafsson and J. Lundgren, “Degrees of freedom and characteristic modes,”
  IEEE Antennas Propag. Mag., pp. 2–12, 2024, (in press).

Harrington1958
R. Harrington, “On the gain and beamwidth of directional antennas,”
  IEEE Trans. Antennas Propag., vol. 6, no. 3, pp. 219–225, 1958.

Stutzman+Thiele1998
W. L. Stutzman and G. A. Thiele, Antenna Theory and Design,
  2nd ed.1em plus 0.5em minus 0.4emNew York, NY: John Wiley &
  Sons, 1998.

Vouk1948
V. Vouk, “Projected area of convex bodies,” Nature, vol. 162, no.
  4113, pp. 330–331, 1948.

Capek+etal2023a
M. Capek, J. Lundgren, M. Gustafsson, K. Schab, and L. Jelinek,
  “Characteristic mode decomposition using the scattering dyadic in arbitrary
  full-wave solvers,” IEEE Trans. Antennas Propag., vol. 71, no. 1, pp.
  830–839, 2023.

Ehrenborg+Gustafsson2020
C. Ehrenborg and M. Gustafsson, “Physical bounds and radiation modes for
  MIMO antennas,” IEEE Trans. Antennas Propag., vol. 68, no. 6, pp.
  4302–4311, 2020.

Gustafsson+etal2020
M. Gustafsson, K. Schab, L. Jelinek, and M. Capek, “Upper bounds on absorption
  and scattering,” New Journal of Physics, vol. 22, no. 073013, 2020.

Ehrenborg+Gustafsson2018
C. Ehrenborg and M. Gustafsson, “Fundamental bounds on MIMO antennas,”
  IEEE Antennas Wireless Propag. Lett., vol. 17, no. 1, pp. 21–24,
  January 2018.

Ehrenborg+etal2021
C. Ehrenborg, M. Gustafsson, and M. Capek, “Capacity bounds and degrees of
  freedom for MIMO antennas constrained by Q-factor,” IEEE Trans.
  Antennas Propag., vol. 69, no. 9, pp. 5388–5400, 2021.

Schab2016
K. R. Schab, “Modal analysis of radiation and energy storage mechanisms on
  conducting scatterers,” Ph.D. dissertation, University of Illinois at
  Urbana-Champaign, 2016.

Gustafsson+Capek2019
M. Gustafsson and M. Capek, “Maximum gain, effective area, and directivity,”
  IEEE Trans. Antennas Propag., vol. 67, no. 8, pp. 5282–5293, 2019.

Agoston2005
M. K. Agoston, Computer graphics and geometric modeling.1em plus
  0.5em minus 0.4emSpringer, 2005.






 Paper Title: Uplink Wave-Domain Combiner for Stacked Intelligent Metasurfaces Accounting for Hardware Limitations.





Uplink Wave-Domain Combiner for Stacked Intelligent Metasurfaces Accounting for Hardware Limitations
    Maryam Rezvani1, Raviraj Adve1, Akram bin Sediq2, Amr El-Keyi2
	1Dept. of Electrical and Computer Engineering,
		University of Toronto, 
		Toronto, Canada 
	
	2Ericsson Canada, 
		Ottawa, Canada, 

		emails: {mrezvani, rsadve}@ece.utoronto.ca, {akram.bin.sediq, amr.el-keyi}@ericsson.com

    August 5, 2024
====================================================================================================================================================================================================================================================================================================


§ ABSTRACT 
    Using refractive metasurfaces (RMTSs) as part of the antenna design offers a promising solution to the ever-increasing demand for improved energy efficiency of wireless communications. To overcome the limitations of using one RMTS layer, a recent proposal is to cascade multiple layers, a structure called stacked intelligent metasurfaces (SIMs) where the desired precoder and combiner is formed in the wave domain. However, while proposing the antenna structure, the analysis did not account for the attendant limitations imposed by the hardware used which has a significant impact on the performance of SIMs. In this paper, we study the achievable sum-rate of a SIM antenna in an uplink wireless communication scenario accounting for hardware limitations. We begin by proposing a system model that captures both the effect of noise and hardware limitations in these systems. We then formulate the achievable sum-rate problem; since optimizing the rate is non-convex, we propose two approaches: a gradient ascent algorithm and an interior point method to find a close-to-optimum combiner. To show the efficiency of using SIMs at a basestation, we compare the achievable sum-rate with that of a digital phased array (DPA). We provide two comparisons: first, in Rayleigh fading and realistic 3GPP channels and then under a constraint of an equal number of radio frequency (RF) chains and equal physical aperture size constraint. Our results show that SIM antennas can surpass DPA performance under an equal number of RF chains but has inferior performance under equal aperture size.



	Holographic multiple-input multiple-output (MIMO) systems, Stacked Intelligent Metasurfaces (SIM), Uplink wireless communications, Wave-domain combiner.





§ INTRODUCTION

Future wireless communication networks must meet the rising demand for higher data rates and increased throughput, while also prioritizing sustainability and energy efficiency <cit.>. To achieve higher data rates, extra-large multiple-input multiple-output (MIMO) antennas, also called extra-large digital phased arrays (DPAs), have been introduced as an enhancement over massive MIMO systems. By increasing the number of antennas in a DPA, the signal-to-noise ratio (SNR) improves, reducing the power required by users. However, using larger DPAs lead to higher power consumption at base stations (BSs) due to the numerous power-hungry radio frequency (RF) chains. 

To tackle BS power consumption, the two main approaches are to introduce low complexity signal processing techniques such as <cit.> or sharing signal processing between digital and analog/wave domains. Hybrid MIMO systems push part of the signal processing to the analog domain using phase shifters <cit.> while, metasurface (MTS) antennas share signal processing between electromagnetic (EM) wave and digital domains <cit.>. Our focus here is on MTS antennas.

MTSs, comprising subwavelength unit-cells, enable manipulation of its properties via voltages applied to diodes in each unit-cell. For example, reflective intelligent surfaces utilize MTSs to manipulate wireless channel properties thereby enhancing channel gain. Similarly, deploying refractive MTSs (RMTSs) at BSs provide the ability to form any desired radiation pattern at the EM level with higher energy efficiency and lower latency <cit.>.



The effect of the unit-cell on the impinging wave is captured as a weighting factor. Using single-layer  unit-cells, the phase and amplitude of the weighting factor follow Lorentzian modulation, i.e., are coupled <cit.>. This coupling limits the radiation patterns that can be achieved by the MTS antenna. Achieving phase-only modulation, spanning [0, 2π) interval while keeping the amplitude constant, requires a “stacked unit-cell" design, where each unit-cell comprises at least three reactive impedance sheets <cit.> where the impedance of each layer is controlled by changing the applied voltage to its varactor diode <cit.>.

Since full control over the radiation pattern requires independent control over amplitude and phase <cit.> of each weighting factor, using only one layer of phase-only RMTS at BS is insufficient to form any desired radiation pattern (e.g., having to null multiuser interference). To tackle this problem, the work in <cit.> introduced a multi-layer structure called stacked intelligent metasurfaces (SIM) where several phase-only modulated RMTSs are stacked in front of a small DPA and wrapped by absorber walls <cit.>. This design is based on diffractive deep neural networks (D2NNs) <cit.>. In SIM antenna, the number of elements (and RF chains) in the DPA is equal to the number of users served by the BS. Also, using a wave-domain (WD) combiner/precoder implemented by the SIM renders the digital combiner redundant, thereby simplifying the required digital signal processing <cit.>.








Since its introduction, different aspects of SIM-assisted wireless communication have been studied. In <cit.>, a joint power allocation and WD precoder algorithm is introduced for downlink communications. Also, in <cit.> a joint precoder and combiner is designed for single-user scenarios where both receiver and transmitter are equipped with SIM antennas. In <cit.>, it is shown that using a SIM antenna, a 2D discrete Fourier transform can be implemented over the air and hence, provide an opportunity to estimate the angle of arrival at an EM level. Furthermore, in <cit.>, it is shown that a BS equipped with a SIM can form any desired beam pattern. 

In all these papers, the focus is on downlink wireless communication where the transmitterBS, is equipped with a SIM antenna. This is while the equally important problem of uplink communication where single-antenna users communicate with a SIM-assisted BS is untouched. The uplink problem is a unique one since SIM must effectively distinguish between the weak signal suffered from channel propagation and noise, and while focusing signal on its DPA, also mitigate noise and interference. 

Furthermore, the previous works often assume SIM structures to be losslessa transmission coefficient of one for each RMTS layer. However, signal power losses occur due to copper and dielectric losses, as well as the resonant nature of each unit-cell <cit.>, making signal detection amidst background noise even more challenging. Also, the current approach to model interlayer EM propagation is based on the Rayleigh-Sommerfeld relation <cit.> which assumes large dimensions of implemented RMTSs compared to the wavelength and the EM field is not evaluated close to the RMTS <cit.>. These assumptions do not hold in SIM structures, leading to an unfair comparison with other antenna designs when using the Rayleigh-Sommerfeld relation to model the interlayer propagation. All these issues motivate us to study the SIM-assisted BSs problem in the uplink while taking into account its realistic hardware limitations.



Contributions: To maximize the achievable sum-rate of an uplink wireless communication system where a BS is equipped with a SIM antenna, we propose a system and SIM model that considers noise and EM propagation effects using closed-form EM wave relations. Then, we formulate an optimization problem to maximize the sum-rate with respect to the weighting factor of each unit-cell in each RMTS layer. Since both the objective function (the achievable sum-rate) and the feasible domain (phase-only weighting factors) are non-convex, we employ a gradient-based ascent (GA) algorithm with adaptive learning rates to find a close-to-optimal WD combiner. Additionally, we explore the interior-point method (IPM), which utilizes the Hessian of the objective function to improve update directions beyond the gradient alone.

In sum, the contributions of this paper are:

	
  * We propose a system model for a BS equipped with a SIM antenna, in the uplink, which captures the effect of noise and hardware limitations in these systems, and accurately models the EM propagation between RMTS layers.
	
  * To design a WD combiner that maximizes achievable sum-rate in the uplink of the SIM antenna, we propose a GA algorithm with a variable step size. Also, we implement an IPM approach and investigate its performance.
	
  * Our simulation results show that both in Rayleigh fading and realistic 3GPP channels, our proposed GA algorithm and IPM can improve the performance of SIM antennas with respect to DPA with an equal number of RF chains in single-user and multi-user scenarios. But under equal aperture size, DPA outperforms SIM antenna.


Notation: Lowercase letters denote scalars. We use lowercase and uppercase boldface letters to denote vectors and matrices, respectively. (·)^H denotes conjugate transpose. We represent the vector fields using an arrow, e.g., A⃗. Furthermore, by A_:,n and A_m,:, we refer to n-th column and m-th row of A, respectively.  Also,  is defined as √(-1) and (x) = sin(π x)/π x. 𝒞𝒩(μ, 𝐑) represents the complex Gaussian distribution with mean μ and covariance matrix 𝐑.







§ SYSTEM AND SIM MODEL


We consider a system where a BS equipped with a L-layer SIM, containing N unit-cells in each layer, serves K users, as depicted in Fig. <ref>. The SIM is placed in front of a M-element DPA where each element is connected to an RF chain. The received signal at the BS is described as

    x = g(G^SIM(√(P_T)Hs+z_ant)+z_RF)

where x∈ℂ^M × 1 is the received signal at BS, P_T is the transmitted signal power of each user assuming equal power transmission, H∈ℂ^N × K is the channel matrix, z_ant∈ℂ^N × 1 is external noise captured by the antenna from the environment; its entries are correlated with same the correlation matrix as the antenna <cit.>. Furthermore, g and z_RF∈ℂ^M× 1 are gain and noise terms, respectively, introduced by the RF chain where z_RF∼𝒞𝒩(0,σ^2_RF). Finally, G^SIM∈ℂ^M × N is the equivalent WD combiner introduced by the SIM structure. In SIM antennas, there is no need for a digital combiner <cit.>, hence, we set M=K.

Similar to <cit.>, we model the effect of the SIM, i.e., G^SIM, as a product of propagation, P^l, and RMTS weighting matrices, W^l, corresponding to the l-th RMTS layer. In the l-th layer W^l is a diagonal matrix
    W^l = diag(e^θ^l_1, e^θ^l_2, …, e^θ^l_N)
 
where θ_i^l is the phase shift introduced by i-th unit cell in l-th layer. Likewise, P^l captures the effect of EM propagation between l-th and (l+1)-th layer.

To model the propagation from DPA to the first RMTS, one common practice is to use Rayleigh-Sommerfeld diffraction theory <cit.>. The theory considers EM fields as scalars and is only applicable when the diffracting aperture is large compared to the wavelength and the diffracted field is not observed close to the aperture <cit.>. However, in SIM antennas, neither condition is satisfied; importantly, the DPA and RMTSs are in a few wavelengths or even subwavelength distances from each other. For example, denoting λ_0 as wavelength, in <cit.>, the interlayer distance is 5λ_0/L for L up to 7. On the other hand, in <cit.>, Rayleigh-Sommerfeld theory is used to model interlayer EM propagation for the D2NN SIM structure since the interlayer distance is at least 5.5λ_0, respectively. We emphasize that accurately modeling the propagation matrix, P, is essential to capture variations in EM power due to propagation within the SIM structure and compare SIM antennas with other antenna types fairly.

To model the propagation matrix from the first RMTS layer to DPA, we use the reciprocity theorem and model EM propagation in the reverse direction <cit.>. In general, the propagation matrix from the DPA to the first RMTS layer depends on the type of antenna elements and their corresponding Green's function <cit.>. Here, to provide a valid and illustrative instantiation, we assume that the elements of the DPA are Hertzian dipoles placed in front of a perfect magnetic conductor[This is a common practice to make the dipole radiation more directive and use the added radiation gain due to the image theory <cit.>.]. We model variations in the amplitude of impinging EM wave, by capturing relative changes in EM power. For a Hertzian dipole, the Poynting vector, in both near-field and far-field, is described as <cit.>

    W⃗   = η_0 /8I_0 l_d/λ_0 ^2 sin^2θ/r^2[1-1/(k_0 r)^3]r̂
       +η_0  k_0 |I_0 l_d|^2cosθsinθ/16 π^2 r^3[1+1/(k_0 r)^2]θ̂

where k_0 = 2 π/λ_0 is the free space wavenumber, I_0 is the excitation current, l_d is the dipole length, η_0 is the free space wave impedance, and r and θ are the distance and azimuth angle between source and evaluation point, respectively. In this paper, we are dealing with uplink communication where BS is in the far field of the users; therefore, only the propagating term of the Poynting vectorits real part, is of importance.

Since the relative change in EM power due to propagation is important, we only take into account the term which depends on observation locationsin^2θ/r^2. Then, by normalizing this term, we make sure that the total power on a half-sphere at distance r remains equal to the power radiated from the source[There are no propagation losses from the dipole antenna to the first RMTS layer since the propagation environment is equivalent to free space.]. Further, by mapping the power coefficient to the complex amplitude of E⃗, we reach the following relation for the propagation of the electric field from the m-th element of the DPA to the n-th unit-cell on the first RMTS layer 

    |P^1_m,n| = (3√(2)/8πsin^2θ_m,n/r^2_m,n)^1/2.


To capture the change in phase of the electric field with respect to the source, we calculate the change in phase of the propagating component of the electric field radiated by a Hertzian dipole <cit.>
    ∡P^1_m,n = π/2- k_0 r_m,n+∡(1+1/ k_0 r_m,n-1/(k_0 r_m,n)^2).

Finally, the (m,n)-th element of the propagation matrix P^1 is given by |P_m,n^1|e^∡P_m,n^1. 

The propagation matrix from the n-th unit-cell on the l-th layer to the n'-th unit cell on the (l+1)-th layer can be calculated using the same methodology. This is because each unit-cell can be modeled as a dipole <cit.> and RMTSs are designed to have zero backward radiation; hence, the sum of the transmission power coefficients on the half sphere in front of each unit-cell should be unity.

By concatenating the propagation matrices and weighting factor of each RMTS, the overall WD combiner of L-layer SIM is given by

W^SIM = P^1 (∏_l=1^L-1W^lP^l+1)W^L.


Insertion Loss: Although RMTSs on each layer are defined as passive and lossless, there are losses due to the propagation of the EM field in the copper (unit-cells) and dielectric. Also, the resonant nature of the unit-cells adds to the losses in each RMTS layer <cit.>. To account for these losses, we introduce the transmission efficiency coefficient, T^SIM, to the SIM model. Therefore, the WD combiner for L-layer SIM is described as:

G^SIM = √((T^SIM)^L)W^SIM.






§ PROBLEM FORMULATION AND COMBINER DESIGN

The achievable sum-rate, R, in the uplink SIM-assisted wireless communication systems can be written as

R = ∑_k=1^K log_2(1+γ_k)


where γ_k is the signal-to-interference-plus-noise ratio for user k defined as

    P_TG^SIM_k,:h_:,k^2/∑_≠ k^K P_TG^SIM_k,:h_:,j^2 +σ_ant^2(G^SIM_k,:Σ_rx(G^SIM_k,:)^H)+σ_RF^2.

We note the two noise terms in the denominator; while the second is the usual noise term in the RF chains, the first term is due to the antenna noise and is colored by the SIM combiner.
To maximize R, we are interested in solving the following optimization problem:

	
	
    max_θ^1,…, θ^L R
    s.t.    W^SIM = P^1 (∏_l=1^L-1W^lP^l)W^L
       W^l = diag(e^θ^l),


where θ^l = [θ^l_1, θ^l_2, …, θ^l_N]^T is the vector of phase shifts introduced by the unit-cells.

Solving (<ref>) is not trivial since L>1 and, since the noise is colored, a simple matched-filtering solution is not available even for the single-user case. Hence, similar to <cit.>, we propose a GA algorithm comprising the following steps: we begin by randomly initializing θ^l  ∀ l∈{1,2,…, L} and calculating gradient of (<ref>) with respect to θ_l^n using:

	
	
    Rθ^n_l = ∑_k=1^KRe{1/log(2)1/1+γ_kγ_kϕ^n_lϕ^n_lθ^n_l}
       γ_kϕ^n_l = (S_pwr,kϕ^n_lZ_pwr,k - Z_pwr,kϕ^n_lS_pwr,k)/Z_pwr,k^2
       S_pwr,kϕ^n_l =
       [P^1_k,:∏_ll=1^l-1(W^llP^ll)]_:,n[∏_ll = l+1^L(P^llW^ll)H_:,k]_n,:H^H_:,kW^H_k,:
       Z_pwr,kϕ^n_l = 
       ∑_j≠ k^K  ([P^1_k,:∏_ll=1^l-1(W^llP^ll)]_:,n [∏_ll = l+1^L(P^llW^ll)H_:,j]_n,:H^H_:,jW^H_k,:)
       +[P^1_k,:∏_ll=1^l-1(W^llP^ll)]_:,n [∏_ll = l+1^L(P^llW^ll)U^H]_n,:UW^H_k,:


where U is the Cholesky decomposition of the antenna correlation matrix Σ_rx, γ_k=S_pwr,k/Z_pwr,k, and ϕ^n_lθ^n_l =  e^θ_l^n.

Then by normalizing the gradient of each layer, we prevent the vanishing or exploding gradient effectRθ^n_l←1/ρ_lRθ^n_l where ρ_l is the maximum gradient length in the l-th layer. Furthermore, the step size μ is calculated using the backtracking line search algorithm <cit.> with α as the initial guess for the step size in each iteration. We look for a proper step length in both directions of the gradient to minimize the chance of ending up in a local optimum. Finally, all phase shifts are updated simultaneously using: θ_l^n ←θ_l^n+μRθ^n_l. This process is repeated until some criteria is met.

As both (<ref>) and (<ref>) are non-convex functions, the GA algorithm can converge to a local optimum; at attempt to mitigate the issue we use IPM. The direction of update in each step in IPM is calculated by considering both Hessian and gradient of (<ref>) <cit.> and hence, a better update direction is chosen. We use Matlab's fmincon <cit.> to find a sub-optimal WD combiner using IPM with approximate Hessian and gradient. We emphasize that using IPM does not eliminate the possibility of converging to a local optimum.





§ NUMERICAL RESULTS

In this section, we document the effectiveness of the proposed GA algorithm and IPM in calculating WD combiner for SIM antenna in single-user and multi-user uplink scenarios where the BS is in the far-field of user(s). Also, as upper and lower performance limits, we report the achievable sum-rate of DPA with an equal physical aperture area as SIM antennathe averaged users' signal power which reaches both antennas are the same, and with equal number of RF chainsequal power consumption[We ignore the power consumption of varactor diodes and driving circuitry of each RMTS unit-cell as being comparatively negligible.].

We consider an outdoor wireless communication problem, where K users communicate with a BS equipped with an antenna of size 4λ_0 × 4λ_0 at 3 GHz carrier frequency with 20 MHz bandwidth. As BS is in the far-field of users, the transmitted power, P_T, is defined as the amplitude of the Poynting vector of the users' transmitted signalP_T = E⃗_user^2/2η_0 [W/m^2]. The SIM thickness, t_SIM is 5λ_0the interlayer distance is 5λ_0/L. Furthermore, the RF chains are assumed to be at room temperatureT_BS = 290  [K]. Finally, g and noise figure of RF chain are set to be 12.5 [dB] and 18.8 [dB][Calculated by considering a low pass filter <cit.> and first three elements in the RF chainlow noise amplifier <cit.>, mixer <cit.>, and in phase/quadrature demodulator <cit.>.], respectively. We repeat the simulation for 10 instances of user placements and 100 channel realizations per instance.  Also, we allow a maximum of 100 and 500 iterations for the IPM and GA algorithms, respectively.

Received signal power: The power captured by each antenna/unit-cell is equal to P_T A_eff where A_eff is the effective area of the antenna <cit.>. In the SIM, each unit-cell is approximated by a continuous current sheet, hence, A_eff is equal to the physical area of each unit-cell A_p. Considering unit-cells of size λ_0/4×λ_0/4 at 3 GHz, A_eff,UC = 0.0025. For massive MIMO, assuming patch antenna as the elements of DPA, A_eff is calculated by expressions in <cit.> that results in A_eff,PA at 3 GHz to be 0.0026.

Antenna noise power: The noise power captured by the antenna is defined by 

P_z_ant = KT_EBA_eff/λ_0^2ΔΩ <cit.> 

where K is the Boltzmann constant, T_E is the environment temperature well approximated by 290  K <cit.>, and ΔΩ is the solid angle of noise source at the antenna equal to 2 π.

Channel model: In rich scattering environments, we use the Kronecker Rayleigh fading model where H = Σ_rxH̃Σ_tx and Σ_tx and Σ_rx are the users' and antennas' correlation matrices, respectively. The entries of H̃ are independent identically distributed (i.i.d.) random variables drawn from a standard normal distributionH̃∼𝒞𝒩(0,I_NK). We assume users are independent of each other hence, Σ_tx = diag(β_1, …, β_K) where β_k is the path loss of the k-th user, here computed using the Winner II urban microcell non-line of sight scenario <cit.>. To calculate Σ_rx, we assume scatterers are distributed in 3D space and hence, ()_(n,n') = (2d_n,n'/λ_0) <cit.> where d_n,n' is the distance between the n-th and n'-th unit-cells. 

In both Rayleigh and 3GPP channels <cit.>, since the ground occupies whole view of the antenna, the external noise correlation matrix is equivalent to the antenna correlation matrix in a rich scattering environmentz_ant∼(0,P_z_ant ). Also, whenever the antenna's correlation matrix is needed, Σ_rx is calculated as described above.

Finally, we assume the DPA is formed by an array of patch antennas with an efficiency of 0.9 <cit.> and the efficiency of each RMTS in SIM, T^SIM, is set to 0.7 <cit.>. For the sake of comparison, we report a simple matched-filtering combiner, where each θ_i^l compensates for the phase of the equivalent channel in front of l-th layer and [P^l]_n,n.



Algorithm Efficiency: We begin by studying the efficiency of the proposed GA algorithm. Fig. <ref> plots the results averaged over 1000 and for 10 randomly selected  Rayleigh fading channel realizations. As shown, in all cases the major improvement is during the first 200 iterations and all cases converge to at least a local optimum. Also, Fig. <ref> plots the achievable sum-rate of SIM using the GA and IPM algorithms for different numbers of layers. As shown, by increasing the number of layers, the achievable sum-rate of both GA and IPM algorithms experience an initial improvement and eventually saturates or experiences a drop. As the number of layers increases, the number of variables increases as well which results in a higher possibility for GA algorithm to converge to a poor local optimum. IPM, using the Hessian of the objective function, can find a better update direction toward the global optimum and escape smaller local optima. Also, when L increases, the interlayer distance decreases, leading to poor interconnectivity and reduced achievable sum-rate.

Single-User Performance For a single-user scenario, depicted in Fig. <ref> is the achievable sum-rate of a 5-layer SIM, using both IPM and GA algorithms, and DPA with a physical aperture size equal to 4λ_0 × 4 λ_0. Also, for comparison, the achievable sum-rate of a DPA with 
M=K=1 RF chains is reported, as is the performance of the SIM with a simple MF combiner. We set α to 1.8 in the GA algorithm. As shown, using IPM and GA algorithm, we gain a significant improvement in performance compared to using a simple MF combiner. Interestingly, using GA provides a higher sum-rate than using IPM in both Rayleigh fading and realistic 3GPP channels. In this case it appears that the flexibility in choosing the learning rate in each iteration helped GA algorithm in single-user cases to skip local optimums and find a closer-to-optimum combiner than IPM. Also, the achievable sum-rate of SIM antenna is lower than that of the DPA using maximum ratio combiner due to higher losses in the system (in comparison with DPA under equal aperture size constraint). However, when the number of RF chains is made equal, putting a SIM in front of a single antenna improves the uplink sum-rate of the user. Here, the SIM acts as a lens that focuses the received signal power by the user on its own receiver antenna.




Multiuser Scenario: Finally, we consider a multi-user scenario with K=2, depicted in Fig. <ref>. In this case, DPA uses zero-forcing to combine the received signal, α is set to 2.2 in the GA algorithm, and we choose a random user to form the MF combiner. As show, in both Rayleigh fading and realistic 3GPP channels, the WD combiner calculated by IPM achieves a higher sum-rate in comparison with GA and MF combiners. The more complex optimization problem due to inter-user interference, increases the possibility of GA algorithm ending up early in a local optimum. It is worth noting that due to the non-convexity of objective function and constraints of (<ref>), performance of GA algorithm is highly dependent on the initial guess of the backtracking algorithm, α. A more sophisticated algorithm for choosing step length may result in a better performance of GA algorithm. 







§ CONCLUSION

In this paper, we study the possibility of using SIM antennas at the BS for uplink wireless communications. We begin by proposing a system model that captures the effect of different noise sources and hardware limitations in SIM antennas. Also, we model the needed interlayer propagation matrices using EM theory to accurately capture variations in signal power. Then we propose a GA algorithm and also apply IPM to design a WD combiner to maximize the achievable sum-rate in uplink communications. Our simulation results show that in both Rayleigh fading and realistic 3GPP channels using SIM antenna improves achievable sum-rate in uplink in comparison with DPA under an equal number of RF chains. On the other hand, DPA achieves higher sum-rates than SIM when both antennas have equal aperture area.





§ ACKNOWLEDGMENTS


This work was supported by the Ericsson Canada. 
Maryam Rezvani gratefully acknowledges insightful discussions with Parham Abbasloo.




IEEEtran





 Paper Title: Scheduling Quantum Annealing for Active User Detection in a NOMA Network.








Scheduling Quantum Annealing for Active User Detection in a NOMA Network
    Romain Piron
Université de Lyon, INSA Lyon, INRIA 

CITI EA 3720

69621 Villeurbanne, France 

romain.piron@insa-lyon.fr
Claire Goursaud
Université de Lyon, INSA Lyon, INRIA 

CITI EA 3720

69621 Villeurbanne, France 

claire.goursaud@insa-lyon.fr


    August 5, 2024
=============================================================================================================================================================================================================================================================



§ ABSTRACT 
Active user detection in a non-orthogonal multiple access (NOMA) network is a major challenge for 5G/6G applications. However, classical algorithms that can perform this task suffer either from complexity or reduced performances. This work aims at proposing a quantum annealing approach to overcome this trade-off. Firstly, we show that the maximum a posteriori decoder of the activity pattern of the network can be seen as the ground state of an Ising Hamiltonian. For N users in a network with perfect channels, we propose a universal control function to schedule the annealing process. Our approach avoids to continuously compute the optimal control function but still ensures high success probability while demanding a lower annealing time than a linear control function. This advantage holds even in the presence of imperfections in the network.



Quantum annealing, local adiabaticity, control function, non-orthogonal multiple access, active user detection




§ INTRODUCTION


One of the major challenges of 5G wireless networks is to achieve ultra-reliable low latency communications (uRLLC) requirements to serve applications as the Industrial Internet of Things <cit.>. However, the allocation of a dedicated time slot or frequency prior to transmission for each user is no longer adapted for typical uRLLC networks because of their growing size <cit.>. Instead, each time that the users wish to transmit a signal, they must complete a random access procedure <cit.>. 

This procedure begins by an activity notification sent by the active users in the uplink channel. To do so, the code division multiple access (CDMA) approach can be used <cit.>. An identification sequence is attributed to each node of the network and all active users simply transmit their code. This approach permits sharing a common time-frequency resource to all the users without interference.

It recently appeared that leveraging non orthogonal multiple access (NOMA) in CDMA systems is promising <cit.>. In practice, it allows to  attribute non orthogonal identification sequences to the nodes. Thus, one can reduce the size of these sequences for a given size of the users' set.

Detecting the active users - or the activity pattern - accurately from the received signal is called active user detection (AUD) <cit.>. It poses a significant challenge in NOMA systems since the cross-correlations between the sequences do not vanish. Thus, the users cannot be detected separately. While methods like the zero-forcing technique <cit.> or traditional correlation receiver decoders <cit.> offer computational efficiency, their performance is limited. Conversely, the maximum a posteriori (MAP) Bayesian estimator <cit.> is the most reliable but computationally demanding, being NP-hard to compute.

In AUD scenarios in which the channel state is known, one can show that the computation of the MAP estimator is a quadratic unconstrained binary optimization (QUBO) <cit.> problem. The equivalence between QUBO problems and Ising Hamiltonians allows one to formulate the AUD problem as a ground state searching problem. However, an exhaustive search over the space of configurations to find the ground state has an exponential cost with the system size. 

Several methods have been investigated to avoid this exhaustive search.  Quantum annealing (QA) originally introduced by Kadowaki and Nishimori as a quantum analogue of thermal annealing in <cit.> is considered today as a promising quantum computing-based strategy for the ground state searching of several classes of Hamiltonians <cit.>. Like many authors today, we use the term quantum annealing to refer to QA-inspired adiabatic quantum computation <cit.>. Much work remains to be done regarding execution time, but QA already outperformed classical thermal annealing in reaching the ground state of some classes of Hamiltonians <cit.>. 

The control function that introduces transverse quantum fluctuations in QA usually needs to be adapted according to the parameters of the target Hamiltonian <cit.>. Unfortunately, in typical AUD models several variables are not deterministic but rather random variables <cit.>. This would require to compute the control function for each received signal. This is why, in this paper, we propose a strategy to build a generic control function to parameterize a QA process for AUD given the size N of the wireless network.

The rest of the paper is organized as follows. Sec. <ref> defines our model and build the Ising Hamiltonian associated with the MAP estimator for AUD. In Sec. <ref>, we briefly review the adiabatic theorem and show how to derive a control function for QA that ensures adiabaticity locally during the annealing process. Then, in Sec. <ref> we adopt a simplified scenario of our model to propose a universal control function to schedule a QA approach for AUD once the size of the network has been fixed. Finally, Sec. <ref> takes into account the channel state information in our AUD model. 



 §.§ Notations


The set of matrices over a field 𝕂 of size N × M is denoted 𝕂^N × M with the usual identification 𝕂^N × 1≡𝕂^N. We respectively denote a scalar, vector, matrix by x, x, X. 

The diagonal matrix whose diagonal elements correspond to the components of a vector x is denoted diag(x). We also introduce 1_N = [1, …, 1] ∈𝕂^N, finally the identity matrix of size M× M corresponds to I_M = diag(1_M).

The multivariate normal distribution with the mean vector μ and the covariance matrix Σ is denoted 𝒩(μ,Σ). The Dirac distribution picked at x is denoted Dirac(x). The uniform distribution over a set Ω is denoted 𝒰(Ω).




§ QA FORMULATION FOR AUD




 §.§ Context and problem defintion


We consider a network with N users. Each user #i is equipped with a single antenna and assigned with a bipolar identification sequence <cit.> - also called code - of length M denoted c_i. This code constitutes a signal that is transmitted by the i-th user. Its power is given by:

    P_signal,i = ‖c_i ‖^2.

It is often convenient to work with normalized signals such that P_signal,i = 1. Hence, the bipolar codes are chosen in the set:

    c_i ∈{-1/√(M), 1/√(M)}^M.

These vectors must also satisfy a quasi-linearly independence to ensure the decodability, which is discussed in Appendix <ref>.

Let us assume that a random subset 𝒩_A of these users wish to transmit their code to a base station (BS) to notify their activity on the wireless channel. This subset constitutes the activity pattern of the network encoded in the bit-string b^(0)∈{0,1}^N so that one can write:

    𝒩_A = {i ∈{1, …, N}   s.t    b_i^(0) = 1}.


Each propagation path between the i-th user and the BS might suffer from attenuation due to the Rayleigh fading coming from the isotropic scattering assumption <cit.>. It is modelled by N fading coefficients (w_i)_i=1,…,N that obey a different probability law depending on the quality of the channels:

    w∼Dirac(1_N)      for perfect channels
    𝒩(0,I_N)      for imperfect channels

where w = (w_i)_i=1,…,N. At the end, one measures the following signal at the receiver side:

    y = ∑_i=1^N w_i b_i^(0)c_i + n,

where n∼𝒩(0,ξ^2 I_M) is an additive white Gaussian noise that models the thermal noise at the BS.

The AUD problem consists in retrieving the activity pattern b^(0) of the network with the knowledge of the received signal y and the fading coefficients w. As mentioned in the introduction, the MAP estimator is the most reliable decoder for such task. It is defined by:

    b̂^MAP = b∈{0,1}Narg max ℙ_𝖻|𝗒,𝗐(b|y,w)

In the next part, we express the probability mass function ℙ_𝖻|𝗒,𝗐(b|y,w) in terms of the variables b_i.



 §.§ QUBO formulation for the MAP detector


Eq. <ref> implies that f_𝗒|𝖻,𝗐 simply corresponds to the probability density function of a Gaussian law. In addition, the Bayes' formula states that <cit.>:

    ℙ_𝖻|𝗒,𝗐(b|y,w) f_𝗒|𝗐(y|w) = f_𝗒|𝖻,𝗐(y|b,w)ℙ_𝖻(b).

From this point, a simple hypothesis consists in assuming a uniform distribution over the activity patterns. In this scenario, the prior distribution ℙ_b is constant which yields:

    ℙ_𝖻|𝗒,𝗐(b|y,w) ∝exp(- ‖y-C·diag(w) ·b‖^2/2 ξ^2),

where the sequences have been concatenated in the matrix C = [c_1 , …, c_N] ∈ℂ^M × N. Hence the MAP detector reads:

    b̂^MAP = b∈{0,1}^Narg min {‖y-C·diag(w) ·b‖^2}.

The computation of b̂^MAP aims at recovering b^(0) with (y,w) as inputs. Developing this objective function yields to the expression:

    ‖y-C·.   . diag(w) ·b‖^2 = ‖y‖^2 - 2 ∑_i=1^N (y· w_i c_i) b_i 
        + ∑_i,j=1^N (w_i c_i · w_j c_j )b_i b_j.

Once one throws away the irrelevant constant ‖y‖^2, our minimization problem corresponds to the general form

    b̂^MAP = b∈{0,1}^Narg min {∑_i,j Q_ij b_i b_j + ∑_i a_i b_i }.

Thus, the MAP detector can be written as a QUBO problem <cit.> expressed in terms of the binary variables b_i. 



 §.§ Building the Ising Hamiltonian


As underlined by the authors of <cit.>, a QUBO problem can be mapped to an Ising Hamiltonian by the change of variable σ_i = 1 - 2 b_i. We defer the details to appendix <ref> and simply give here the final expression of the problem Hamiltonian:

    H_P = - ∑_i h_i σ_i - ∑_i < j J_ijσ_i σ_j,

expressed with classical Ising spins σ_i ∈{+1 ≡ ↑, -1 ≡ ↓} and the parameters:

    h_i    = - w_i (c_i ·ỹ) 
    
        J_ij   = -1/2 (w_i w_j)  (c_i ·c_j)
        ,

where the signal ỹ is defined by:

    ỹ = y - 1/2∑_j=1^N w_j c_j.

In terms of Ising variables, an inactive (resp. active) user corresponds to σ_i =  ↑ (resp. σ_i =  ↓). The MAP estimator is the ground state of this Hamiltonian. It is the spin configuration σ^(0)∈{↑,↓}^N associated to b^(0) that minimizes the energy of the system. The decodability constraint previously mentioned about the codes c_i ensures that the ground state is unique in the idealized situation w = 1_N and ξ = 0. 

Let us quickly mention a simple sanity check that confirms that this Hamiltonian is well suited to our problem. We adopt the scenario w = 1_N, ξ = 0 and relax the NOMA constraint by considering orthogonal identifications sequences, ie. c_i ·c_j = 0 for i  j. Then, the activity pattern is trivially recovered by:

    b^(0)_i = |y·c_i|.

On the other hand, Eq. <ref> in this ideal situation reduces to:

    h_i    = ||c_i||^2 σ_i^(0)
    
        J_ij   = 0
        ,

where σ^(0) is the spin configuration associated to the initial activity pattern b^(0). The ground state of H_P is trivially σ^(0), as one might reasonably expect. 

The ground state of an Ising Hamiltonian can be found with perfect accuracy through an exhaustive search in the space of configurations {↑, ↓}^N. However, such task would require 𝒪(2^N) iterations which is why it is known to be NP-hard. In order to perform ground state searching with QA, we promote the Ising spins σ_i to quantum spins σ̂_̂î^̂ẑ [Recall that σ̂_̂î^̂α̂ denotes the Pauli matrices acting on the i-th qubit, where α = x,y,z]. The problem Hamiltonian is thus encoded in the operator:

    Ĥ_P = - ∑_i h_i σ̂_̂î^̂ẑ - ∑_i < j J_ijσ̂_̂î^̂ẑσ̂_̂ĵ^̂ẑ.




§ STATE OF THE ART FOR QA SCHEDULING






 §.§ Basic principle of QA


Given a quantum Hamiltonian Ĥ_P whose ground state is hard to find, one uses a control Hamiltonian Ĥ_C with a simple ground state to initialize the system in this known configuration. In the initial formulation of QA by Kadowaki and Nishimori <cit.>, Ĥ_C is chosen as the uniform transverse field Hamiltonian:

    Ĥ_C = - ∑_i=1^N σ̂_̂î^̂x̂,

whose ground state is the uniform superposition |+⟩^⊗ N. The QA process slowly evolves the system from an initial dynamic dictated by Ĥ_C towards a final dynamic encoded by Ĥ_P through the global Hamiltonian:

    Ĥ(t) = (1-u(t)) Ĥ_P + u(t) Ĥ_C.


The function u(t) is a control function obeying the initial condition u(0) = 1 so that the system starts in the ground state |+⟩^⊗ N. If the control function decreases slowly towards 0 during a period T, the final state of the system is expected to be the ground state of Ĥ_P with high probability. This period T is called the annealing time and its duration can be estimated thanks to the adiabatic theorem.



 §.§ Adiabatic approximation


The so-called adiabatic theorem is a mathematical statement on the time evolution of the global Hamiltonian Ĥ(t) to ensure that the instantaneous state |ψ(t)⟩ of the system remains close to the instantaneous ground state |ϕ_0(t)⟩. Several formulations exist with different precision levels reviewed in <cit.>. If one denotes {ε_n(t), |ϕ_n(t)⟩} the instantaneous eigensystem of Ĥ(t) and Δ(t) = ε_1(t) - ε_0(t) is the spectral gap between the ground state and the first excited state, the most commonly used adiabatic approximation reads <cit.>:

    |⟨ϕ_1(t) | ∂_t Ĥ(t) | ϕ_0(t) ⟩ |/Δ(t)^2   ≪ 1    ∀ t ∈ [0, T] ⇒ P_QA(T) ≈ 1,

where we introduced the notation used by <cit.>:

    P_QA(t) = |⟨ϕ_0(T) | ψ(t) ⟩ |^2.

The success probability of QA naturally corresponds to P_QA(T). In our case, the time dependence of Ĥ(t) is fully contained in the control function u. Using Eq. <ref>, the adiabatic approximation reads:

    | du/dt|  |⟨Ĥ_C - Ĥ_P ⟩_1,0(u) |/Δ(u)^2≪ 1   ∀ t ∈ [0,T]   ⇒ P_QA(T) ≈ 1.

which constraints the control function u. It is quite easy to show that the numerator |⟨Ĥ_C - Ĥ_P ⟩_1,0(u) | scales at most as 𝒪(Poly(N)) (see <cit.> for instance). Thus, a qualitative discussion of the adiabatic approximation as a function of the system size is often focused on the spectral gap Δ^2. At the end, one usually uses the following constraint on the control function u:

    | du/dt| Δ(u)^-2≪ 1   ∀ t ∈ [0,T].




 §.§ Linear scheduling


Several theoretical analysis of QA use a linear control function <cit.> to parameterize the annealing process:

    u_lin(t) = 1 - t/T_lin.

In this case, T_lin naturally corresponds to the annealing time. However, the first derivative of u is constant which heavily constraints T_lin. Indeed, Eq. <ref> immediately imposes:

    T_lin≫Δ^-2_min,

where Δ^2_min = 0 ≤ u ≤ 1min Δ^2(u). However, if the system encounters a first order quantum phase transition during the annealing process, it is known that the minimum of the gap closes exponentially with the system size <cit.>. Concretely, it means that T_lin∼ e^α N with α > 1 so QA would offer no advantage over an exhaustive search to compute b̂^MAP.

The work <cit.> underlines that in this case one can enhance the annealing time by looking for a control function with a non constant derivative.



 §.§ Optimized control function


Recall that u is a decreasing function from u(0) = 1 to u(T) = 0, which means that du/dt < 0. As it is done in <cit.>, we can fix an arbitrary small constant ϵ > 0 and impose:

    du/dt   = - ϵΔ^2(u(t)) 
    
            u(0)    = 1
        ,

which directly comes from Eq. <ref>. We refer to the solution of ordinary differential equation (ODE) as the optimized control function. Qualitatively, this equation tells that the larger the gap, the faster the control function can decrease. Integrating both sides of this ODE from t=0 to t=T yields the same expression than <cit.> for the annealing time:

    T = 1/ϵ∫_0^1 du/Δ^2(u).

These two equations <ref>, <ref> underline that once the gap is known, one can fully determine the annealing time and the control function associated to a problem encoded in Ĥ_P.



 §.§ An example


Going back to AUD, one can build the quantum Ising Hamiltonian <ref> once the received signal y and the channel coefficients w are known. Thus, we call a pair (y, w) a problem instance that allows to compute the Ising parameters of Eq. <ref>. Since the time dependence of the full Hamiltonian Ĥ is entirely contained in the control function, one can take u ∈ [0,1] as an affine parameter to evaluate the spectral gap as a function of u before even knowing the time dependence u(t). Thus, we denote Δ^2_(y, w)(u) the spectral gap associated to the problem instance (y, w). From Eq. <ref> and <ref>, one defines the associated control function:

    du_(y, w)/dt   = - ϵΔ^2_(y, w)(u_(y, w)(t)) 
    
            u_(y, w)(0)    = 1
            ,

which is the optimal one for (y, w). The associated annealing time reads:

    T_(y,w) = 1/ϵ∫_0^1 du/Δ^2_(y, w)(u).


As an example, we considered a network of N=8 users and generated a problem instance (y,w) in the following scenario:

    Activity pattern:    b^(0) = (1,0,0,0,0,0,0,0) 
    Perfect channels:    w∼Dirac(1_N) 
    No additive noise:    ξ = 0
        .

We show on Fig. <ref> the two first energy levels of the global Hamiltonian Ĥ(u). Fig. <ref> gives the corresponding spectral gap Δ^2_(y,w). These results have been obtained by exact diagonalization of the Hamiltonian using the QuTiP[<https://qutip.org>] library. Then we numerically evaluate the annealing time T_(y,w) and solve Eq. <ref> on the interval [0,T_(y,w)] to obtain the optimized control function (Fig. <ref>). To do so, we fixed our precision parameter to ϵ = 0.1. The evolution of the overlap during the annealing process scheduled with this control function is finally shown on Fig. <ref>. One can see that the success probability P_QA(T_(y,w)) reached at the end of the evolution is very close to 1, as expected.
However, it would be very long and inconvenient to require to compute T_(y,w) and u_(y,w) for each instance (y,w) of our problem. The goal of the following sections is to emphasize the possibility to build a generic approach to parameterize a QA process for AUD.



§ PROPOSITION: GENERIC CONTROL FUNCTION WITH PERFECT CHANNELS


We would like to check if, despite the randomness of the parameters (y, w), we can estimate a generic control function for a given size of the network N. We first consider a simple scenario with perfect channels.



 §.§ Mean gap in the absence of additive noise




Let us begin with a scenario where the size of the network N is fixed and where the additive Gaussian noise is neglected, which amounts to fix ξ = 0. We randomly generated 30 samples of (y, w) to check how the associated gaps Δ^2_(y,w) behave. The signals y are generated from uniform samples of activity patterns b^(0) and the channel coefficients are given by w = 1_N. Interestingly, Fig. <ref> shows that a pattern emerges for the shape of the gap against u despite the dependence of the local magnetic fields h_i in y. 

These results motivate the definition of the mean gap:

    Δ^2_(mean),N(u) = 𝔼_(y,w)(Δ^2).

In our specific case, this quantity can be exactly computed. Indeed, the channel coefficients are fixed to  w = 1_N and the received signal y is fully determined from the initial activity pattern b^(0). Using that b^(0) is uniformly distributed, one has:

    Δ^2_(mean),N = 1/2^N∑_b^(0)∈{0,1}^NΔ^2_(y, w=1_N)(u).

Then, we can use this mean gap to compute the associated annealing time T_(mean),N with Eq. <ref>:

    T_(mean),N = 1/ϵ∫_0^1 du/Δ^2_(mean),N(u),

and the resulting control function u_(mean),N(t) from Eq. <ref>:

    du_(mean),N/dt   = - ϵΔ^2_(mean),N(u_(mean),N(t)) 
    
            u_(mean),N(0)    = 1
        .


Once for all, we adopt the precision level ϵ = 0.1. We reported the shape obtained for u_(mean),N(s) for N = 6,…,9 on Fig. <ref> with respect to the normalized time s = t/T_(mean),N. Then, we must test for each value of N whether these functions are good candidates to schedule a QA process on the period t ∈ [0, T_(mean),N] to estimate b̂^MAP.









 §.§ Definition of the metrics


We will compare for several (y,w) the success probability of a QA process scheduled with our approach with the one that would been obtained with an optimal scheduling for each problem instance. To do so, one need to define appropriate metrics. We first generate a pair (y, w) and simulate a QA process parameterized with u_(mean),N by numerically solving the Schrödinger's equation:

    i ∂_t |ψ(t)⟩ = Ĥ(u_(mean),N(t)) |ψ(t)⟩   ,    t ∈ [0,T_(mean),N],

with the initial condition |ψ(0)⟩ = |+⟩^⊗ N (we work with natural units where ħ =1) using again the QuTiP library. The obtained success probability is denoted with the short-hand notation:

    P_QA(T_(mean),N ; (y, w)) ≡ P_QA^(mean)(y, w).

On the other hand, the reference success probability is the one corresponding to a QA process scheduled with the well-suited optimized control function u_(y,w) (Eq. <ref>) and the corresponding annealing time T_(y,w) (Eq. <ref>). We adopt also a short hand notation:

    P_QA(T_(y,w) ; (y, w)) ≡ P_QA^(opti)(y, w).


Since the instances (y,w) are randomly distributed, P_QA^(mean) P_QA^(opti) are random variables. Thus, we propose to use their expectation values to quantify the reliability of each scheduling strategy. We turn on the additive Gaussian noise to realize the performance analysis, hence the expectation values are given by:

    𝔼_(y,w)(P_QA^(α)) = ∑_b^(0)∈{0,1}^N∫ dn f(b^(0), n) P_QA^(α)(y,w),

for α∈{mean, opti}. The function f(b^(0), n) is the probability density function associated to the instances (y, w). The channel coefficients are still fixed to w = 1_N but the signal y is now generated from the sample of an activity pattern b^(0) and a Gaussian vector n. Those two parameters are independently distributed, respectively following the laws 𝒰({0,1}^N) and 𝒩(0,ξ^2). Thus, the weights of Eq. <ref> are given by:

    f(b^(0), n) = 1/2^N×1/(2πξ)^M/2exp(-‖n‖^2/2 ξ^2).


We also need a metric to quantify the intensity of the Gaussian noise. A well adapted one is the signal-to-noise ratio (SNR) expressed in dB. Since the power of the signal sent individually by each user is normalized to P_signal,i = 1, the SNR roughly reads:

    SNR  = -20 log_10(ξ).




 §.§ Reliability of our strategy


The instances (y, w) have no longer a finite support. Thus, we generated N_samples = 500 pairs (y, w = 1_N) according to the weights of Eq.<ref> for N = 6, …,9 to estimate the distributions of P_QA^(mean) and P_QA^(opti). The expectation values for α∈{mean, opti} are estimated by:

    𝔼_(y,w)(P_QA^(α)) = 1/N_samples∑_(y,w) ∈samples P_QA^(α)(y, w).


We reported on Fig. <ref> the obtained histograms for a noise level corresponding to SNR = 20dB. We can observe that the values of P_QA^(mean) are more spread out along the axis than the values of P_QA^(opti) which is not surprising. Indeed, the control function u_mean,N cannot be well suited for all possible outcomes (y,w). Nevertheless, the expectation value 𝔼_(y,w)(P_QA^(mean)) is always above 0.8 which is reasonably below 𝔼_(y,w)(P_QA^(opti)).

We also reported the obtained distributions for a level of noise corresponding to SNR = 15dB. The gap between 𝔼_(y,w)(P_QA^(mean)) and 𝔼_(y,w)(P_QA^(opti)) is higher but our approach can still offer a mean success probability above 0.7.

Thus, our generic control function appears as a good compromise to schedule a QA approach for any set of parameters (y, w) given a size of the network N








 §.§ Advantage over a linear control function


Once the annealing time T_(mean),N is known thanks to Eq. <ref>, one might question the necessity to use the control function u_(mean),N on the annealing period [0, T_(mean),N]. Indeed, the differential equation <ref> to obtain the optimized control function associated to a given gap results from several approximations. Thus, one might be tempted to simply use a linear control function with an annealing period T_lin,N = T_(mean),N and expect the resulting success probability to be not too low even if the condition of Eq. <ref> is not satisfied. To answer this question, we define the success probability of a QA process planned according to this strategy:

    P_QA(T_(lin),N = T_(mean),N ; (y,w)) ≡ P_QA^(lin)(y,w).


Following the same approach than above, we reported on Fig. <ref> the distributions of P_QA^(lin) and P_QA^(mean) for a level of noise corresponding to SNR = 20dB. The expectation value 𝔼_(y,w)(P_QA^(mean)) is higher than 𝔼_(y,w)(P_QA^(lin)) which confirms the advantage of using u_mean,N.

As the noise increases, Fig. <ref> shows that the gap between the two expectation values 𝔼_(y,w)(P_QA^(mean)) and 𝔼_(y,w)(P_QA^(lin)) is reduced. Nevertheless at SNR = 15dB, our control function is still expected to yield a success probability higher than the one that would been obtained with a linear control function



§ CONTROL FUNCTION IN THE GENERAL CASE


In the previous section we proposed a generic method to evaluate an appropriate control function and demonstrated its validity on random problem instances with thermal noise at the BS. However in more realistic scenarios, the received signal can also be impacted by the channel coefficients. In this section, we take into account the channels between the users and the BS. From now on, the channel coefficients are normal variables to account for Rayleigh fading:

    w∼𝒩(0,I_N).




 §.§ Behavior of the gap









First of all, let us go back to our scenario of N=8 users with an initial activity pattern b^(0) = (1,0,0,0,0,0,0,0). We generate one random problem instance (y,w) with no additional Gaussian noise by independently sampling the N channel coefficients according to the standard normal distribution.

As done before, we reported on Fig. <ref> the eigenvalues ε_0,1 of the global Hamiltonian Ĥ and the corresponding spectral gap. This time, the minimum of the gap is reached at the end of the annealing process when the control function vanishes.

This new behavior can be explained by the modifications induced in the spectrum of the Ising Hamiltonian Ĥ_P. In the case where a user i is strongly attenuated by a coefficient w_i close from 0, the spin flip σ_i → -σ_i will not significantly modify the associated eigenenergy. Thus, taking into account the attenuation on the propagation paths between the users and the access point strongly reduces the gaps between the eigenvalues of the associated Ising Hamiltonian Ĥ_P. It suggests that the mean gap should be estimated again in this new scenario.



 §.§ Mean control function with fading coefficients


As done previously, we first neglect the additive Gaussian noise in order to evaluate the mean gap from several problem instances. From this quantity, we derive the associated mean control function u_(mean),N defined on [0,T_(mean),N]. Since the channel coefficients w are now also randomly distributed, Eq. <ref> is generalized to:

    Δ^2_(mean),N(u) = ∑_b^(0)∈{0,1}^N∫ dw f(b^(0),w) Δ^2_(y,w)(u).

Since b^(0) and w are independently distributed, the above probability density function is simply given by:

    f(b^(0),w) = 1/2^N×1/(2π)^N/2exp(-‖w‖^2/2).

Contrarily to the scenario with perfect channels, the mean gap cannot be evaluated exactly through Eq. <ref>. Thus, we sample the probability distribution f(b^(0),w) with N_samples, gap = 2000 samples and evaluate the mean gap with the estimator:

    Δ^2_(mean),N(u) = 1/N_samples, gap∑_(y,w) ∈samplesΔ^2_(y,w)(u).

As done previously, we reported on Fig. <ref> the shape of the control functions against the normalized time s = t/T_(mean),N for N = 6, …, 9. These shapes indicate that the first derivative is monotonically decreasing over the annealing period, which is expected regarding the previous comments about the behavior of the gaps Δ^2_(y,w). Let us now check whether this approach still offers a good compromise in terms of success probability.







 §.§ Mean success probability


As we did with the perfect channel scenario, we would like to check that the mean control function still offers a good success probability when the noise is taken into account. We use again the metrics P_QA^(mean) and P_QA^(opti). The expression of their expectation values is modified according to the distribution of the instances (y,w):

    𝔼_(y,w)(P_QA^(α)) = ∑_b^(0)∈{0,1}^N∫ dn dw f(b^(0), n,w) P_QA^(α)(y,w).

for α∈{mean,opti}. Using again the independence of the parameters used to generate a pair (y,w), the above probability distribution is given by:

    f(b^(0),n,w)     = 1/2^N×1/(2πξ)^M/2exp(-‖n‖^2/2 ξ^2) 
    ×   1/(2π)^N/2exp(-‖w‖^2/2).


An estimation of the distributions of P_QA^(mean) and P_QA^(opti) at SNR = 15dB is shown on the histograms of Fig. <ref>. These results have been obtained by performing again N_samples = 500 samples of problem instances to evaluate P_QA^(mean)(y,w) and P_QA^(opti)(y,w). Unfortunately, the mean success probability 𝔼_(y,w)(P_QA^(mean)) is around 0.65 while a well-suited control function for each problem instance still ensures a mean success probability 𝔼_(y,w)(P_QA^(opti)) above 0.9. We must then investigate a way to improve the mean control function in this more complex AUD scenario.



 §.§ Time dilatation


The reduction of the mean success probability 𝔼_(y,w)(P_QA^(mean)) is likely due to the fact that u_(mean),N decreases too fast at the end of the annealing process. Indeed, the example previously shown on Fig. <ref> suggests that when the control function approaches 0, the gap closes which means that u must decrease slowly enough to offer a good reliability. In order to force u_(mean),N to decrease slower, we propose to apply a time dilatation factor λ to its associated annealing period [0, T_(mean),N]. In practice, it amounts to use the time-dilated mean control function:

    u_(mean),λ,N(t) = u_(mean),N(t/λ).

Consequently, the annealing period is extended to [0, λ T_(mean),N]. Of course, we aim at keeping λ reasonably low to preserve the order of magnitude of the annealing time. 

Nevertheless, we show on Fig. <ref> that time dilatation factors of λ = 2,3,4 already allow to boost the mean success probability 𝔼_(y,w)(P_QA^(mean)). These results correspond to a network with N=8 users and a level of additive noise fixed to SNR = 15dB. For instance, picking λ = 2 allows to reach a mean success probability above 0.75 which is significantly better than the results obtained in the previous paragraph.

We also compared again our proposal to a linear control function parameterized over the same annealing period [0,λ T_(mean),N]. The results on Fig. <ref> confirm that u_(mean),λ,N offers significantly a higher mean success probability 𝔼_(y,w)(P_QA^(mean)) than a linear control function.

Thus, our strategy is also promising in this more complex AUD scenario. A time dilatation can be applied to the mean control function to improve the performances.



§ CONCLUSION




 §.§ Contributions

In this work, we proposed a generic strategy to parameterize a QA process with the usual transverse Hamiltonian to compute the MAP estimator for AUD in a NOMA network. Despite the randomness of several parameters involved in the couplings and the local fields of the corresponding targeted Ising Hamiltonian, the evolution of the spectral gap of the global Hamiltonian against u(t) follows a recurrent shape. We have shown that our mean control function ensures a reasonable expectation value for the success probability of QA after (if wanted) applying a time dilatation operation. This function has the advantage to be universal once the size of the network N is fixed and permits to reach better performances than the simple linear control function.



 §.§ Scalability with N

The work conducted in this paper is limited to small network sizes. For larger values of N, the computation of the quantum gaps like those involved in Eq. <ref> would be too long with the exact diagonalization method of the full Hamiltonian used in this work. A further improvement would be to use Quantum Monte Carlo methods as done by <cit.> to extract quantum gaps involved in the mean gap estimator in order to compute it for N ≥ 9.



 §.§ Knowledge of the channel coefficients

Regarding the AUD model, another improvement would be to relax some assumptions we made. In fact, the knowledge of the channel coefficients w at the access point is a strong assumption that does not hold true in practice. Some modern formulations of AUD assume that the statistics of the channel vector w (which is a Gaussian law in this work) is known by the access point but not the realizations. A further perspective to this work could be to adapt the signal processing model to such case.




§ ACKNOWLEDGMENT


The authors would like to thank Lélio Chetot and Bruno Fedrici for helpful discussions.

This work was supported by the ANR under the France 2030 program, grant "NF-PERSEUS : ANR-22-PEFT-0004".






§ CHOICE OF THE SEQUENCES



The identification sequences of the users must be chosen such that each activity pattern b^(0) yields a different signal y in the absence of imperfections. It means that in the case ξ = 0 and w = 1_N, one requires:

    {y(b^(0),n = 0, w=1_N) .   = . y(b^(1),n = 0, w=1_N)}
    ⇒   {b^(0) = b^(1)}.

One can show that this condition is ensured if and only if:

    {∃λ∈{-1,0,1}^N   s.t  ∑_i=1^N λ_i c_i = 0}⇒{λ = 0}.

Throughout this work, we used coding schemes constructed by a random selection of N codes of size M followed by an exhaustive verification of the condition <ref>. We give below the matrices C = [c_1 , …, c_N] ∈ℂ^M × N corresponding to the codes we used for N = 6,…,9.

    C_6    = 1/√(5)[ -1 -1  1  1 -1 -1;  1  1 -1  1  1 -1; -1 -1  1  1  1  1;  1 -1  1  1 -1 -1; -1  1  1  1 -1  1 ],


    C_7    = 1/√(6)[  1  1  1  1 -1 -1 -1;  1  1  1  1  1 -1  1;  1 -1 -1  1 -1  1  1; -1  1  1  1  1 -1 -1;  1  1  1 -1 -1  1 -1; -1 -1  1 -1 -1 -1  1 ], 
    C_8    = 1/√(6)[  1  1 -1  1  1 -1  1 -1;  1 -1 -1 -1  1 -1  1  1; -1  1  1 -1  1 -1 -1  1;  1  1 -1 -1  1  1 -1 -1;  1  1 -1  1  1  1 -1  1; -1  1 -1  1 -1 -1 -1 -1 ], 
    C_9    = 1/√(7)[ -1 -1 -1 -1 -1 -1 -1 -1 -1; -1 -1  1 -1 -1 -1  1 -1  1; -1 -1 -1  1 -1  1 -1 -1  1; -1  1  1 -1  1 -1 -1  1 -1; -1 -1  1  1  1 -1  1 -1  1; -1  1  1  1  1  1  1  1  1; -1 -1  1 -1 -1  1  1  1  1 ].




§ QUBO - ISING MAPPING


One uses the change of variable σ_i = 1-2 b_i in the expression of the objective function <ref> without taking into account the irrelevant constant ||y||^2:

    ‖y-C·.   . diag(w) ·b‖^2 ∼ -2 ∑_i=1^N (y· w_i c_i) (1/2(1-σ_i)) 
        + ∑_i,j=1^N (w_i c_i · w_j c_j )(1/2(1-σ_i)) (1/2(1-σ_j)).

We throw away the constant terms (ie. not depending on the σ's) to obtain:

    ‖y-C·.   . diag(w) ·b‖^2 ∼∑_i=1^N σ_i ( w_i y·c_i - 1/2 w_i c_i ·∑_j w_j c_j) 
        + 1/4∑_i,j=1^N (w_i w_j c_i ·c_j) σ_i σ_j.

Using that σ_i^2 = 1, the diagonal contribution of the double sum is also an irrelevant constant. Since the sum is symmetric under i ↔ j, we can use ∑_i  j≡ 2 ∑_i < j. Hence the expression of the couplings in the main text:

    J_ij = - 1/2 (w_i w_j) (c_i ·c_j).

As for the local fields, one can directly identify:

    h_i = -w_i c_i ·(y - 1/2∑_j w_j c_j).

It matches the expression of the main text:

    h_i = -w_i c_i ·ỹ,

with the signal ỹ expressed as in Eq. <ref>
*





