Paper Title: Distributed Adaptive Time-Varying Optimization with Global Asymptotic Convergence.


 
          
















Distributed Adaptive Time-Varying Optimization with Global Asymptotic Convergence
    Liangze Jiang, Zheng-Guang Wu, and Lei Wang 

The authors are with College of Control Science and Engineering, Zhejiang University, Hangzhou 310027, China (e-mail: zetojiang; nashwzhg; lei.wangzju @zju.edu.cn). (Corresponding author: Zheng-Guang Wu.)
    August 5, 2024
==============================================================================================================================================================================================================================================================



§ ABSTRACT 
In this note, we study distributed time-varying optimization for a multi-agent system. We first focus on a class of time-varying quadratic cost functions, and develop a new distributed algorithm that integrates an average estimator and an adaptive optimizer, with both bridged by a Dead Zone Algorithm. Based on a composite Lyapunov function and finite escape-time analysis, we prove the closed-loop global asymptotic convergence to the optimal solution under mild assumptions. Particularly, the introduction of the estimator relaxes the requirement for the Hessians of cost functions, and the integrated design eliminates the waiting time required in the relevant literature for estimating global parameter during algorithm implementation. 

We then extend this result to a more general class of time-varying cost functions. Two numerical examples verify the proposed designs.






Distributed optimization, quadratic
optimization, time-varying cost functions, adaptive gain.





§ INTRODUCTION

Designing and reconstructing the dynamic behavior of groups through optimization and control techniques propels the advancement of distributed optimization theory <cit.>, especially in physical applications such as sensor networks <cit.>, robot networks <cit.> and power systems <cit.>. Recently, there is a trend towards modeling and solving large-scale optimization problems in dynamic environments (e.g., see <cit.> and the references
therein). To cope with constantly changing environmental information, the modeling of cost functions in dynamic environments is typically based on real-time information acquisition by agents (see Fig. <ref>)<cit.>. 

However, the dynamic nature of the environment may disrupt this process, which in turn leads to the `fragility' of cost functions. For instance, in multi-camera localization of moving objects, cameras with fixed positions fail to capture images of mobile objects outside their field of view<cit.>. In multi-robot target tracking, obstacles or limited perceptual distance prevent some robots from obtaining target information<cit.>. When information acquisition is hindered due to environmental changes, weak or missing local cost functions cannot guarantee their uniformly strong convexity about decision variables (e.g., see Example 2 in Section <ref>). Moreover, a realistic topic in autonomous optimization is reducing the dependence of algorithms on global parameter of the cost functions and the communication graph<cit.>. These observations prompt us to investigate distributed time-varying optimization.




Intuitively, during the search for the time-varying optimal solution, gradient information provides only error feedback relative to the current optimal solution. By introducing the time derivative of the gradient as compensation<cit.>, the modified gradient descent method can solve the time-varying optimization problem exactly[The considered time-varying optimization setting is different from the online optimization<cit.> where the structural information on the cost functions is unavailable and non-zero tracking errors are pursued. In contrast, in our problem setting, each agent can construct a complete local cost function by adequately acquiring information about the environment.]. This basic idea is further expanded to distributed versions to tackle large-scale time-varying optimization problems in <cit.>, some of which consider equality <cit.> and inequality <cit.> constraints. In <cit.>, a robust control method is introduced to solve distributed time-varying quadratic optimization (DTQO) with coupled cost functions. However, as mentioned earlier, the cost functions established in dynamic environments may exhibit time-varying and non-strongly convex properties. In such cases, the existing algorithms cannot be directly applicable, as they require the Hessians of all local cost functions to be positive definite<cit.>, identical<cit.>, diagonal <cit.> and time-invariant<cit.>. Although one design in <cit.> considers nonpositive Hessians of the local cost functions, its assumption of constant upper bounds on certain complex functions limits the types of cost functions. Moreover, its convergence result is semi-global since the initial states of agents are needed for the design. Additionally, the waiting time in <cit.> implies a two-stage algorithm implementation, requiring knowledge of the initial states of agents, the communication graph, or other prior global parameters of the cost functions to determine the size of it before running the algorithm.
 
 
Motivated by this, we start with a basic DTQO and develop a distributed algorithm that seamlessly integrates an average estimator with an adaptive optimizer, based on the nonsingular matrices output by the Dead Zone Algorithm (DZA). We then extend this result to address the distributed optimization problem of a general class of cost functions, not limited to quadratic cost functions<cit.>. The contributions lie in: (1) We propose distributed optimization algorithms without relying on positive definite, identical, diagonal or time-invariant Hessians of all local cost functions. To the best of our knowledge, these are the first result of distributed optimization for the aforementioned types of cost functions. (2) By constructing a composite Lyapunov function and introducing the concept of the finite escape time, we establish the asymptotic convergence of closed-loop outputs to the optimal solution in the global sense, rather than in the semi-global sense <cit.>. (3) Different from <cit.>, the proposed design does not require the waiting time during algorithm implementation, thereby removing the requirement for prior parameters used to determine its size. Such design also relaxes the assumption that the upper bounds of certain time-varying parameters or their high-order time derivatives are known in advance <cit.>. 

 
 



Notations. The set of real numbers is represented by ℝ. Let 1_N denote the vector (1,⋯,1)^⊤∈ℝ^N and I_m denote the m-dimensional identity matrix. For a matrix A∈ℝ^m× m, we use λ_p(A), p∈{1,⋯,m}, to represent the p-th smallest eigenvalue of A. For a vector or a matrix X, we use |X|_p to represent the l_p-norm of X. |X| is the abbreviation of |X|_2. The signum function is denoted by (y), with y∈ℝ. For a vector x=(x_1,⋯,x_n)^⊤, define (x)=((x_1),⋯,(x_n))^⊤ and S(x)=(S(x_1),⋯,S(x_n))^⊤, where S(x_i)=x_i/(|x_i|+ϵ_1η_t) with ϵ_1>0 and η_t∈ℝ. For a matrix A=[a_ij]∈ℝ^m× n, define (A)=[(a_ij)]∈ℝ^m× n and use A_⊗ to denote A⊗ I_m. Denote {A_i}, i=1,⋯,N as a diagonal matrix with diagonal entries composed of A_1,⋯,A_N, where A_i∈ℝ^m_i× m_i. The gradient of a function f(x,t):ℝ^m×ℝ_+→ℝ with respect to x is denoted by ∇ f(x,t). Let ∇_t f(x,t) denote the partial derivative of ∇ f(x,t) with respect to time t.






§ PRELIMINARIES AND PROBLEM STATEMENT




 §.§ Preliminaries

A differentiable function f(x,t):ℝ^m×ℝ_+→ℝ is uniformly ζ-strongly convex in x if there exists ζ>0 such that (a-b)^⊤(∇ f(a,t)-∇ f(b,t))≥ζ |a-b|^2, ∀ a,b∈ℝ^m, ∀ t≥0. ∇ f(x,t) is uniformly θ-Lipschitz in x if there exists θ>0 such that |∇ f(a,t)-∇ f(b,t)|≤θ|a-b|, ∀ a,b∈ℝ^m, ∀ t≥0. 

The communication network is described by an undirected graph 𝒢=(𝒩,ℰ), where 𝒩={1,…,N} is the node set and ℰ⊆𝒩×𝒩 is the edge set. Denote the weighted adjacency matrix of 𝒢 as 𝒜=[a_ij]_N× N. An edge (i,j)∈ℰ means that nodes i,j can receive information from each other. Define a_ij=1 if (j,i)∈ℰ, and a_ij=0 otherwise. Let 𝒩_i={j∈𝒩:(j,i)∈ℰ} denote the set of node i's neighbors. The Laplacian of a graph 𝒢 is defined as L=[l_ij]_N× N with l_ii=∑_j≠ ia_ij and l_ij=-a_ij for j≠ i. If there is a path from any node to any other node in graph 𝒢, it is called connected. Denote the incidence matrix associated with 𝒢 as D=[d_ik]_N× |ℰ|, where d_ik=1 if the edge e_k enters the node i, d_ik=-1 if the edge e_k leaves the node i, and d_ik=0 otherwise.


(Graph connectivity) The graph 𝒢 is undirected and connected.


By Assumption <ref>, it is known that the Laplacian matrix L of the graph possesses a zero eigenvalue, corresponding to the eigenvector 1_N, while the remaining eigenvalues are positive <cit.>, which satisfy λ_1(L) = 0 < λ_2(L) ≤⋯≤λ_N(L).



 §.§ Problem Definition

Consider a group of N agents interacting over a communication network 𝒢=(𝒩,ℰ). Each agent i∈𝒩 is assigned a local cost function defined by 

    f_i(x_i,t)=1/2x_i^⊤H_i(t)x_i+R_i^⊤(t)x_i+d_i(t),

where x_i∈ℝ^m is the decision variable, and H_i(t)∈ℝ^m× m, R_i(t)∈ℝ^m and d_i(t)∈ℝ are time-varying parameters. We aim to design a distributed algorithm such that each agent solves the optimization problem

    min_r∈ℝ^mf(r,t)=∑_ i∈𝒩f_i(r,t).

The optimal solution of the above optimization problem is defined as r^*(t)=min_r∈ℝ^mf(r,t). To guarantee the first-order optimality condition ∇f(r^*(t),t)=0 holds, ∀ t≥ 0, there is (d/dt)∇ f(r^*(t),t)=H(t)ṙ^*(t)+∇_t f(r^*(t),t)≡0, where H(t)=∑_i∈𝒩H_i(t). Then, if H(t) is invertible, it follows that ṙ^̇*̇(t)≡-H^-1(t)∇_t f(r^*(t),t). The following assumption guarantees the uniqueness of r^*(t) and the positive definiteness of the Hessian H(t), ∀ t≥0.


(Uniform strong convexity) The global cost function f(r,t) is uniformly h_1-strongly convex with respect to r. That is, there exists a known h_1>0 such that λ_1(H(t))≥ h_1, ∀ t≥0.


By this assumption, each f_i(x_i,t) here is not required to be uniformly strongly convex<cit.>. It can even be non-convex. Moreover, the Hessians of all f_i(x_i,t) do not have to be diagonal <cit.> or time-invariant <cit.>, nor do they have to be completely identical as required in <cit.> and <cit.>.


If the local cost functions are all convex, and there exists a strongly convex one with a known strongly convex coefficient (or a more conservative lower bound) h, then, h_1 can directly take h (see <cit.> for details).




(Boundedness) For any i∈𝒩, |H_i(t)|, |R_i(t)| and |Ṙ_i(t)| are bounded, ∀ t≥0, but their upper bounds are unknown. Moreover, there exists a known h_2>0 such that |Ḣ_i(t)|_∞≤h_2, ∀ t≥0.

We stress that Assumption <ref> guarantees the linear boundedness of ∇_t f_i(x_i,t) with respect to x_i, which is more relaxed than the assumptions that directly impose known and constant bounds on ∇_t f_i(x_i,t) in <cit.> and (d/dt)∇_t f_i(x_i,t) in <cit.>. By the derivations below (<ref>), Assumptions <ref> and <ref> guarantee the boundedness of r^*(t) and thus ṙ^*(t). Moreover, we only require knowledge of the lower bound of the Hessian's eigenvalue and the upper bound of Hessian's changing rate. Additional prior knowledge of cost functions, such as the upper bounds of (∂/∂ t)∇_t f_i(x_i,t) in <cit.>, ⃛r^*(t) in <cit.>, R̈_i(t) in <cit.> and ∇ f_i(x_i,t) in <cit.>, is not necessary here.


In practice, the parameters of cost functions are typically constrained by the properties of physical objects. For example, consider the problem of multi-UAVs tracking of a target<cit.> with the cost function f_i=|x_i-r(t)|^2, where r(t) is the trajectory of the target vehicle. The limited energy and engine power of the vehicle, in fact, must constrain |r(t)| and |ṙ(t)| within a certain range, respectively, but the specific upper bounds of |r(t)| and |ṙ(t)| may be unknown in advance. Assumption <ref> aligns with these observations. 





§ DESIGN BASED ON ESTIMATOR AND OPTIMIZER


Revisiting the centralized optimization algorithm in <cit.>, we can design ṙ=-∇ f(r,t)-H^-1(t)∇_t f(r,t) to guide r to asymptotically converge to the optimal r^*(t). Inspired by this, the following design adopts distributed `∇ f_i' feedback and `NH^-1(t)∇_t f_i' feedforward[The scaling factor N is instrumental for the subsequent derivations.]. In view of the heterogeneity of H_i(t), i∈𝒩, the technique of dynamic average consensus is employed to estimate the average of all Hessians H(t)/N.



 §.§ Average Estimator and Output Reconstruction


For agent i∈𝒩, we first consider the average estimator

    ξ̇_i=   -ω∑_j∈𝒩_i(z_i-z_j),    z_i=ξ_i+H_i(t),

where ω>h_2, ξ_i∈ℝ^m× m, and z_i∈ℝ^m× m is the output of estimator. The initial conditions satisfy ∑_j∈𝒩ξ_i(0)=0 and ξ_i(0) is symmetric. The convergence of z_i to H(t)/N in finite time can be guaranteed by <cit.>. However, since the H_i(t) may not be positive definite, z_i(t) inevitably passes through singular points during its evolution. Considering z^-1_i(t) as a part of feedforward term, we propose Algorithm <ref> to avoid the impact of singular z_i(t), where h_0>0 is to be determined and ℳ={1,⋯,m}. In Algorithm <ref>, we use min_p∈ℳ|λ_p(z_i(t))| to quantify how close z_i(t) approaches singular points, and constrain ẑ_i(t) to be within a `safe range' away from singular points. Consequently, min_p∈ℳ|λ_p(ẑ_i(t))|≥ h_0 holds for all t≥0, regardless of whether the input matrix z_i(t) is singular. We thus refer to Algorithm <ref> as the Dead Zone Algorithm, and the inverse of the output of DZA, ẑ_i^-1(t), remains well-defined during the evolution of z_i(t). 



From the perspective of ensuring nonsingularity, the DZA plays the same role as the projection method in <cit.>. However, since the integrated design shown in Fig. <ref> is adopted in our work, we also need to provide a sufficient condition for parameter selection in the DZA to ensure closed-loop convergence, rather than just ensuring nonsingularity. The following lemma illustrates an important relation between z_i(t) and the `shaped' ẑ_i(t), if we select a suitable parameter h_0. This relation is instrumental for the analysis in Section <ref>.

    Suppose Assumptions <ref>-<ref> hold. For system (<ref>) and Algorithm <ref> with h_0=γ h_1/N, the boundary condition |ẑ_i(t)-H(t)/N|≤k_1|z_i(t)-H(t)/N| holds ∀ t≥ 0, with k_1>0.


See Appendix <ref>.



Different from <cit.>, although the finite-time average estimator (<ref>) is adopted here, the size of convergence time (i.e., the waiting time in <cit.>) is unnecessary for agents to obtain, because we do not employ the execution logic of a two-stage algorithm implementation.










 §.§ Adaptive Optimizer

For agent i∈𝒩, consider the following adaptive optimizer

    ẋ_i=   2.13ex-∑_j∈𝒩_i(α_ij(x_i-x_j)+β_ijS(x_i-x_j))_ Adaptive consensus coordinator+ϕ_i,
    ϕ_i=   -k∇ f_i(x_i,t)-ẑ_i^-1(t)∇_t f_i(x_i,t),
    α̇_ij=   |x_i-x_j|^2,   β̇_ij=|x_i-x_j|_1-mϵ_1η_t,  j∈𝒩_i,

where the function S(·) is defined in Notations, η_t=exp(-ϵ_2t) with ϵ_2>0, k>0 is to be determined, α_ij(0)=α_ji(0)≥0 and β_ij(0)=β_ji(0)≥ mϵ_1/ϵ_2 with ϵ_1>0. As a continuous approximation of the signum function, the function S(·) reduces the chattering of x_i in application.

In (<ref>), ϕ_i incorporates distributed `∇ f_i' feedback and `ẑ_i^-1(t)∇_t f_i' feedforward. The consensus coordinator utilizes adaptive control gains α_ij and β_ij to handle unknown global parameter (see details in Appendix <ref>). Fig. <ref> illustrates the structure of the proposed design for agent i, where dashed arrows represent information received from neighboring agents. Note that if x_i denotes the position of agent i, and x_i-x_j can be obtained through relative measurements (e.g. the acoustic and vision sensors), only the variable z_j is transmitted through the communication channel. Therefore, this design reduces the communication resources usage compared to <cit.>.


The DZA bridges the average estimator and adaptive optimizer, enabling the implementation without relying on the waiting time (required in <cit.>). With this cascaded structure, the co-design of these three modules is needed to ensure the output x_i converges to the time-varying but unknown r^*(t), which is a major challenge of this note. Additionally, the dead-zone nonlinearity introduced by DZA and the inverse operation of its output, (·)^-1, also cause difficulties in the subsequent convergence analysis.


The trajectory of ẑ_i(t) generated by Algorithm <ref> is discontinuous but measurable and locally essentially bounded. Thus, the Filippov solutions of the proposed design always exist. Since the Lyapunov functions used in convergence analysis are continuously differentiable, the proof still holds without using the nonsmooth analysis<cit.>.

 


§ CONVERGENCE ANALYSIS


Define x=(x_1,⋯,x_N)^⊤, ϕ=(ϕ_1^⊤,⋯,ϕ_N^⊤)^⊤ and a diagonal matrix B, whose order is equal to the number of edges in the graph 𝒢. The nodes corresponding to non-zero elements in column i of matrix D are denoted as i_1 and i_2, respectively. The diagonal element B_ii of B is defined as β_i_1i_2. Then, we rewrite (<ref>) as

    ẋ   =-L̅_⊗x-(DB)_⊗S(D^⊤_⊗x)+ϕ,

where L̅=[l^α_ij(t)]∈ℝ^N× N with l^α_ii(t)=∑_j≠ iα_ij(t) and l^α_ij(t)=-α_ij(t) for j≠ i. Define the consensus error e=M_⊗ x with M=I_N-1_N1_N^⊤/N, and the error between average state and global optimal solution x̃=1^⊤_Nx/N-r^* with 1^⊤_N=1_N^⊤⊗ I_m. This implies the relation between x, e and x̃, i.e.,

    x=e+1_N⊗x̃+1_N⊗ r^*.
  
With ∇ F(x,t)=(∇ f_1(x_1,t),⋯,∇ f_N(x_N,t))^⊤, we obtain 

    ė=   -L̅_⊗e-(DB)_⊗S(D^⊤_⊗e)+M_⊗ϕ,
    ẋ̃̇=   -k/N1^⊤_N∇ F(x,t)-1/N∑_i∈𝒩ẑ^-1_i∇_t f_i(x_i,t)-ṙ^*,

where ṙ^*=-H^-1(t)∑_i∈𝒩∇_t f_i(r^*,t).

The following theorem gives the main result of this note.

Suppose Assumptions <ref>-<ref> hold. For system (<ref>)-(<ref>) with Algorithm <ref>, let h_0=γ h_1/N and k>h_2N√(m)/h_1^2+ϵ_3 with 0<γ<1 and ϵ_3>0. Then, the optimization problem (<ref>) is solved, i.e., lim_t→∞x_i(t)-r^*(t)=0, i∈𝒩.

 
See Appendix <ref>.


From a system theoretic perspective, the analysis of all subsystems is naturally unified within the common time domain, which differs from that of the waiting time-based design<cit.>. Specifically, the Lyapunov analysis for e and x̃ subsystems is established based on (<ref>)-(<ref>) and Lemma <ref>. Therefore, the entire system is considered as an interconnected system, as shown in Fig. <ref>.









§ EXTENSIONS: NON-QUADRATIC COST FUNCTION

In this section, we consider local cost functions f_i(x_i,t) of a general form as

    f_i(x_i,t)=1/2x_i^⊤H_i(t)x_i+R_i^⊤(t)x_i+f̂_i(x_i,t), i∈𝒩,
 
where f̂_i(x_i,t) is not limited to the quadratic form. Let ρ_i(x_i,t) denote the Hessian of f_i(x_i,t). We then make the following assumptions on f_i(x_i,t).

For ∀ t≥0 and ∀ i∈𝒩, there exist μ_j>0, j=1,⋯,5 such that 

    
  *  The global cost function f(r,t) is uniformly μ_1-strongly convex and ∇f̂_i(x_i,t) is uniformly globally Lipschitz, 
    
  *  |∇f̂_i(x_i,t)|≤μ_2|x_i|+μ_3,
    
  * ∇_tf̂_i(x_i,t)=g_i(t)x_i+s_i(x_i,t) where |g_i(t)|≤μ_4, |s_i(x_i,t)|≤μ_5s̅(t)<∞ and ∫_0^∞s̅^2(t)  dt<∞.





Although Assumption <ref> requires ∇f̂_i(x_i,t) and ∇_tf̂_i(x_i,t) to satisfy certain conditions, it holds for many cost functions, as shown in Example 1 of Section <ref>. Similar assumptions are also adopted in <cit.>.


With each f_i(x_i,t) defined by (<ref>), the algorithm for solving the optimization problem (<ref>) follows the design process in Section <ref>. However, the estimator (<ref>) cannot be applied here to estimate ∑_i∈𝒩ρ_i(x_i,t)/N with ρ_i(x_i,t) replacing H_i(t). The reason is that ρ̇_i(x_i,t) may contain ẋ_i, whose upper bound is unknown before implementing algorithm. Consequently, the fixed gain ω cannot be determined in advance due to the unknown upper bound of |ρ̇_i(x_i,t)|_∞. To address this issue, we introduce a new average estimator adapted from <cit.>: 

    ξ̇_i=   -∑_j∈𝒩_iω_ij(t)(z_i-z_j),  z_i=ξ_i+ρ_i,  i∈𝒩,

where ω_ij>((N-1)/2)(|ρ̇_i|_∞+|ρ̇_j|_∞) is not a fixed but a state-based gain. The initial conditions satisfy ∑_j∈𝒩ξ_i(0)=0 and ξ_i(0) is symmetric. Similar to the proof in <cit.>, z_i can be shown to converge to ∑_i∈𝒩ρ_i(x_i,t)/N in a finite time. The next corollary extends Theorem <ref> to address the distributed optimization problem with  more general cost functions (<ref>).


Suppose Assumptions <ref>, <ref>, <ref> hold. For system (<ref>) and (<ref>)-(<ref>) with Algorithm <ref>, let h_0=γμ_1/N and k>(h_2+μ_4)N√(m)/μ_1^2+ϵ_4 with 0<γ<1 and ϵ_4>0. Then, the optimization problem (<ref>) is solved, i.e., lim_t→∞x_i(t)-r^*(t)=0, i∈𝒩.

 
See Appendix <ref>. 


    
    Different from the estimator (<ref>), the implementation of estimator (<ref>) does not require a known upper bound on |ρ̇_i|_∞, but at the expense of requiring additional exchange of information |ρ̇_i|_∞ with neighboring agents. 















§ SIMULATIONS



Example 1. Consider 20 agents interacting over a circular communication network. With the decision variable y=(y_1,y_2,y_3)∈ℝ^3, each agent is assigned with a local cost function. For agent i=1,⋯,6, f_i=∑_j=1^32y_j^2 +0.1isin (t) y_j. For agent i=7,⋯,12, f_i=(y_1+cos(t)y_2+0.1iy_3)^2+cos(t)y_1+0.1iy_2+y_3. The agent i=13,⋯,20 is assigned with f_13=∑_j=1^3exp(-t)y_j^2, f_14=∑_j=1^3sin(t) y_j^2, f_15=∑_j=1^3-(y_j-2)^2, f_16=∑_j=1^3sin(y_j), f_17=∑_j=1^3(tanh(t)+1)ln(1+exp(y_j)), f_18=∑_j=1^3y_j/(y_j^2+1), f_19=∑_j=1^3-(1/(t+1))exp(-y_j^2) and f_20=∑_j=1^3y_j^2+exp(-t)ln(1+y_j^2), respectively, where f_19 is inspired by the density function of normal distribution, and f_17 and f_20 are adapted from <cit.>. 

In fact, except for f_15 and f_17, all local cost functions have their own global minimum. However, only f_1-f_6 and f_20 are uniformly strongly convex. f_13 and f_17 are uniformly strictly but not strongly convex, f_7-f_12 are just uniformly convex, while f_14, f_15, f_16, f_18 and f_19 are even non-convex. 






It can be checked that Assumptions <ref>, <ref>, <ref> are satisfied. By applying the distributed algorithm in Corollary <ref> with h_0=0.5 and k=35, the optimization problem (<ref>) is solved. With x_i corresponding to the decision variable y of agent i, Fig. <ref> shows the trajectory of |x_i-r^*(t)| that converges to zero asymptotically. 



Example 2. Consider a scenario in Fig. <ref>, where N=6 UAVs are used as relay nodes for a communication system supporting long distance robots collaboration. The signal power of robot i∈𝒩 is modeled by <cit.> as g_i(x_i,t)=p_i(t)/|x_i-d_i(t)|^2, x_i≠ d_i(t), in a 2D scenario, where p_i(t)>0 and d_i(t)∈ℝ^2 denotes the signal transmission power and the position of robot i, respectively. The communication among these robots is assisted by UAVs with relay capabilities<cit.>. Meanwhile, the UAVs are required to form a formation to expand communication coverage. 
To establish reliable communication links, all UAVs need to search for positions as close as possible to the corresponding robots. This leads to the optimization problem:

    min_x_1,⋯,x_N∑_ i∈𝒩f_i(x_i,t),    s.t.  x_i-x_j=τ_i-τ_j,

where f_i(x_i,t)=g_i^-1(x_i,t) and τ_i∈ℝ^2 denotes a desired formation. Each UAV i can obtain the real-time parameters p_i(t) and d_i(t) of robot i to construct f_i(x_i,t). The optimal solution of this problem, x^*_i=r^*+τ_i, i∈𝒩, can be regarded as the optimal relay position of UAV i, where r^* is the optimal solution of min∑_ i∈𝒩f_i(r+τ_i,t). Moreover, the following two practical cases are considered:

    
  * The signal transmission power of a robot falls below p_0,
    
  * The attacker's actions damage a robot at time instant t_0.

In both cases, the robots lose the ability to transmit their parameters p_i(t) and d_i(t) to UAVs, preventing the UAVs from constructing f_i(x_i,t). To simulate it, the corresponding local cost functions are set to zero[To ensure almost everywhere differentiability of parameters, a low-pass filter is used to attenuate the coefficient 1/p_i to 0, thereby attenuating the cost function to 0. Although there are non-differentiable points, the proof in this note still holds without using the nonsmooth analysis (Remark <ref>).], which is non-strongly convex.
 
 
As an example, suppose that the parameters of power model (SI units are used here) are given by p_1=p_2=10, p_3=10exp(-0.05t), p_4=8, p_5=10/(1+0.06t), p_6=6+sin(0.3t), d_1=(30,30)^⊤, d_2=(30+10cos(0.3t),0)^⊤, d_3=(-30,-30)^⊤, d_4=(-30,-30+10sin(0.3t))^⊤, d_5=(-30,0)^⊤ and d_6=(-30,30)^⊤. Let p_0=2 and t_0=67s. Following the above problem setting, p_3<p_0 if t≥ 32s. Assume that robot 5 is damaged by the attacker at t=t_0. The desired formation is given by a regular hexagon as τ_1=(5,10)^⊤, τ_2=(10,0)^⊤, τ_3=(5,-10)^⊤, τ_4=(-5,10)^⊤, τ_5=(-10,0)^⊤ and τ_6=(-5,10)^⊤. The initial states are given by x_1(0)=(-5,10)^⊤, x_2(0)=(-15,10)^⊤, x_3(0)=(-25,10)^⊤, x_4(0)=(-25,0)^⊤, x_5(0)=(-15,0)^⊤ and x_6(0)=(-5,0)^⊤. A circular communication network 𝒢 is also used for UAVs. One may check that Assumptions <ref>-<ref> are satisfied. Then, each UAV modeled as a single-integrator dynamics is driven by the proposed distributed algorithm in Theorem <ref> with h_0=0.1, ω=1 and k=20 to search the optimal relay position. Fig. <ref> shows the trajectories of x_i, i∈𝒩, r^*(t) and the snapshots of them at t=40s, 90s, 140s. Fig. <ref> shows the trajectory of |x_i-τ_i-r^*(t)|, i∈𝒩. It can be seen that even if f_3=0 when t≥ 32s and f_5=0 when t≥ 67s, the asymptotical convergence of x_i to τ_i+r^*(t) still be achieved.





§ CONCLUSION


This note proposes a systematical design incorporating average estimator, DZA and adaptive optimizer to solve DTQO. Its advantages include not relying on the strong convexity of local cost functions, identical or time-invariant Hessians, waiting time during algorithm implementation, or certain prior knowledge of the communication graph and global parameter information of cost functions, while guaranteeing closed-loop convergence in the global sense. The result is also extended to a class of non-quadratic cost functions. This advancement enables large-scale multi-robot systems to achieve autonomous optimization more flexibly in dynamic environments. 







 §.§  Two Auxiliary Lemmas


    <cit.>: Let A,B∈ℝ^m× m be Hermitian and let |·|_u be an unitarily invariant norm on ℝ^m× m. Then, |λ^↓(A)-λ^↓(B)|_u≤|A-B|_u, where λ^↓(A)(λ^↓(B)) is the diagonal matrix whose diagonal entries are the nonincreasingly ordered eigenvalues of A(B).



    Suppose Assumptions <ref>-<ref> hold. For system (<ref>)-(<ref>) with Algorithm <ref>, the state x_i(t) does not have a finite escape time, i.e., for any t̅∈[0,∞)  the state x_i(t) is bounded for all t∈[0,t̅]. 
    
        
    
    Denote x_ip∈ℝ the p-th row element of x_i, where i∈𝒩 and p∈ℳ. Similar subscripts also appear in certain matrices that follow. We take x_ip as an example to check its boundedness in a finite time. The dynamics of x_ip is given by
    
    ẋ_ip=   -kH_ipx_i-kR_ip-ẑ_ip^-1Ḣ_ix_i-ẑ_ip^-1Ṙ_i
       +c_1(t)∑_j∈𝒩_iS(x_ip-x_jp)+W_1+W_2,

    where 0≤ c_1(t)=∫_0^tmϵ_1 η_t dt=(mϵ_1/ϵ_2)(1-η_t)<∞, W_1=-∑_j∈𝒩_i(∫_0^t |x_i-x_j|^2  dt) (x_ip-x_jp), and W_2=-∑_j∈𝒩_i(∫_0^t|x_i-x_j|_1 dt)S(x_ip-x_jp). We first consider the first half of (<ref>), i.e.
    
    ẋ_ip=   -kH_ipx_i-kR_ip-ẑ_ip^-1Ḣ_ix_i-ẑ_ip^-1Ṙ_i
       +c_1(t)∑_j∈𝒩_iS(x_ip-x_jp)=:g_i(x_ip,t).

    For all t∈[0,t̅], we know g_i(x_ip,t) is piecewise continuous with respect to t. With Assumptions <ref>-<ref> satisfied, the boundedness of H_i, Ḣ_i, ẑ_i^-1 can be guaranteed, which implies that g_i(x_ip,t) is global Lipschitz with respect to x_ip. Then, by <cit.>, we know (<ref>) has a unique solution over [0,t̅]. In view of the fact that t̅ can be arbitrarily large, we make the following statement,
    
        
  S1. x_ip in system (<ref>) does not have a finite escape time, ∀ i∈𝒩 and ∀ p∈ℳ.
    
    We further consider W_1 and W_2 in system (<ref>). Assume that i^*,p^*∈max_i∈𝒩,p∈ℳ|x_ip|. Without loss of generality, let x_i^*p^*>0. If x_i^*=x_j, ∀ j∈𝒩_i^*, we have W_1=W_2=0. If there exists at least one j∈𝒩_i^* such that x_i^*≠ x_j, it follows that W_1,W_2≤ 0. Similarly, assume that x_i^*p^*<0. If x_i^*=x_j, ∀ j∈𝒩_i^*, we have W_1=W_2=0. If there exists at least one j∈𝒩_i^* such that x_i^*≠ x_j, then we have W_1,W_2≥0. Therefore, we make another statement,
    
        
  S2. W_1+W_2≤0 if x_i^*p^*>0 and W_1+W_2≥0 if x_i^*p^*<0.
       
    
By combining statements S1 and S2, we conclude from (<ref>) that the trajectory of x_i^*p^* does not escape to infinity at a finite time. With the fact that i^*,p^*∈max_i∈𝒩,p∈ℳ|x_ip|, thus, x_i(t) does not have a finite escape time, ∀ i∈𝒩.
 



 §.§ Proof of Lemma <ref>

Define the average of all Hessians, H̅(t)=H(t)/N. By the definition of δ_i below Lemma <ref>, we have δ_i=z_i(t)-H(t)/N=z_i(t)-H̅(t). Next, we will establish the relation between |ẑ_i-H̅| and |δ_i| in the following two cases:

(1) The non-singular case of matrix z_i(t). Recall Algorithm <ref>. For all t≥ 0, Once min_p∈ℳ|λ_p(z_i(t))|≥ h_0, we have z_i(t)=ẑ_i(t), implying |ẑ_i(t)-H̅|=|δ_i|.

(2) The (near-)singular case of matrix z_i(t).
For all t≥ 0, if min_p∈ℳ|λ_p(z_i(t))|<h_0, we have

    min_p∈ℳ|λ_p(z_i(t))|<h_0=γ h_1/N< λ_1(H̅(t)),

where the last inequality follows the fact that λ_1(H̅(t))≥ h_1/N given by Assumption <ref>. Before establish the relation between |ẑ_i-H̅| and |δ_i|, we first show the symmetry of z_i. Rewrite (<ref>) as ż_i=-ω∑_j∈𝒩_i (z_i-z_j)+Ḣ_i(t). Since H_i(t) is symmetric, Ḣ_i(t) must also be symmetric, ∀ t≥0. Symmetric z_i(0) indicates that z_i(0) is symmetric as well. It is clear that the function (·) does not change the symmetry of its matrix variable. Thus, z_i is always symmetric. By Lemma <ref> along with (<ref>), we conclude that |δ_i|≥ (1-γ)h_1/N>0. Consequently, we can always find a positive constant k^s_i such that |ẑ_i(t)-H̅|≤ k^s_i|δ_i|.

Now, based on the analysis of the above two case, we have
|ẑ_i(t)-H̅|≤max{1,k^s_i}|δ_i|=k_1|δ_i|, ∀ t≥ 0, where k_1=max_i∈𝒩{1,k^s_i}. This means |ẑ_i-H̅| can always be linearly bounded by |δ_i|, whether z_i(t) is singular or not.





 §.§ Proof of Theorem <ref>

Recall the definition of δ_i in Appendix <ref>, and let δ=(δ_1,⋯,δ_N)^⊤. The convergence analysis of the δ subsystem has been established by <cit.> and is thus omitted here.
The following proof focuses on the analysis of e and x̃ subsystems, and the comprehensive analysis of the closed-loop system.

(1) Analysis of e subsystem. Consider the function

    V_1=e^⊤e+1/2∑_i∈𝒩∑_j∈𝒩_i(α_ij-α̅)^2+1/2∑_i∈𝒩∑_j∈𝒩_i(β_ij-β̅)^2,

where α̅ and β̅ are positive constants to be determined. Its derivative along (<ref>) and (<ref>) is given by

    V̇_1=   ∑_i∈𝒩∑_j∈𝒩_i(α_ij-α̅)α̇_ij+∑_i∈𝒩∑_j∈𝒩_i(β_ij-β̅)β̇_ij
       -2e^⊤L̅_⊗e+Λ_3+Λ_4,

where Λ_3=-2e^⊤(DB)_⊗S(D^⊤_⊗e) and Λ_4=2e^⊤M_⊗ϕ. Note

    Λ_3=   -2∑_i∈𝒩∑_j∈𝒩_iβ_ije_i^⊤S(e_i-e_j)
    
    =   -∑_i∈𝒩∑_j∈𝒩_iβ_ij(e_i-e_j)^⊤S(e_i-e_j)
    ≤    -∑_i∈𝒩∑_j∈𝒩_iβ_ij(|e_i-e_j|_1-mϵ_1 η_t),

where the second equality follows from Assumption <ref> and and third one follows the fact that β_ij≥0 and yS(y)≥|y|-ϵ_1 η_t with y∈ℝ. By (<ref>), we obtain |ϕ|≤ϕ_1|x|+ϕ_2 with ϕ_1=|{kH_i+ẑ_i^-1Ḣ_i}|, ϕ_2=|((kR_1+ẑ_1^-1Ṙ_1)^⊤,⋯,(kR_N+ẑ_N^-1Ṙ_N)^⊤)^⊤|. The boundedness of H_i, Ḣ_i, ẑ_i^-1, R_i and Ṙ_i implies the boundedness of ϕ_1 and ϕ_2. By denoting m̅=|M_⊗|, we have

    Λ_4≤   2m̅ϕ_1|e|^2+2m̅ϕ_1√(N)|e||x̃|+2m̅(ϕ_1√(N)|r^*|+ϕ_2)|e|
    ≤   b_1|e|^2+σ_1h_1/N|x̃|^2+2m̅(ϕ_1√(N)|r^*|+ϕ_2)|e|,

where σ_1>0, b_1=2m̅ϕ_1+m̅^2ϕ_1^2N^2/(σ_1h_1), the first inequality is obtained by (<ref>), and the second inequality follows from Young's inequality <cit.>. By Assumption <ref> again, we have

    ∑_i∈𝒩∑_j∈𝒩_i(α_ij-α̅)α̇_ij=   ∑_i∈𝒩∑_j∈𝒩_i(α_ij-α̅)|e_i-e_j|^2
    
    =   2e^⊤(L̅-α̅L)_⊗e.

Substituting (<ref>), (<ref>), (<ref>) and (<ref>) into (<ref>) yields

    V̇_1≤    -2α̅e^⊤L_⊗e+b_1|e|^2-β̅∑_i∈𝒩∑_j∈𝒩_i|e_i-e_j|_1+σ_1h_1/N|x̃|^2
       +2m̅(ϕ_1√(N)|r^*|+ϕ_2)|e|+mϵ_1 N^2β̅η_t.

With the fact that e^⊤(1_N⊗ I_m)≡ 0, we conclude from <cit.> that -e^⊤L_⊗e≤-λ_2(L)|e|^2. It can be shown that

    -∑_i∈𝒩∑_j∈𝒩_i|e_i-e_j|_1
    
    =   -2|D^⊤_⊗e|_1≤-2√(e^⊤(DD^⊤)_⊗e)≤-2√(λ_2(L))|e|,

where the last inequality follows from L=DD^⊤. By selecting β̅=m̅(ϕ_1√(N)|r^*|+ϕ_2)/√(λ_2(L)), we have

    V̇_1≤    -2λ_2(L)α̅|e|^2+b_1|e|^2+σ_1h_1/N|x̃|^2+mϵ_1 N^2β̅η_t.


(2) Analysis of x̃ subsystem. Inspired by the form of ẑ^-1_i∇_t f_i(x_i,t), adding ±H^-1∑_i∈𝒩∇_t f_i(x_i,t) in (<ref>) yields

    ẋ̃̇=   -k/N1^⊤_N∇ F(x,t)+Λ_1+Λ_2,

where Λ_1=H^-1∑_i∈𝒩∇_t f_i(r^*,t)-∇_t f_i(x_i,t) and Λ_2=∑_i∈𝒩(H̅^-1-ẑ^-1_i)∇_t f_i(x_i,t)/N. The latter follows from H^-1=H̅^-1/N. We find that Λ_1 can be bounded by e and x̃, i.e.,


    Λ_1=-H^-1∑_i∈𝒩Ḣ_i(e_i+x̃)≤h_2√(mN)/h_1|e+1_N⊗x̃|.

Note that H̅^-1-ẑ^-1_i=H̅^-1(ẑ_i-H̅)ẑ^-1_i. By Lemma <ref>, along with the boundedness of H̅^-1 and ẑ^-1_i, there must be a positive constant k_2=k_1|H̅^-1| max_i∈𝒩{|ẑ^-1_i|} such that

    Λ_2≤1/N∑_i∈𝒩k_2|δ_i||∇_t f_i(x_i,t)|≤1/Nk_3|δ||x|+1/√(N)k_4|δ|,

where k_3=k_2max_i∈𝒩{|Ḣ_i|}, k_4=k_2max_i∈𝒩{|Ṙ_i|} and the second inequality is obtained by |∇_t f_i(x_i,t)|≤|Ḣ_i||x_i|+|Ṙ_i|. Consider a Lyapunov function candidate V_2=|x̃|^2/2. Thus,

    V̇_2=   -k/Nx̃^⊤1^⊤_N∇ F(x,t)+x̃^⊤Λ_2+x̃^⊤Λ_1
    
    =   -k/Nx̃^⊤1^⊤_N∇ F(1_N⊗(x̃+r^*),t)+x̃^⊤Λ_2-k/Nx̃^⊤1^⊤_N
       ×(∇ F(x,t)-∇ F(1_N⊗(x̃+r^*),t))+x̃^⊤Λ_1.

Invoking Assumption <ref>, f(r(t),t) is uniformly h_1-strongly convex with respect to r(t). This implies

    -k/Nx̃^⊤1^⊤_N∇ F(1_N⊗(x̃+r^*),t)
    ≤   -k/Nx̃^⊤(∇ f(x̃+r^*,t)-∇ f(r^*,t))≤-kh_1/N|x̃|^2.

By Assumption <ref>, ∇ f_i(x_i,t) is uniformly θ_i-Lipschitz with θ_i=min_t≥0|H_i(t)|. By denoting θ=max_i∈𝒩{θ_i}, we have 

    -k/Nx̃^⊤1^⊤_N(∇ F(x,t)-∇ F(1_N⊗(x̃+r^*),t))
    ≤   kθ/√(N)|x̃||e|≤σ_2h_1/N|x̃|^2+b_2|e|^2,

where σ_2>0 and b_2=k^2θ^2/(4σ_2h_1). We also conclude that

    x̃^⊤Λ_2≤   1/Nk_3|x̃||δ||x|+1/√(N)k_4|x̃||δ|
    ≤   k_3/√(N)|δ||x̃|^2+k_3|δ|/N|x̃||e|+k_3|r^*|+k_4/√(N)|x̃||δ|
    ≤   b_3|δ||x̃|^2+(σ_3+σ_4)h_1/N|x̃|^2+(b_4|e|^2+b_5)|δ|^2,

where σ_3,σ_4>0, b_3=k_3/√(N), b_4=k_3^2/(4σ_3h_1 N), b_5=(k_3|r^*|+k_4)^2/(4σ_4h_1). With a positive constant σ_5, we have 

    x̃^⊤Λ_1≤σ_5h_1/N|x̃|^2+b_6|x̃|^2+b_7|e|^2,

where b_6=h_2N√(m)/h_1 and b_7=mN^2h_2^2/(4σ_5h_1^3). Substituting (<ref>), (<ref>), (<ref>) and (<ref>) into (<ref>) yields

    V̇_2≤   (∑_i=2^5σ_i-k)h_1/N|x̃|^2+b_6|x̃|^2+b_3|δ||x̃|^2
       +(b_2+b_7)|e|^2+(b_4|e|^2+b_5)|δ|^2.


(3) Closed-loop synthesis. Given V_1 and V_2, we further consider the composite Lyapunov function candidate of e and x̃ subsystems, i.e., V=V_1+V_2. Combining (<ref>) and (<ref>) yields

    V̇=   V̇_1+V̇_2
    ≤-l_1|x̃|^2-l_2|e|^2+W_3+mϵ_1 N^2β̅η_t,

where l_1=(k-∑_i=1^5σ_i)h_1/N-b_6, l_2=2λ_2(L)α̅-b_1-b_2-b_7 and W_3=b_3|x̃|^2|δ|+b_4|e|^2|δ|^2+b_5|δ|^2. Then, one may first choose k>b_6N/h_1+ϵ_3, where ϵ_3>0 such that l_1>0. By choosing α̅>(b_1+b_2+b_7)/(2λ_2(L)), we have l_2>0. Integrating both sides of (<ref>) yields

    V(t)+l_1∫_0^∞ |x̃|^2  dt+l_2∫_0^∞ |e|^2  dt-∫_0^∞ W_3 dt
    ≤    V(0)+mϵ_1N^2β̅/c <∞.

We next give the boundedness analysis of ∫_0^∞ W_3 dt. By Lemma <ref>, x_i is bounded in a finite time, and so is e. With the fact that r^* is bounded concluded by Assumption <ref>, we know x̃ is also bounded in a finite time by (<ref>). Moreover, by <cit.>, there must be a finite time instant T such that for any 0≤ t<T, δ is bounded, and for any t≥ T, δ_i=z_i(t)-H̅(t)=0, ∀ i∈𝒩. Thus, it is clear that for any 0≤ t<T, W_3 is bounded, which together with the continuity of e, x̃ and δ given by (<ref>), (<ref>) and (<ref>), respectively, results in that ∫_0^T W_3 dt is bounded. Note that ∫_0^∞ W_3 dt=∫_0^T W_3 dt+∫_T^∞ W_3 dt=∫_0^T W_3 dt, implying the boundedness of ∫_0^∞ W_3 dt. By recalling (<ref>), we know V is bounded, and e,x̃∈ℒ_2. It follows from V_1 and V_2 that e, x̃, α_ij and β_ij are bounded, and thus x is bounded. This implies the boundedness of ∇ f_i(x_i,t) and ∇_t f_i(x_i,t). Then, by (<ref>) and (<ref>), ė and ẋ̃̇ are bounded. By Barbalat's Lemma <cit.>, we have lim_t→∞e(t)=0 and lim_t→∞x̃(t)=0, which further implies lim_t→∞x_i(t)=r^*(t).



 §.§  Reanalysis of Finite Escape Time for Corollary <ref>



    Suppose Assumptions <ref>, <ref>, <ref> hold. For system (<ref>) and (<ref>)-(<ref>) with Algorithm <ref>, the state x_i(t) does not have a finite escape time, i.e., for any t̅∈[0,∞)  the state x_i(t) is bounded for all t∈[0,t̅]. 
    
        
    
    Denote x_ip∈ℝ the p-th row element of x_i, where i∈𝒩 and p∈ℳ. Similar subscripts also appear in certain matrices that follow. We take x_ip as an example to check its boundedness in a finite time. According to Assumption <ref>.(<ref>), without loss of generality, let ∇f̂_i(x_i,t)=Y(x_i,t)+Z(x_i,t) which satisfies |Y(x_i,t)|≤μ_2|x_i| and |Z(x_i,t)|≤μ_3. Then the dynamics of x_ip is given by
    
    ẋ_ip=   -k(H_ipx_i+R_ip+Y_ip+Z_ip)
       -ẑ_ip^-1(Ḣ_ix_i+Ṙ_i+g_ix_i+s_i)
       +c_1(t)∑_j∈𝒩_iS(x_ip-x_jp)+W_1+W_2,

    where 0≤ c_1(t)=∫_0^tmϵ_1 η_t dt=(mϵ_1/ϵ_2)(1-η_t)<∞, W_1=-∑_j∈𝒩_i(∫_0^t |x_i-x_j|^2  dt) (x_ip-x_jp), and W_2=-∑_j∈𝒩_i(∫_0^t|x_i-x_j|_1 dt)S(x_ip-x_jp). We first consider the first half of (<ref>), i.e.
    
    ẋ_ip=   -k(H_ipx_i+R_ip+Y_ip+Z_ip)
       -ẑ_ip^-1(Ḣ_ix_i+Ṙ_i+g_ix_i+s_i)
       +c_1(t)∑_j∈𝒩_iS(x_ip-x_jp)=:g_i(x_ip,t).

    For all t∈[0,t̅], we know g_i(x_ip,t) is piecewise continuous with respect to t. With Assumptions <ref>, <ref>, <ref> satisfied, we know g_i(x_ip,t) is global Lipschitz with respect to x_ip. Then, by <cit.>, we know (<ref>) has a unique solution over [0,t̅]. In view of the fact that t̅ can be arbitrarily large, we make the following statement,
    
        
  S1. x_ip in system (<ref>) does not have a finite escape time, ∀ i∈𝒩 and ∀ p∈ℳ.
    
    We further consider W_1 and W_2 in system (<ref>). Assume that i^*,p^*∈max_i∈𝒩,p∈ℳ|x_ip|. Without loss of generality, let x_i^*p^*>0. If x_i^*=x_j, ∀ j∈𝒩_i^*, we have W_1=W_2=0. If there exists at least one j∈𝒩_i^* such that x_i^*≠ x_j, it follows that W_1,W_2≤ 0. Similarly, assume that x_i^*p^*<0. If x_i^*=x_j, ∀ j∈𝒩_i^*, we have W_1=W_2=0. If there exists at least one j∈𝒩_i^* such that x_i^*≠ x_j, then we have W_1,W_2≥0. Therefore, we make another statement,
    
        
  S2. W_1+W_2≤0 if x_i^*p^*>0 and W_1+W_2≥0 if x_i^*p^*<0.
       
    
By combining statements S1 and S2, we conclude from (<ref>) that the trajectory of x_i^*p^* does not escape to infinity at a finite time. With the fact that i^*,p^*∈max_i∈𝒩,p∈ℳ|x_ip|, thus, x_i(t) does not have a finite escape time, ∀ i∈𝒩.
 



 §.§ Proof of Corollary <ref>

Define a diagonal matrix Ω, whose order is equal to the number of edges in the graph 𝒢. The nodes corresponding to non-zero elements in column i of matrix D are denoted as i_1 and i_2, respectively. The diagonal element Ω_ii of Ω is defined as ω_i_1i_2. Then, by defining z=(z_1,⋯,z_N) and  ρ=(ρ_1,⋯,ρ_2), system (<ref>) can be rewritten as ż=-(DΩ)_⊗(D^⊤_⊗z)+ρ̇. Define consensus error δ̅=M_⊗z. Then, the dynamics of δ subsystem can be written as

    δ̇̅̇=   -(DΩ)_⊗(D^⊤_⊗δ̅)+M_⊗ρ̇.

(1) Analysis of δ̅ subsystem. 
Then, consider the Lyapunov function candidate 
W=(δ̅^⊤δ̅). Its derivative is given by

    Ẇ=   -2(δ̅^⊤(DΩ)_⊗(D^⊤_⊗δ̅))+2(δ̅^⊤M_⊗ρ̇).

We first analyze its last term. It can be concluded that

    2(δ̅^⊤M_⊗ρ̇)=   1/N∑_i=1^N∑_j=1^N(δ̅_i-δ̅_j)^⊤(ρ̇_i-ρ̇_j)
    ≤   1/N∑_i=1^N∑_j=1^N|δ̅_i-δ̅_j|_1|ρ̇_i-ρ̇_j|_∞,

which is obtained by Hölder's inequality <cit.>. By triangular inequality |ρ̇_i-ρ̇_j|_∞≤|ρ̇_i|_∞+|ρ̇_j|_∞, we have

    2(δ̅^⊤M_⊗ρ̇)≤   1/N∑_i=1^N∑_j=1^N(|ρ̇_i|_∞+|ρ̇_j|_∞)|δ̅_i-δ̅_j|_1
    ≤   max_i{∑_j=1,j≠ i^N(|ρ̇_i|_∞+|ρ̇_j|_∞)|δ̅_i-δ̅_j|_1}
    ≤   N-1/2∑_i∈𝒩∑_j∈𝒩_i(|ρ̇_i|_∞+|ρ̇_j|_∞)|δ̅_i-δ̅_j|_1,

where the last inequality is obtained by using Assumption <ref>. Then, with the similar analysis in (<ref>), we have 

    -2(δ̅^⊤(DΩ)_⊗(D^⊤_⊗δ̅))=-∑_i∈𝒩∑_j∈𝒩_iω_ij||δ̅_i-δ̅_j||_1.

Substituting (<ref>) and (<ref>) into (<ref>) yields

    Ẇ≤   ∑_i∈𝒩∑_j∈𝒩_i(N-1/2(χ̅_i+χ̅_j)-ω_ij)||δ̅_i-δ̅_j||_1.

With the similar analysis in (<ref>), if we select ω_ij>((N-1)/2)(|ρ̇_i|_∞+|ρ̇_j|_∞), we have Ẇ≤-2√(λ_2)W^1/2. According to <cit.>, there exists a finite time T_1 such that for all t≥ T_1, δ̅(t)=0, i.e., z_i(t)=z_j(t), ∀ i,j∈𝒩. Moreover, by using Assumption <ref>, we have

    1^⊤_Nξ̇=-(1^⊤_NDΩ)_⊗(D^⊤_⊗z)=0.

By recalling the initial condition ∑_i∈𝒩ξ_i(0)=0, we have ∑_i∈𝒩ξ_i(t)=0. Combining (<ref>) yields

    ∑_i∈𝒩z_i(t)=∑_i∈𝒩ρ_i,   ∀ t≥ 0.

With the fact that z_i(t)=z_j(t), ∀ i,j∈𝒩, ∀ t≥ T_1, we thus have z_i=ρ̅:=∑_i∈𝒩ρ_i(x_i,t)/N, ∀ t≥ T_1.

Define z̃=1^⊤_Nz/N-ρ̅. Then we have

    z=δ̅+1_N⊗z̃+1_N⊗ρ̅.
 
With the fact that 1^⊤_Nδ̅≡ 0, we have 

    ∑_i∈𝒩z_i(t)=Nz̃+∑_i∈𝒩ρ_i.

By recalling (<ref>), we know that z̃≡0. This means that δ̅_i=z_i-ρ̅=:δ_i. Therefore, the finite-time convergence of δ̅_i is equivalent to the finite-time convergence of δ_i, which will be used for the following analysis.

(2) Analysis of e subsystem. Consider the function

    V_1=e^⊤e+1/2∑_i∈𝒩∑_j∈𝒩_i(α_ij-α̅)^2+1/2∑_i∈𝒩∑_j∈𝒩_i(β_ij-β̅)^2,

where α̅ and β̅ are positive constants to be determined. Its derivative along (<ref>) and (<ref>) is given by

    V̇_1=   ∑_i∈𝒩∑_j∈𝒩_i(α_ij-α̅)α̇_ij+∑_i∈𝒩∑_j∈𝒩_i(β_ij-β̅)β̇_ij
       -2e^⊤L̅_⊗e+Λ_3+Λ_4,

where Λ_3=-2e^⊤(DB)_⊗S(D^⊤_⊗e) and Λ_4=2e^⊤M_⊗ϕ. Note

    Λ_3=   -2∑_i∈𝒩∑_j∈𝒩_iβ_ije_i^⊤S(e_i-e_j)
    
    =   -∑_i∈𝒩∑_j∈𝒩_iβ_ij(e_i-e_j)^⊤S(e_i-e_j)
    ≤    -∑_i∈𝒩∑_j∈𝒩_iβ_ij(|e_i-e_j|_1-mϵ_1 η_t),

where the second equality follows from Assumption <ref> and and third one follows the fact that β_ij≥0 and yS(y)≥|y|-ϵ_1 η_t with y∈ℝ. By (<ref>), we obtain |ϕ|≤ϕ_1|x|+ϕ_2 with 

    ϕ_1=   |{k(H_i+μ_2I_m)+ẑ_i^-1(Ḣ_i+μ_4I_m)}|,
    ϕ_2=   |[ [ k(R_1+μ_31_m)+ẑ_1^-1(Ṙ_1+|s(x_1,t)|1_m);                                       ⋮; k(R_N+μ_31_m)+ẑ_N^-1(Ṙ_N+|s(x_N,t)|1_m);                                         ]]|.

The boundedness of H_i, Ḣ_i, ẑ_i^-1, R_i, Ṙ_i and |s(x_i,t)| implies the boundedness of ϕ_1 and ϕ_2. With m̅:=|M_⊗|, we have

    Λ_4≤   2m̅ϕ_1|e|^2+2m̅ϕ_1√(N)|e||x̃|+2m̅(ϕ_1√(N)|r^*|+ϕ_2)|e|
    ≤   b_1|e|^2+σ_1μ_1/N|x̃|^2+2m̅(ϕ_1√(N)|r^*|+ϕ_2)|e|,

where σ_1>0, b_1=2m̅ϕ_1+m̅^2ϕ_1^2N^2/(σ_1μ_1), the first inequality is obtained by (<ref>), and the second inequality follows from Young's inequality <cit.>. By Assumption <ref> again, we have

    ∑_i∈𝒩∑_j∈𝒩_i(α_ij-α̅)α̇_ij=   ∑_i∈𝒩∑_j∈𝒩_i(α_ij-α̅)|e_i-e_j|^2
    
    =   2e^⊤(L̅-α̅L)_⊗e.

Substituting (<ref>), (<ref>), (<ref>) and (<ref>) into (<ref>) yields

    V̇_1≤    -2α̅e^⊤L_⊗e+b_1|e|^2-β̅∑_i∈𝒩∑_j∈𝒩_i|e_i-e_j|_1+σ_1μ_1/N|x̃|^2
       +2m̅(ϕ_1√(N)|r^*|+ϕ_2)|e|+mϵ_1 N^2β̅η_t.

With the fact that e^⊤(1_N⊗ I_m)≡ 0, we conclude from <cit.> that -e^⊤L_⊗e≤-λ_2(L)|e|^2. It can be shown that

    -∑_i∈𝒩∑_j∈𝒩_i|e_i-e_j|_1
    
    =   -2|D^⊤_⊗e|_1≤-2√(e^⊤(DD^⊤)_⊗e)≤-2√(λ_2(L))|e|,

where the last inequality follows from L=DD^⊤. By selecting β̅=m̅(ϕ_1√(N)|r^*|+ϕ_2)/√(λ_2(L)), we have

    V̇_1≤    -2λ_2(L)α̅|e|^2+b_1|e|^2+σ_1μ_1/N|x̃|^2+mϵ_1 N^2β̅η_t.


(3) Analysis of x̃ subsystem. Denote ρ=∑_i∈𝒩ρ_i(x_i,t). Here we have ṙ^*=-ρ^-1(t)∑_i∈𝒩∇_t f_i(r^*,t). Then, adding ±ρ^-1∑_i∈𝒩∇_t f_i(x_i,t) in (<ref>) yields

    ẋ̃̇=   -k/N1^⊤_N∇ F(x,t)+Λ_1+Λ_2,

where Λ_1=ρ^-1∑_i∈𝒩∇_t f_i(r^*,t)-∇_t f_i(x_i,t) and Λ_2=∑_i∈𝒩(ρ̅^-1-ẑ^-1_i)∇_t f_i(x_i,t)/N. The latter follows from ρ^-1=ρ̅^-1/N. We conclude that

    Λ_1=   -ρ^-1∑_i∈𝒩(Ḣ_i+g_i)(e_i+x̃)+s_i(x_i,t)-s_i(r^*,t)
    ≤   (h_2+μ_4)√(mN)/μ_1|e+1_N⊗x̃|+2Nμ_5/μ_1s̅(t).

Note that ρ̅^-1-ẑ^-1_i=ρ̅^-1(ẑ_i-ρ̅)ẑ^-1_i. By using the similar analysis in Lemma <ref>, we have |ẑ_i(t)-ρ̅|≤ k_1|δ_i|, ∀ t≥ 0. Along with the boundedness of ρ̅^-1 and ẑ^-1_i, there must be a positive constant k_2=k_1|ρ̅^-1| max_i∈𝒩{|ẑ^-1_i|} such that

    Λ_2≤1/N∑_i∈𝒩k_2|δ_i||∇_t f_i(x_i,t)|≤1/Nk_3|δ||x|+1/√(N)k_4|δ|,

where k_3=k_2max_i∈𝒩{|Ḣ_i|+|g_i|}, k_4=k_2max_i∈𝒩{|Ṙ_i|+|s_i|} and the second inequality is obtained by 

    |∇_t f_i(x_i,t)|≤(|Ḣ_i|+|g_i|)|x_i|+|Ṙ_i|+|s_i|.

Consider a Lyapunov function candidate V_2=|x̃|^2/2. Thus,

    V̇_2=   -k/Nx̃^⊤1^⊤_N∇ F(x,t)+x̃^⊤Λ_2+x̃^⊤Λ_1
    
    =   -k/Nx̃^⊤1^⊤_N∇ F(1_N⊗(x̃+r^*),t)+x̃^⊤Λ_2-k/Nx̃^⊤1^⊤_N
       ×(∇ F(x,t)-∇ F(1_N⊗(x̃+r^*),t))+x̃^⊤Λ_1.

Invoking Assumption <ref>.(<ref>), f(r(t),t) is uniformly μ_1-strongly convex with respect to r(t). This implies

    -k/Nx̃^⊤1^⊤_N∇ F(1_N⊗(x̃+r^*),t)
    ≤   -k/Nx̃^⊤(∇ f(x̃+r^*,t)-∇ f(r^*,t))≤-kμ_1/N|x̃|^2.

Moreover, ∇ f_i(x_i,t) is uniformly θ_i-Lipschitz with θ_i=min_t≥0|ρ_i(x_i,t)|. By denoting θ=max_i∈𝒩{θ_i}, we have 

    -k/Nx̃^⊤1^⊤_N(∇ F(x,t)-∇ F(1_N⊗(x̃+r^*),t))
    ≤   kθ/√(N)|x̃||e|≤σ_2μ_1/N|x̃|^2+b_2|e|^2,

where σ_2>0 and b_2=k^2θ^2/(4σ_2μ_1). We also conclude that

    x̃^⊤Λ_2≤   1/Nk_3|x̃||δ||x|+1/√(N)k_4|x̃||δ|
    ≤   k_3/√(N)|δ||x̃|^2+k_3|δ|/N|x̃||e|+k_3|r^*|+k_4/√(N)|x̃||δ|
    ≤   b_3|δ||x̃|^2+(σ_3+σ_4)μ_1/N|x̃|^2+(b_4|e|^2+b_5)|δ|^2,

where σ_3,σ_4>0, b_3=k_3/√(N), b_4=k_3^2/(4σ_3μ_1 N), b_5=(k_3|r^*|+k_4)^2/(4σ_4μ_1). With σ_5,σ_6>0, we have 

    x̃^⊤Λ_1≤σ_5μ_1/N|x̃|^2+b_6|x̃|^2+b_7|e|^2+σ_6μ_1/N|x̃|^2+b_8s̅^2(t),

where b_6=(h_2+μ_4)N√(m)/μ_1, b_7=mN^2(h_2+μ_4)^2/(4σ_5μ_1^3) and b_8=2N^2μ_5/(4σ_6μ_1^2). Substituting (<ref>), (<ref>), (<ref>) and (<ref>) into (<ref>) yields

    V̇_2≤   (∑_i=2^6σ_i-k)μ_1/N|x̃|^2+b_6|x̃|^2+b_3|δ||x̃|^2
       +(b_2+b_7)|e|^2+(b_4|e|^2+b_5)|δ|^2+b_8s̅^2(t).


(4) Closed-loop synthesis. Given V_1 and V_2, we further consider the composite Lyapunov function candidate of e and x̃ subsystems, i.e., V=V_1+V_2. Combining (<ref>) and (<ref>) yields

    V̇=   V̇_1+V̇_2
    ≤-l_1|x̃|^2-l_2|e|^2+b_8s̅^2+W_3+mϵ_1 N^2β̅η_t,

where l_1=(k-∑_i=1^6σ_i)μ_1/N-b_6, l_2=2λ_2(L)α̅-b_1-b_2-b_7 and W_3=b_3|x̃|^2|δ|+b_4|e|^2|δ|^2+b_5|δ|^2. Then, one may first choose k>b_6N/μ_1+ϵ_4, where ϵ_4>0 such that l_1>0 and  choose α̅>(b_1+b_2+b_7)/(2λ_2(L)) such that l_2>0. Integrating both sides of (<ref>) yields

    V(t)+l_1∫_0^∞ |x̃|^2  dt+l_2∫_0^∞ |e|^2  dt
       -b_8∫_0^∞s̅^2(t) dt -∫_0^∞ W_3 dt≤ V(0)+mϵ_1N^2β̅/c <∞.

According to Assumption <ref>.(<ref>), we know ∫_0^∞s̅^2(t) dt is bounded. We next give the boundedness analysis of ∫_0^∞ W_3 dt. By Lemma <ref>, x_i is bounded in a finite time, and so is e. With the fact that r^* is bounded concluded by Assumption <ref>, we know x̃ is also bounded in a finite time by (<ref>). Moreover, according to (1) Analysis of δ subsystem, there exies a finite time instant T_1 such that for any 0≤ t<T_1, δ is bounded, and for any t≥ T_1, δ_i=z_i(t)-ρ̅(t)=0, ∀ i∈𝒩. Thus, it is clear that for any 0≤ t<T_1, W_3 is bounded, which together with the continuity of e, x̃ and δ given by (<ref>), (<ref>) and (<ref>), respectively, results in that ∫_0^T_1 W_3 dt is bounded. Note that ∫_0^∞ W_3 dt=∫_0^T_1 W_3 dt+∫_T_1^∞ W_3 dt=∫_0^T_1 W_3 dt, implying the boundedness of ∫_0^∞ W_3 dt. By recalling (<ref>), we know V is bounded, and e,x̃∈ℒ_2. It follows from V_1 and V_2 that e, x̃, α_ij and β_ij are bounded, and thus x is bounded. This implies the boundedness of ∇ f_i(x_i,t) and ∇_t f_i(x_i,t). Then, by (<ref>) and (<ref>), ė and ẋ̃̇ are bounded. By Barbalat's Lemma <cit.>, we have lim_t→∞e(t)=0 and lim_t→∞x̃(t)=0, which further implies lim_t→∞x_i(t)=r^*(t).


IEEEtran




 Paper Title: Assessing the Reliability Benefits of Energy Storage as a Transmission Asset.








Assessing the Reliability Benefits of Energy Storage as a Transmission Asset
    David Sehloff, Member, IEEE,
Jonghwan Kwon, Member, IEEE,
Mahdi Mehrtash, Senior Member, IEEE,

Todd Levin, Member, IEEE,
Benjamin F. Hobbs, Life Fellow, IEEE

    August 5, 2024
===================================================================================================================================================================



§ ABSTRACT 
Utilizing energy storage solutions to reduce the need for traditional transmission investments has been recognized by system planners and supported by federal policies in recent years. This work demonstrates the need for detailed reliability assessment for quantitative comparison of the reliability benefits of energy storage and traditional transmission investments. First, a mixed-integer linear programming expansion planning model considering candidate transmission lines and storage technologies is solved to find the least-cost investment decisions. Next, operations under the resulting system configuration are simulated in a probabilistic reliability assessment which accounts for weather-dependent forced outages. The outcome of this work, when applied to TPPs, is to further equalize the consideration of energy storage compared to traditional transmission assets by capturing the value of storage for system reliability.



Transmission expansion planning, joint resource and transmission expansion, co-optimization, energy storage as transmission, reliability




§ INTRODUCTION

New electricity transmission infrastructure and energy storage installations are recognized as key enablers of meeting decarbonization goals while maintaining the reliability of power systems. Traditionally, the services these assets can provide have been treated as distinct, but recently, energy storage has been considered as a potential source of transmission services. Storage operated as a transmission asset may offer preferable capital costs and construction timelines compared to traditional transmission solutions, while bringing unique advantages for supporting the system and enhancing both reliability and resilience.

The development of regulatory guidelines for this application of energy storage is evolving in the United States. In 2017, the Federal Energy Regulatory Commission (FERC) issued a policy statement allowing dual use of energy storage assets, that is, cost recovery through both the regulated transmission planning practices (TPPs) and participation in competitive markets <cit.>. Between 2020 and 2023, FERC issued orders accepting proposals from three independent system operators defining a new class of asset, ”Storage as Transmission-Only Asset” (SATOA). A storage installation of this type may be selected through TPPs provided that it addresses a specific transmission system need and follows other requirements ensuring it provides only transmission services <cit.>. As a result, the Midcontinent Independent System Operator considered a SATOA in its TPP and identified it as superior to a transmission line, or wires, solution, leading to its selection and plans for permitting and construction <cit.>. Despite this movement toward consideration of energy storage for transmission services, existing TPPs have barriers and disincentives which make them inadequate for evaluation of energy storage solutions <cit.>, one of which is evaluation of the reliability contributions of energy storage compared with traditional wires solutions.

The co-optimization of transmission and energy storage planning has been assessed under various distinct perspectives. One common perspective is the centralized co-planning of transmission and storage to avoid the need for new transmission installations, thereby reducing investment costs <cit.>. Variations of this perspective include incorporation of security constraints and high renewable penetration <cit.>. Konstantelos and Strbac included additional flexible network and non-network solutions in this centralized planning approach and demonstrated the value of these solutions in reducing risk under uncertain evolution of system capacity <cit.>. Go et al. took the perspective of co-planning generation, transmission, and energy storage, showing the benefits of a co-optimization approach to reduce investment costs while meeting renewable portfolio standard constraints <cit.>. Dvorkin et al. considered transmission and energy storage co-planning from the perspective of a merchant energy storage owner in a multi-level framework finding maximum energy storage system profit with centralized least-cost transmission expansion decisions <cit.>. Aside from these approaches considering avoided costs and profits, Zach and Auer compared transmission and energy storage investments with a welfare maximization approach <cit.>. Kwon et al. introduced a framework for optimal sizing and siting of energy storage as a transmission asset in order to satisfy specific needs identified in TPPs <cit.>. However, previous studies primarily focus on the economic value of transmission and storage investments, lacking consideration of the reliability contributions of these investments. 

Many approaches for assessing the reliability of power systems have been applied to transmission expansion planning. The expected unserved energy (EUE) is commonly used to capture the energy shortfall, for example in <cit.>, and  metrics such as loss of load expectation (LOLE) and loss of load hours (LOLH) can provide additional insights into the temporal nature of shortfalls <cit.>. Similar probabilistic metrics have been introduced, such as loss of load cost (LOLC) <cit.>. An alternative to probabilistic reliability metrics is to ensure robustness to a defined set of transmission and generation contingencies through N-1 <cit.> or N-k<cit.> security constraints. The 2022 IEEE Composite Power System Reliability report <cit.> highlights the need to develop composite system reliability methods which incorporate storage, common and dependent mode failure models, and weather conditions into composite system reliability analysis. Sundar et al. <cit.> applied temperature-dependent forced outage rates to natural gas combined cycle and hydropower plants for resource adequacy assessment, not considering the transmission network. Other studies including <cit.> have considered uncertainty from renewable generation and included representation of the transmission system, but they do not model weather-dependent or correlated generation outages. 

Means for assessing the reliability value of energy storage have been a subject of much recent work, as briefly reviewed in <cit.>. Stephen et al. demonstrated that the approach by which storage dispatch is modeled can have a large impact on resource adequacy metrics <cit.>, particularly whether system reliability (as in <cit.>) or economic performance (as in <cit.>) is the assumed dispatch objective. However, these studies focus on resource adequacy with limited or no modeling of the transmission system and do not take into account the concept of SATOA in assessing the reliability values.

This study considers the viewpoint of a system operator co-optimizing transmission and energy storage for least-cost investment and operations and introduces a framework to quantify the reliability contribution that energy storage as a transmission asset provides at a system level and to compare these benefits with those of transmission upgrades.

This study is the continuation of the conference paper <cit.>, which was presented at the Texas Power and Energy Conference (TPEC 2023). The main contribution of  <cit.> was to show the advantage of joint resource and transmission expansion planning in comparison with the resource-only expansion planning and project-by-project screening of expansion plans considering different future scenarios representing uncertainties that exist in emission reduction policies and forecasted demand. This article significantly extends the authors’ previous work  <cit.> by conducting a comparative analysis of reliability outcomes stemming from various investment strategies: focusing solely on transmission, solely on storage, or permitting investments in both. The aim is to gain comprehensive insights into the reliability contributions offered by SATOA and traditional transmission lines.

The contributions of this paper are the following:

    
  * An analysis framework that supports the assessment of the economic and reliability value of both SATOA and traditional transmission assets. 
    
  * A composite reliability assessment model considering weather-dependent correlated generator outages, pre- and post-contingency operations including storage as a transmission only asset, and the transmission network represented with a DC power flow model
    
  * A case study demonstrating this framework for a representative high-renewable, limited-transmission system, including findings highlighting the importance of capturing reliability contributions of transmission assets including storage in transmission planning, demonstrating that SATOA can offer comparable benefits to transmission line investments.


The remainder of the paper is organized as follows. Section <ref> presents the methodology for the transmission and energy storage expansion planning and the probabilistic reliability assessment and their coupling, Section <ref> presents a case study on a simplified ERCOT-like system, and Section <ref> concludes.



§ METHODS

This section presents an overview of the two-stage analysis framework, as shown in Fig. 1, employed in this study. In the first stage, the Johns Hopkins Stochastic Multi-stage Integrated Network Expansion (JHSMINE) model was utilized for optimal planning of transmission and storage expansion.
In the second stage, the expansion decisions derived from JHSMINE are passed to the Argonne Low-carbon Electricity Analysis Framework (A-LEAF) for a comprehensive probabilistic reliability assessment. 





 §.§ Transmission and Storage Expansion

JHSMINE is a cost-minimizing capacity expansion planning tool that considers several significant aspects of the expansion planning models such as unit commitment, linearized power flow, renewable energy policies and energy trading credits, etc., and has been used in several projects <cit.>. In this paper, JHSMINE is used to find expansion planning solutions with three different cases. In the first case, only the transmission investment option is active. In the second case, storage investment is the only available option. Lastly, in the third case, both transmission and storage investments are considered as investment options. It is intuitive that conducting joint storage and transmission expansion planning reduces costs relative to the cases that allow either transmission-only or storage-only expansion planning. However, assessing all three of these cases allows us to  evaluate the contribution of each individual technology to system reliability. Although JHSMINE is capable of running a stochastic multistage expansion planning model; here we apply its deterministic expansion planning mode for a single planning horizon.

The objective function of JHSMINE is to minimize the probability-weighted of total system cost as represented by (<ref>). The objective function consists of two terms: investment cost and operation cost, and both are realized at the scenario tree node (s,y), where s and y are indices for scenarios and years, respectively. JHSMINE discounts the investment cost back to the beginning of the planning horizon using D_y, where δ is the interest rate. JHSMINE assumes that the operation condition (e.g., load, policies, fuel price, etc.) of operation node will repeat until the next operation simulation interval.  All cash flows for operations are assumed to be end-of-year flows, and the discounting formulas are defined accordingly as DA_y.




    min   obj=∑_(s,y)SP_s,y(D_y· invc_s,y+DA_y· oprc_s,y),   
         D_y=1/(1+δ)^y-y_origin,
         DA_y=D_y· P(1,Y_y,δ)=D_y(∑_t=1^Y_y1/(1+δ)^t)



JHSMINE calculates the investment cost of new facilities using (<ref>), which includes the expansion costs of facilities that are newly expanded and the salvage revenue for facilities that are newly retired. 




    invc_s,y   =GEXC_s,y,k· gincexp_s,y,k
       -∑_(s^',y^')GSAL_s,y,k·gincret_s,y,k+LEXC_s,y,l·lincexp_s,y,l
       -∑_(s^',y^')LSAL_s,y,l·lincret_s,y,l+EEXC_s,y,j· eincexp_s,y,j
       -∑_(s^',y^')ESAL_s,y,j·eincret_s,y,j



JHSMINE calculates the operation cost using (<ref>), which is composed of seven terms: the fixed operation and maintenance (O&M) cost of generation and storage facilities represented by (<ref>); the variable O&M cost of generation and storage operations modeled by (<ref>); the fuel cost of generation and storage operations shown in (<ref>); the start-up cost from generation unit commitment represented by (<ref>); and finally the value of lost load modeled by (<ref>).




    oprc_s,y=fomc_s,y+vomc_s,y+stuc_s,y+voll_s,y
       fomc_s,y=∑_kGFOM_k· GNPL_k· gstat_s,y,k
       +∑_jEFOM_j· EGCP_j· estat_s,y,j
       vomc_s,y=∑_rp,hRPW_y,rp(∑_kGVOM_k·g o p t_s,y,rp,h,k
       +∑_jEVOM_j· e d i s_s,y,rp,h,j)
       fuelc_s,y=∑_rp,hRPW_y,rp[∑_fFC_s,y,f,rp,h
       ·(∑_k∈ K_fGHR_k· g o p t_s,y,rp,h,k+∑_j∈ J_fEHR_j· e d i s_s,y,rp,h,j)]
       stuc_s,y=∑_rp,hRPW_y,rp
       ·(∑_kGSUC_k· G N P L_k· g s u p_s,y,rp,h,k)
       voll_s,y=∑_rp,hRPW_y,rp· V O L L(∑_inload_s,y,rp,h,i)



The feasibility space of JHSMINE is constructed by a set of equality constraints and a set of inequality constraints. Among the equality constraints are nodal power balances in the system, DC power flow representation, energy balances for storage, etc. The set of inequality constraints includes thermal limits of transmission lines, output power limit of generating units, energy storage representations, etc. [29], [30].
The unit commitment constraints in JHSMINE are expanded based on the “Tight Relaxed Unit Commitment” (TRUC) Constraints in <cit.>, For classic unit commitment without generation expansion, readers are referred to <cit.> and <cit.>. Under TRUC, unit commitment variables, i.e., operating status, start-up, and shut-down variables, can be relaxed; in the meanwhile, each unit commitment constraint is still physically meaningful. For the power flow representation, a disjunctive version of the DC Optimal Power Flow (DCOPF) is modeled in JHSMINE.
The detailed model formulations of JHSMINE, including the set of constraints, can be found in <cit.>.



 §.§ Probabilistic Reliability Assessment

A-LEAF is an integrated power system simulation framework that includes a suite of generation and transmission capacity expansion, unit commitment, and system reliability assessment models[<https://www.anl.gov/esia/a-leaf>]. In this paper, we employ the probabilistic reliability assessment model in A-LEAF, which conducts Monte Carlo simulations to quantify the risk of unserved energy (i.e., the system’s reliability level) and calculates various reliability metrics.  Furthermore, the reliability assessment model was enhanced to specifically consider the operation of SATOA.

Inputs to the reliability assessment model include the generation portfolio, generator and storage characteristics, transmission topology, and time series data for load, temperature, and variable renewable energy availability. The model first generates outage scenarios for each generating asset in the system using a two-state Markov model. This model captures the dynamic evolution of individual unit-level outage states over time. During each distinct time period, the availability state of each generating asset is stochastically determined, taking into account the unit's state transition probabilities. In this study, we utilized weather-dependent state transition probabilities that are calculated using the methodology outlined in <cit.>. After generating outage scenarios, the model conducts a Monte Carlo simulation with a two-stage structure, as shown in Fig <ref>. In the first stage, the model solves an economic dispatch problem for each operating day to obtain steady-state dispatch setpoints for all the resources. The model does not dispatch the SATOA resources in the first stage because, by definition, these resources should maintain their full state-of-charge (SOC) levels to be used when the system has contingencies. In the second stage, for each outage scenario, the model solves a post-contingency system re-dispatch problem to assess the system’s ability to handle generator outages and determine expected unserved energy. The transition between the steady-state dispatch schedule and the post-contingency redispatch is constrained by both available headroom and the ramp rate of individual assets. In addition, the second stage simulates the post-contingency cases in a myopic way, meaning that it performs hour-by-hour simulations without anticipating future outages in the redispatch algorithm. The outcomes from all trials are then used to compute reliability metrics, such as EUE and LOLH. In this paper, we present the second stage model formulation in (9)-(28). The second stage model solves the following optimization problem for each time period during a post-contingency event.




    min     ∑_it(MC_i(d_it^up+d_it^dn)+∑_nt(VOLL·ens_nt))




    -(∑_k(n,) f_kt-∑_k(,n) f_kt)+  ∑_i,Bus(i)=n g_it  -    ∑_i,Bus(i)=n,i∈Ψ_ESchg_it
       +ens_nt-D_nt=0,     ∀ n∈ N






    f_kt   =∑_nPTDF_kn· p_nt^inj, 	   ∀ k∈ K
    
    p_nt^inj   =∑_i,Bus(i)=n g_it -  ∑_i,Bus(i)=n,i∈Ψ_ES(chg_it +ens_nt-D_nt), 	   ∀ n∈ N
    
    -F_k^+   ≤ f_kt≤ F_k^+, 	   ∀ k∈ K






    g_it≤ S_itCAP_i,	   ∀ i∈ I∈Ψ_VRE^FIX
       0≤soc_it≤ SOC_i^+,	   ∀ i∈ I∈Ψ_ES
       soc_it-S̅O̅C̅_i,t-1-ϵ_i^Cchg_it+1/ϵ_i^Dg_it=0,	   ∀ i∈ I∈Ψ_ES




    g_it=G̅_it+d_it^up-d_it^dn   ∀ i∈ I
       ∑_iC_cit(g_it-chg_it)-∑_iC_cit(G̅_it-C̅H̅G̅_it)
       ≤∑_i(1-C_cit)(G̅_it-C̅H̅G̅_it)
       g_it≤ C_citCAP_i	   ∀ i∈ I
       chg_it≤ C_citCAP_i	   ∀ i∈ I
       d_it^up≤CAP_iR_i^10   ∀ i∈ I
       d_it^dn≤CAP_iR_i^10   ∀ i∈ I
       chg_it-C̅H̅G̅_it≤CAP_iR_i^10   ∀ i∈ I
       -chg_it+C̅H̅G̅_it≤CAP_iR_i^10   ∀ i∈ I
       g_it-G̅_it-1≤CAP_iR_i^H	   ∀ i∈ I
       G̅_it-1-g_it≤CAP_iR_i^H	   ∀ i∈ I
       chg_it-C̅H̅G̅_it-1≤CAP_iR_i^H	   ∀ i∈ I
       C̅H̅G̅_it-1-chg_it≤CAP_iR_i^H	   ∀ i∈ I



The objective function (<ref>) minimizes the cost of load shedding and the redispatch of generators. System operating constraints are enforced through the system balance equation (<ref>) and PTDF-based power flow constraints (<ref>-<ref>). The scheduling of variable renewable energy assets is bounded by the hourly generation profiles in (<ref>). The model tracks the SOC of energy storage assets with upper and lower bounds in (<ref>-<ref>). The generation redispatch in each hour of a post-contingency event is constrained by equations (<ref>-<ref>). Specifically, generation redispatch is represented as deviations from the steady-state dispatch setpoint in (<ref>). Constraint (<ref>) ensures that the total amount of generation redispatch is less than the total generation lost, considering the potential of load shedding. Asset availability during the post-contingency is captured in (<ref>) and (<ref>) by the generator availability status, C_cit, which has a value of one if an asset is available. The deviations from the steady-state setpoints are restricted by ramp rate constraints in (<ref>-<ref>). Lastly, the inter-temporal constraints between the current and previous periods are enforced by equations (<ref>-<ref>).





§ CASE STUDY

This section presents a case study demonstrating the framework on an aggregated version of the Texas power system.


 §.§ Description

The network used in this case study is a simplified representation of the 
Electric Reliability Council of Texas (ERCOT) system, aggregated to seven zones as shown in Fig. <ref>. These zones are balancing areas defined in <cit.>, from which we source our zonal load and wind and solar resource data. Table <ref> gives the non-coincident peak values for each zone.



Generation units and parameters are also taken from <cit.>, which is based on published data for real units. Each generation unit in the system is assigned to a zone but still modeled individually in both the simulation of outages and the optimization of  dispatch. Table <ref> gives the total capacity by zone and by resource type. The type “NG” includes natural gas simple cycle combustion turbine and combined cycle plants, and “OGS” represents any oil or gas steam plants.




The transmission network used in this case study is an aggregated, synthetic network that is based on the existing transmission lines in the ERCOT system. The network is constructed by estimation of topology and line parameters using publicly available geographic data from the Homeland Infrastructure Foundation-Level Data <cit.> and aggregated to a zonal DC power flow model using the methods introduced in <cit.>. Each inter-zonal link is represented by its reactance and thermal capacity, as shown in Table <ref>. 





 §.§ Transmission and Storage Expansion

The JHSMINE transmission and storage expansion model is applied to this system to find optimal expansion decisions for each of three cases: transmission expansion (TEP), battery storage expansion (BEP), and joint  battery storage and transmission expansion (BTEP). The single peak load hour of each day is considered, for a total of 365 unique operating conditions. Due to this daily resolution, generator ramp constraints and battery charging and discharging ramp constraints are not modeled in the expansion problem, but they are considered in the detailed reliability assessment. The TEP problem represents a TPP scenario in which the interface between Zone 3 and 7 has been identified as a limiting constraint, and planners seek the optimal sizing of the upgrade based on a least-cost expansion formulation. This link may be upgraded to increase its thermal capacity at a cost of 1.6 thousand USD per MW-mile, with the MW increase modeled as a continuous variable between 0 and 3,000 MW. The BEP problem represents an alternative approach which considers energy storage as candidate upgrades throughout the system. We consider a generic four-hour duration battery energy storage system (BESS) that can be installed in any zone in 400 MW increments up to 1,200 MW total, with an investment cost of 0.35 million USD per MWh. The BTEP problem represents a combination of the TEP and BEP, allowing co-optimization of link 3-7 upgrade and storage investments.

Table <ref> shows the optimal expansion results of each problem. The TEP resulted in an upgrade to increase the capacity of this link by 1,495 MW, at a cost of 80.46 million USD, resulting in very large operational cost savings. The reason for this is that the alleviation of this severe transmission constraint allows the renewable production in Zone 7 to be sent to other zones, significantly reducing curtailment of these resources. No storage solution in this example was able to offer this benefit, or any operational benefit outweighing the investment cost, as the results of the BEP show no storage investments. The BTEP result further shows that no storage investment resulted in a net benefit, even if coupled with the transmission upgrade. It is important to note that no load shedding is observed throughout the 365 periods of operation in the expansion analysis with JHSMINE.





 §.§ Assessing Reliability Benefits of Potential Expansion Decisions

Next, the reliability is assessed using A-LEAF. To better understand the reliability contributions associated with SATOA additions, ten candidate storage installations are examined in Zone 3, including two- and four-hour systems with capacities ranging from 300 MW to 1500 MW. To assess the reliability, we first generate a set of 1,000 random generator outage scenarios for each hour of the year, then we simulate operations under each of the configurations for each outage scenario. Finally, for each configuration, we compute the EUE and LOLH from the outcomes of all scenarios. The reliability assessment model simulates system dispatch across all 8,760 hourly periods in the year. The inclusion of these operational details and contingency modeling enables the reliability outcomes under each configuration to be captured with greater accuracy.

In contrast to the expansion model, unserved load is observed under each configuration. Fig. <ref> shows the EUE by zone for each configuration. The TEP result gives a significant improvement compared to the reference case. The energy storage investments demonstrate improvements which increase with storage capacity, from a slight improvement over the reference case with a 2 hour, 300 MW system, to a large improvement with a 4 hour, 1500 MW system. The storage investments do not outperform the transmission expansion planning results, however. Table <ref> shows the EUE values for the reference, TEP, and three of the storage configurations in greater detail. The values in the table are in MWh, followed by the relative change compared to the reference case as a percentage in parentheses. In the reference and storage cases, the EUE in Zone 3 is the largest of any zone. The TEP result completely eliminates the unserved energy in Zone 3, though several other zones experience increases in EUE. This demonstrates that transmission upgrades may lead to significant changes in the patterns of generation dispatch and power flow through a network, potentially influencing the distribution of reliability throughout the network.






Similarly, Fig. <ref> shows the LOLH by zone for each configuration, and  Table <ref> gives additional detail. The TEP again gives the best performance under this metric. The storage solutions result in relative improvements in LOLH which are quite similar to their improvements in EUE. The TEP solution also results in similar relative improvements in LOLH as it does in EUE. 













The lost load events which make up the annual metrics in Tables <ref> and <ref> occur on a relatively small number of days. This can be seen in Fig. <ref>, which shows the EUE by zone in the TEP, BEP, and Reference cases on the four days with the highest lost load.  August 2nd and 1st experience the most severe shortfalls, accounting for 53% of the yearly EUE in the reference case. The behavior exhibits a similar pattern across the four days.





Fig. <ref> shows the hourly dispatch by generation type for one sample outage scenario on August 2nd. The rows of the figure make up the results for all the zones, and the columns represent the results under the Reference, BEP in Zone 3, and TEP configurations, respectively. The realization of the stochastic characteristics in this single sample are identical between the three configurations; any differences across the three columns are only the result of the storage or transmission additions. Overlaid on each plot are the hourly demand profiles in black, along with a red area representing any unserved energy in the zone. The system outcomes shown in Fig. <ref>, though only representing a single sample on a single day (Aug. 2nd) are illustrative of several observed across reliability metrics. In the storage configuration, the new storage installed in Zone 3 is discharged in hours 12 through 16, contributing to a decrease in unserved energy in this zone. The reduced curtailment of wind and photovoltaic generation in Zone 7 can be seen in the TEP solution, as well as the unserved load event that occurs in hour 15 in that zone.







 §.§ Cost and Reliability

In order to compare the reliability outcomes between the expansion configurations, we compute the cost of reduction of both expected unserved energy and loss of load hours by dividing the annual investment costs shown in <ref> by the reduction in each metric compared with the reference case. Fig. <ref> shows these costs for the storage configurations in Zone 3 and for the TEP configuration. The TEP configuration offers the lowest cost of reliability improvements in both EUE and LOLH. The costs of reliability improvements under the storage options are slightly higher in this case. The smaller storage units, however, give more comparable costs of improvement and offer lower capital costs, potentially allowing more to be placed throughout the network in situations where more distributed application is advantageous. Note that this comparison does not include other values, such as operational cost savings, that could result from these investments. Operational savings may often present an advantage of transmission investments compared to SATOA investments, since SATOA must reserve its capacity for essential transmission services.









§ CONCLUSIONS

This paper presents a two-stage analysis framework to quantify the reliability benefits of transmission assets, including SATOA. The framework consists of an optimal storage and transmission expansion planning methodology formulated as a mixed integer linear program using a DC power flow representation, along with a probabilistic reliability assessment methodology that simulates system operations under stochastic generator outage scenarios.

This framework is used to demonstrate that transmission and storage each may be identified as appropriate solutions for different aspects of reliability needs. Traditional least cost expansion tools provide valuable insights but may not capture factors which influence reliability, such as correlated generator outages, and they need to be coupled or integrated with detailed reliability assessment to quantitatively evaluate of the reliability benefits of storage as a transmission asset. The case study presented here demonstrated that the upgrade of a severely constrained transmission corridor connecting a zone with excess renewable generation to a high-load zone with poor reliability was optimal from a least-cost expansion perspective and also provided significant reliability benefits. Storage as a transmission asset was not able to outperform the reliability contributions of the transmission upgrade in this scenario, but large storage installations achieved nearly comparable improvements of 71.7% in EUE and 68.8% in LOLH, compared to the TEP solution's 84.2% and 75.8%, respectively. In addition, smaller storage additions showed capital costs per unit improvement of 25.1 thousand USD/MWh EUE reduction and 21.9 million USD/h LOLH reduction, compared to 16.3 thousand USD/MWh and 17.3 million USD/h, respectively, under the TEP solution. Given the lower total cost and potentially greater feasibility of installation, storage solutions for incremental improvements may offer advantages in certain cases. This underscores the need for planning tools which accurately assess both the economic and reliability contributions of both traditional transmission and storage as transmission only solutions for quantitative comparisons of which solution best meets the planning needs at hand.

This work also points to several important future directions. The inability of SATOA to bring operational savings due to its exclusion from market participation may be a barrier to its adoption. The reliability implications of allowing partial market participation should be studied. In the case study, the aggregated network obscures detailed power flow constraints and therefore potentially underestimates transmission congestion and its reliability impacts. A finer network resolution would more accurately capture whether the reliability benefits of storage are primarily realized locally or alternatively whether they can be allocated across the system via existing transmission infrastructure. Incorporating a forced outage model for transmission lines would also improve the characterization of the reliability benefits of each transmission asset. Future work includes the application of the framework with more detailed transmission network models to reflect these complex effects of congestion as well as the incorporation of models of weather-dependent transmission outage behavior. Additional future work will examine the participation of energy storage both as a transmission and market asset, identifying operational constraints which can enable this dual use while maintaining reliability benefits.



§ NOMENCLATURE

tocsectionNomenclature


 §.§ JHSMINE Sets and Indices

[G]

  F Fuel types, index f.

  G Generation technologies, index g.

  H	Hours within a representative period (e.g., 24 hours), index h.

  I	Buses, index i.

  J	Energy storages, index j.

  K	Generators, index k.

  L	Transmission lines, index l.

  RP	Representative periods, index rp.

  S	Scenarios, index s.

  T	Hours of a planning period (e.g., 8760 hours), index t.

  W	States/Provinces, index w.

  Y	Years, index y.



 §.§ JHSMINE Parameters

[CTAXs,y,w]

  CTAX_s,y,w Carbon price or tax of state w in the scenario tree node (s,y), unit $/ton CO2e.

  D_y	            Discounting factor of year y to the beginning of the planning horizon.

  DA_y	            Accumulative discounting factor of year y to the beginning of the planning horizon. 

  EER_j 	        Emission rate of the storage facility j, unit ton CO2e/MWh.

  EEXC_s,y,j 	    Expansion cost of the storage facility j in scenario tree node (s,y), unit M$. 

  EFOM_j	        Fixed O&M cost of the storage j, unit $/MW-year.

  EHR_j 	        Heat rate of the storage j, unit MMBTU/MWh.

  ESAL_s,y,j,s^',y^' 	Salvage revenue of storage facility j if expanded in the scenario tree node (s^',y^') and retired in node (s,y), unit M$.

  EVOM_j 	        Variable O&M cost of the storage j, unit $/MWh.

  FC_s,y,rp,h,f	    Price of the fuel f at hour h of representative period rp in scenario tree node (s,y), unit $/MMBTU.

  GER_k	            Emission rate of the generator k, unit tonCO2e/MWh.

  GEXC_s,y,k	    The expansion cost of the generator k in scenario tree node (s,y), unit M$.

  GFOM_k 	        Fixed operating and maintenance cost of the generator k, unit $/MW-year.

  GHR_k	            Average heat rate of generator k, unit MMBTU/MWh.

  GNPL_k	        Nameplate capacity of the generator k, unit MW. 

  GSAL_s,y,k,s^',y^'	The salvage revenue of the generator k if it is expanded in scenario tree node (s^',y^') and retired in the node (s,y), unit M$.

  GSUC_k	        Start-up cost of generator k per unit of the capacity, unit $/MW.

  GVOM_k	        Variable operating and maintenance cost of the generator k, unit $/MWh.

  LEXC_s,y,l 	    Expansion cost of the transmission line l in the scenario tree node (s,y), unit M$.

  LSAL_s,y,l,s^',y^' 	Salvage revenue of the transmission line l if it is built in the scenario tree node (s^',y^') and retired in the node (s,y), unit M$.

  RACP_w 	        Alternative compliance penalty for RPS of state w, unit $/MWh 

  RPW_y,rp	        # of periods represented by representative period rp in year y.

  SP_s,y	        Scenario probability, unitless.

  VOLL	            Value of lost load, unit $/MWh.



 §.§ JHSMINE Variables



  §.§.§ Expansion and Retirement

[CTAXs,y,w]

  eincexp_s,y,j Storage incremental expansion, 1 if the storage j becomes commissioned in (s,y), binary, unitless.

  eincret_s,y,j	Storage incremental retirement, 1 if the storage j is decommissioned in (s,y), binary, unitless.

  gincexp_s,y,k 	Generator incremental expansion, 1 if the generator k becomes commissioned in (s,y), binary, unitless.

  gincret_s,y,k	Generator incremental retirement, 1 if the generator k is decommissioned in (s,y), binary, unitless.

  lincexp_s,y,l 	Transmission line incremental expansion, 1 if the transmission line l becomes commissioned in (s,y), binary, unitless.

  lincret_s,y,l	Transmission line incremental retirement, 1 if the transmission line l is decommissioned in (s,y), binary, unitless.



  §.§.§ Operations

[CTAXs,y,w]

  edis_s,y,rp,h,j Discharge of the storage j at the hour h, nonnegative, unit MW.


  gopt_s,y,rp,h,k 	Power output of the generator k at the hour h, nonnegative, unit MW.

  gsup_s,y,rp,h,k 	Start-up action of the generator k at the beginning of the hour h, binary, unitless.  

  nload_s,y,rp,h,i 	Load shedding at bus i at the hour h, nonnegative, unit MW.

  nrps_s,y,rp,h,w 	Non-compliance with RPS policy, unit MW, nonnegative.



  §.§.§ Objective Function

[fomc_s,y]

  obj Objective function, unit $.

  invc_s,y	Investment cost occurs at the scenario tree node (s,y), unit $.

  oprc_s,y	Operation cost occurs at the scenario tree node (s,y), unit $.

  fomc_s,y	Fixed O&M cost occurs at the scenario tree node (s,y), unit $.

  fuelc_s,y	Fuel cost occurs at the scenario tree node (s,y), unit $.

  vomc_s,y	Variable cost occurs at the scenario tree node (s,y), unit $.

  stuc_s,y	Start-up cost occurs at the scenario tree node (s,y), unit $.

  ctax_s,y	Carbon tax payment occurs at the scenario tree node (s,y), unit $.

  voll_s,y	Lost load cost occurs at the scenario tree node (s,y), unit $.

  rpsc_s,y	Cost of renewable portfolio standards non-compliance penalty, occurs at the scenario tree node (s,y), unit $.



 §.§ A-LEAF Sets and Indices

[Ψ_VRE^FIX∈ I]

  d∈ D	Set of representative day groups.

  i∈ I	Set of generator technologies.

  h∈ H	Set of hour periods.

  t∈ T	Set of sub-hour periods.

  c∈ C	Set of risk scenario.

  k∈ K	Set of transmission lines.

  n∈ N	Set of buses.

  Ψ_ES∈ I	Set of energy storage generators.

  Ψ_VRE^FIX∈ I	Set of variable renewable energy generators with fixed generation profiles



 §.§ A-LEAF Parameters

[C̅H̅G̅_it]

  C_cit	Generator availability status (1: available, 0: unavailable)

  CAP_i	Power capacity, index i, unit MW

  C̅H̅G̅_it	Steady-state energy storage charging, unit MW

  D_nt	Power demand, indices n,t, unit MW

  ϵ_i^C	Energy storage charging efficiency, index i

  ϵ_i^D	Energy storage discharging efficiency, index i

  F_k^+	Transmission capacity, index k, unit MW

  G̅_it	Steady-state generator dispatch setpoint, indices i,t, unit MW

  MC_i	Marginal variable cost, index i, unit $

  PTDF_kn	Power transfer distribution factor, indices k,n

  R_i^10	10 mins ramp capacity, index i, unit MW

  R_i^H	Hourly ramp capability, index i, unit MW

  S_it	Renewable generation profile, indices i,t, unit MW

  SOC_i^+	Maximum state-of-charge, index i

  VOLL	Value of lost load, unit $/MWh.




 §.§ A-LEAF Variables

[p_nt^inj]

  chg_it	Storage charging, indices i,t, unit MW

  ens_nt	Energy not served, indices n,t, unit MWh

  f_kt	Power flow, indices k,t, unit MW

  g_it	Power output, indices i,t, unit MW

  p_nt^inj	Net power injection, indices n,t, unit MW

  d_it^up	Generator redispatch up, indices i,t, unit MW

  d_it^dn	Generator redispatch down, indices i,t, unit MW





IEEEtran




 Paper Title: Impact of Geographical Separation on Spectrum Sharing Markets.








Impact of Geographical Separation on Spectrum Sharing Markets


    
Kangle Mu, Zongyun Xie, Igor Kadota, and Randall Berry
Department of Electrical and Computer Engineering

    Northwestern University

 
    {kangle.mu, zongyun.xie, kadota, rberry}@northwestern.edu


    August 5, 2024
=============================================================================================================================================================================================================



§ ABSTRACT 
With the increasing demand for wireless services, spectrum management agencies and service providers (SPs) are seeking more flexible mechanisms for spectrum sharing to accommodate this growth. Such mechanisms impact the market dynamics of competitive SPs. Prior market models of spectrum sharing largely focus on scenarios where competing SPs had identical coverage areas.  We depart from this and consider a scenario in which two competing SPs have overlapping but distinct coverage areas. We study the resulting competition using a Cournot model.
 Our findings reveal that with limited shared bandwidth, SPs might avoid overlapping areas to prevent potential losses due to interference. Sometimes SPs can strategically cooperate by agreeing not to provide service in the overlapping areas and, surprisingly, customers might also benefit from such cooperation under certain circumstances. Overall, market outcomes exhibit complex behaviors that are influenced by the sizes of coverage areas and the bandwidth of the shared spectrum.



spectrum management, dynamic spectrum assignment, market modeling




§ INTRODUCTION

Spectrum sharing is receiving increased interest to meet the ever growing demands for wireless services. Examples include the recent U.S. National Spectrum Strategy <cit.> and programs such as the Spectrum Innovation Initiative: National Radio Dynamic Zones (SII-NRDZ) that seeks to further advance dynamic spectrum sharing <cit.>. 

Spectrum sharing involves both technical and economic dimensions in that it impacts how service providers (SPs) compete with each other. 
Models of competition with various forms of shared spectrum have been studied including 
<cit.>.  In these works, it was assumed that all competing SPs had the same coverage area, so that any customer could be served by any SP.  In this paper we depart from this and consider an example where two competing SPs are sharing the same spectrum and have distinct, partially overlapping coverage areas.  For example, this could model two WiFi providers sharing the same band of unlicensed spectrum, but with different coverage due to the placement of their access points (APs).  We seek to understand the impact of the geographic separation between the SPs on their competition.  


We consider a scenario in which two competing SPs each have an AP at a distinct location with overlapping coverage. We assume that the same spectrum band is used by two SPs.  We categorize the coverage into two types: dedicated areas served exclusively by one AP, and an overlapping area served by both APs. 
The SPs compete for a pool of customers spread across these areas.   The users are congestion-sensitive in that the price they are willing to pay depends on a congestion cost, which in turn varies across these areas, modeling different levels of interference that may occur. 

As in <cit.>, we adopt a Cournot competition model in which both SPs specify the number of customers they want to serve in both the dedicated and overlapping sub-markets. 
Our main results are as follows:

  
  * We prove that a unique Nash equilibrium always exists in the proposed model.
  
  * With limited bandwidth, SPs typically avoid overlapping areas to minimize the risk of significant congestion, which could adversely affect their revenue. However, with sufficient bandwidth, SPs will enter the overlapping sub-market. If this happens, the SPs may incur revenue losses due to the competition in the overlapping market, and consumer surplus may be redcued.
  
  * SPs might want to cooperate by agreeing not to serve users in the overlapping areas to avoid competition. Surprisingly, sometimes consumers can also benefit from such cooperation in the sense of total consumer surplus, which in turn leads to higher social welfare. However, this may also raise concerns about fairness, as no customers are served in the overlapping areas.
  
  * Market outcomes, including consumer surplus and social welfare, exhibit a complex dynamic and may not necessarily increase with the bandwidth provided to the SPs. This suggests that regulators need to carefully determine the amount of shared spectrum to optimize these outcomes.





Regarding related work, we follow the stream of modeling wireless spectrum as congestible resources <cit.> and our analysis builds on the framework in <cit.> where a market with intermittent spectrum is considered. Here, we instead consider a non-intermittent band of spectrum and account for the geographical differences in the locations of SPs. Our work is also related to work on access point or base station placement, e.g. <cit.>, but these studies focus more on the technical aspects of spectrum usage rather than its economic impacts.  This work also has ties to approaches that SPs could use to co-ordinate their spectrum usage, such as the use of Spectrum Consumption Models (SCM) <cit.>. 




§ MARKET MODEL

We consider a model in which there are 2 SPs (SP1 and SP2), each deploying an AP in distinct locations but with overlapping coverage. Both SPs compete for a common pool of non-atomic customers, who are categorized into three groups according to the coverage area they fall under. Let A and B be the sets of users under the coverage of SP1 and SP2, respectively. Then, we have three sub-markets with the corresponding sets of customers denoted by A\ B, AB = A∩ B, and B\ A, and with the market sizes , , and , respectively. For ease of analysis, we assume customers are non-atomic and, without loss of generality, the total market size ++=1.

In each sub-market, Cournot competition is considered in which SPs announce the quantity of users they want to serve, and this in turn leads to a market-clearing price <cit.>. Let x_1^, x_1^, x_2^, and x_2^ denote the quantities of users served by the SPs indicated by their subscripts. Fig. <ref> shows how users served by different SPs fall into different sub-markets. Since SPs can only serve users under their own coverage, we have the following constraints
rl
0 ≤    x_1^≤, 

0 ≤    x_1^ + x_2^≤, 

0 ≤    x_2^≤.


We define the market-clearing prices of each sub-market as follows
ll
p_d^    = 1 - x_1^/,

p_d^    = 1 - x_1^+x_2^/,

p_d^    = 1 - x_2^/.




These definitions are based on the following assumptions:


  * We assume that the whole market has the following linear (inverse) demand function
ll
p_d   = 1 - x,

where x=x_1^ + x_1^ + x_2^ + x_2^ is the total quantity of users served in the whole market.

  * Customers, differing in their valuation (i.e., utility) of the wireless service, are assumed to be uniformly distributed among sub-markets. In other words, within each sub-market, there are all types of users ranging from high-value users who are willing to pay for the service with a higher price to low-value users who have a limited budget. The mass of different types of users is proportional to the sub-market sizes.

One can verify that under definition (<ref>)–(<ref>), the only way to get the same market-clearing price across three sub-markets is to serve users in proportion to the sizes of sub-market, and the resulting price is given by (<ref>).

Next, we derive the consumer surplus in each sub-market. Take market  as an example. Equation (<ref>) specifies the inverse demand in market , i.e., it indicates the minimum price at which a mass of  x_1^ customers would accept service.  It follows that the surplus of the xth user is given by 1-x/m^-p_d^. To derive the consumer surplus of the entire sub-market, we need to integrate the surplus over x from 0 to x_1^. This results in the consumer surplus of each sub-market given by
ll
CS^    =  (x_1^)^2/2,

CS^    =  (x_1^+x_2^)^2/2,

CS^    = (x_2^)^2/2.


We refer to the market clearing prices in (<ref>)–(<ref>) as delivered prices. We assume that users are sensitive to congestion that is measured in terms of latency costs. Then the service price charged by an SP to its users is given by the difference between the delivered price and the latency cost. This models users that may avoid low-cost but poor-quality wireless services. A user will use the service of a SP only if the sum of both costs is lower than their valuation of this service. 

We assume the latency cost incurred by users on a band in a given sub-market depends on the total number of users impacting that sub-market divided by its bandwidth, where as described below a user may impact (e.g. cause interference) a sub-market even if it is not present in that sub-market.
 If both SPs use the same band, the traffic from both should be considered in this calculation. Assume a band with bandwidth W is used by both SPs. Considering their different coverage, we define the latency costs for the three sub-markets as follows
ll
l^   = x_1^+x_1^+x_2^/W,

l^   = x_1^+x_1^+x_2^+x_2^/W,

l^   = x_1^+x_2^+x_2^/W.

We have a few comments on (<ref>)–(<ref>):

    
  * Users within the same sub-market incur the same latency cost regardless of which AP they are connected to. To simplify the model, we assume the latency incurred by users mainly comes from the congestion caused by user traffic as opposed to path loss or shadowing.
    
  * The latency model here is motivated by WiFi in which carrier-sense multiple access with collision avoidance (CSMA/CA) is used for multiple access. Namely, we assume that at most one user within the range of a given AP can transmit at a time (or be transmitted to by the AP). For example, the latency of users in  will then depend on x_1^,  x_2^, and x_1^ as all of these users are within range of AP1.  Likewise, the latency of  users in  will depend on the number of users within range of either AP as these users are within range of both APs.[This is  a  simplification of an actual WiFi setting made to capture the key feature that with different geographic coverage, latency costs will depend on users within and external to a sub-market.] 
    
    
  * From (<ref>) one can conclude that market AB is always “more crowded” compared to market  and  as users there always incur higher latency costs. This may give SPs a preference for market  and . In the next section, we will see how this preference would change after factoring in market sizes. 


The revenue of an SP is the product of its service price and the quantity of users it serves. Thus, the revenue of each SP is given by
rl
R_1    = x_1^ (p_d^- l^) + x_1^ (p_d^- l^),

R_2    = x_2^ (p_d^- l^) + x_2^ (p_d^- l^).

Note here we  are assuming that the SPs can differentiate their prices across sub-markets, which requires them to know which users are in which sub-market.  This could be learned through measurements or shared if the SPs coordinate with each other, e.g. by using SCMs <cit.>.

The SPs' goal is to maximize their revenue (<ref>) and (<ref>) by carefully choosing the quantities of users to serve for each sub-market (i.e., x_1^, x_1^, x_2^, and x_2^). The revenues of SP1 and SP2 couple with each other's decision through both the delivered prices (<ref>)–(<ref>) and the latency costs (<ref>)–(<ref>). Such coupling makes it a game between SP1 and SP2. We will discuss the Nash equilibrium and the corresponding market outcomes in the next section.



§ MAIN RESULTS

In this section, we first characterize the Nash equilibrium of this two-player game, and based on that, we will examine welfare measures such as consumer surplus and social welfare.


 §.§ Equilibrium



There always exists a unique Nash equilibrium for any bandwidth W, and sub-market sizes , , and .

For symmetric cases in which =, the quantities of users served by SP1 and SP2 at the equilibrium are given as follows:

ll
    x_1^   =
     
    Wm^/2(W+m^),     0 ≤W < m^/2


      W m^/C,     W≥m^/2
   

   x_1^   =
     
    0,     0 ≤W < m^/2


      (2W-m^) m^/3C,     W≥m^/2
    

   x_2^    = x_1^,   x_2^ = x_1^, 














where
c
C=2(W+m^+m^)-m^m^/W.



Due to space considerations, we only provide the proof for the symmetric case, i.e., =. 

Before diving into the proof of uniqueness, we first provide some intuition, which is important for understanding how the proof is constructed. 

One can solve the following first-order conditions to gain insight into a potential equilibrium:
c
   [∂R_1/∂x_1^ ∂R_1/∂x_1^  ∂R_2/∂x_2^ ∂R_2/∂x_2^]^T
    =0.

The solution is given by the W≥m^/2 cases in (<ref>) and (<ref>), in which x_1^ will be negative if W< m^/2. Then we can guess that both SPs may abandon market  (i.e., x_1^=x_2^=0) when the bandwidth W is not large enough. But when the bandwidth is larger than m^/2, the solution to (<ref>) might be an equilibrium. Thus we will consider two cases, namely, W≥m^/2 and W < m^/2, and prove the uniqueness of equilibrium for each case.

Case W≥m^/2: We first show that the equilibrium is unique by showing it is a potential game and the potential function is strictly concave. Then we show that the solution to (<ref>) is feasible and thus is indeed the unique equilibrium. 

Let 𝐱_1=[x_1^, x_1^]^T and 𝐱_2=[x_2^, x_2^]^T. Then we can construct a potential function from revenue (<ref>) and (<ref>) as follows:

cl
   Φ(𝐱_1, 𝐱_2) =  

   - [(1/+1/W)x_1^^2 + (1/+1/W)x_1^^2

   + (1/+1/W)x_2^^2 + (1/+1/W)x_2^^2

   +(1/+1/W)x_1^x_2^ + 2/Wx_1^x_1^ 

   +1/Wx_1^x_2^ + 1/Wx_2^x_1^+2/Wx_2^x_2^]

   + x_1^ + x_2^ + x_1^ + x_2^.
   

One can verify that 
c
Φ(𝐱^'_1, 𝐱_2)-Φ(𝐱_1, 𝐱_2) = R_1(𝐱^'_1, 𝐱_2) - R_1(𝐱_1, 𝐱_2), 

Φ(𝐱_1, 𝐱^'_2)-Φ(𝐱_1, 𝐱_2) = R_2(𝐱_1, 𝐱^'_2) - R_2(𝐱_1, 𝐱_2).



We can rewrite (<ref>) in a quadratic form 
c
Φ(𝐱) = -𝐱^T𝐀𝐱 + 1^T𝐱,


where 
l
𝐀=

[   1/+1/W      1/W     1/2W        0;      1/W   1/+1/W 1/2+1/2W     1/2W;     1/2W 1/2+1/2W   1/+1/W      1/W;        0     1/2W      1/W   1/+1/W ],



and 
c
𝐱=[x_1^   x_1^   x_2^   x_2^]^T.

Next, we prove the uniqueness by showing that 𝐀 is positive definite given m^=m^, i.e., 𝐲^T𝐀𝐲>0, ∀𝐲∈ℝ^4.

From (<ref>), we observe that 𝐲^T𝐀𝐲 is a decreasing function of m^ as 1/m^ only appears in the diagonal. We can also show that it is a decreasing function of m^ as 
c
[ y_2 y_3 ]
[   1/+1/W 1/2+1/2W; 1/2+1/2W   1/+1/W ]
[ y_2; y_3 ]

= (1/+1/W )(y_2+y_3)^2+y_2^2+y_3^2/2.

Thus we can bound 𝐲^T𝐀𝐲 from below by substituting in the maximum values of m^ and m^:
cl
𝐲^T𝐀𝐲    ≥𝐲^T𝐀𝐲|_m^=2W,  m^=1

   =1/W(3/2y_1^2+2y_1y_2+y_1y_3

   +3/2y_4^2+2y_3y_4+y_2y_4

   +(W+1)(y_2^2+y_3^2+y_2y_3)).

The lower bound (<ref>) is a quadratic function of y_1 and y_4, whose minimizer is given by y_1^⋆=-(2y_2+y_3)/3 and y_4^⋆=-(2y_3+y_2)/3. Thus by plugging in y_1^⋆ and y_4^⋆, we have
cl
𝐲^T𝐀𝐲    ≥1/6W(y_2-y_3)^2+1/2((y_2+y_3)^2+ y_2^2+y_3^2)

   ≥0,

where the equality holds only when y_2=y_3=0 leading to y_1^⋆=y_4^⋆=0. Thus we can conclude that 𝐲^T𝐀𝐲>0 for all 𝐲≠ 0, which by definition proves 𝐀 is positive definite. Therefore, the potential function (<ref>) is strictly concave, and thus the equilibrium is unique. The existence of an equilibrium is obvious as the set of 𝐱 is compact because of (<ref>)–(<ref>). 

To show (<ref>) and (<ref>) are indeed the solution, we only need to prove that they are feasible, i.e., x_1^≤ and x_1^≤/2. The first inequality is obvious since W/C≤ 1. For the second inequality, we substitute m^ with 1-2m^ and then it can be reduced to 
c
2W^2+(6-4m^)W+6(m^)^2-3m^ ≥0,


where the LHS is a quadratic function of W, with W=m^-3/2<0 as the minimizer. Recall that W≥/2, thus the actual minimizer is W^⋆ = /2. Setting W=W^⋆ in (<ref>), we have
c
9/2()^2 ≥0,

which always holds. Therefore, the solution in (<ref>) and (<ref>) is feasible and thus is the equilibrium.

Case W< m^/2: The potential function (<ref>) is not necessarily concave with W< m^/2 so we will adopt a different approach. The equilibrium(s) is always given by the solution to the following problem
c
   𝐱≥0max Φ(𝐱) 

   s.t.  (<ref>)–(<ref>).
And, as mentioned before, the first-order solution (i.e., the solution to ∇Φ(𝐱)=0, which is equivalent to (<ref>)) is not feasible. Thus the equilibrium(s) must lie on a boundary[Again, the existence of an equilibrium follows from the compact set formed by (<ref>)–(<ref>).]. Next, we will show the uniqueness by excluding all boundaries except one.

First, it is easy to rule out the boundaries on market-size constraints (i.e., x_1^=, x_1^ + x_2^=, and x_2^=). For example, with x_1^=, the delivered price p_d^ is zero. This results in a non-positive service price p_d^- l^ charged by SP1 in market . Thus, SP1 can be better off if it stops serving users in this market (i.e., by letting x_1^=0) as it is currently paying these users for using its service. 

Second, to rule out the boundaries on non-negative constraints (i.e., x_1^=0, x_1^=0, x_2^= 0, and x_2^=0), we first prove the following lemmas which will help us check all combinations of these boundary conditions.


Any 𝐱 such that x_1^=0 but x_1^>0 (or x_2^=0 but x_2^>0) is not an equilibrium.

The intuition is that it is impossible to have an equilibrium in which an SP is willing to serve users in a more crowded market rather than a less crowded one.

    With x_1^=0 and x_1^>0, we have 
    cl
           p_d^- l^

        =   1-x_1^+x_2^/W

        >   1-x_1^+x_2^/-x_1^+x_2^+x_2^/W

        =   p_d^- l^,
    
    which shows that the service price in market  is higher than that in market .

    Consider a deviation 𝐱^'_1=[Δ, x_1^-Δ]^T from 𝐱_1=[0, x_1^]^T with Δ>0. The difference in SP1's revenue for any 𝐱_2 is
    cl
           R_1(𝐱^'_1,𝐱_2)-R_1(𝐱_1,𝐱_2)

        =    Δ[(p_d^- l^ - Δ/)

            - (p_d^- l^ + Δ/ ) + x_1^/].
    
    Given (<ref>), we can find always find a small enough Δ such that R_1(𝐱^'_1,𝐱_2)-R_1(𝐱_1,𝐱_2)>0, indicating 𝐱^'_1 is a profitable deviation from 𝐱_1 and thus it is not an equilibrium.
    




Any 𝐱 such that x_1^=0 and x_1^=0 (or x_2^=0 but x_2^=0) is not an equilibrium.

This lemma suggests that an SP cannot completely squeeze another SP out of the entire market.

    Consider a deviation 𝐱^'_1=[Δ, 0]^T from 𝐱_1=[0, 0]^T with Δ>0. For any 𝐱_2, the revenue of SP1 is 
    cl
           R_1(𝐱^'_1,𝐱_2)

        =   Δ(1-Δ/-Δ+x_2^/W)

        =   Δ(1-x_2^/W-(1/+1/W)Δ).
    
    Note that 1-x_2^/W>0 always holds for any 𝐱_2 at an equilibrium, otherwise SP2 would incur a negative service price in both market  and . Thus there always exists a small enough Δ such that R_1(𝐱^'_1,𝐱_2)>0, indicating 𝐱^'_1 is a profitable deviation from 𝐱_1 and thus it is not an equilibrium.




Any 𝐱 such that x_1^=0 but x_2^>0 (or x_2^=0 but x_1^>0) is not an equilibrium.

This lemma suggests that an SP cannot completely squeeze another SP out of market .

    Consider a strategy profile 𝐱=[x_1^, 0, x_2^,  x_2^]^T with x_2^>0. If it is an equilibrium, we must have x_1^>0 by Lemma <ref> and x_2^>0 by Lemma <ref>. Thus, to be an equilibrium, 𝐱 needs to be the solution to the following equations
    c
   [∂R_1|_x_1^=0/∂x_1^  ∂R_2|_x_1^=0/∂x_2^ ∂R_2|_x_1^=0/∂x_2^]^T
    =0. 

One can verify that the solution always has x_1^x_2^<0, indicating there is no feasible solution to (<ref>).


With Lemma <ref>–<ref>, we can check all 15 combinations of boundary conditions.[There are 4 x's with total 4 1+4 2+4 3+4 4=15 different combinations of boundary conditions.] For example, x_1^=0 and x_1^,x_2^,x_2^>0 can be ruled out by Lemma <ref>. The only boundary condition not eliminated is where x_1^, x_2^>0 and x_1^, x_2^=0. x_1^ and x_2^ are the solution to the following equations 
c
[∂R_1|_x_1^=x_2^=0/∂x_1^  ∂R_2|_x_1^=x_2^=0/∂x_2^]^T
=0.

The solution is given by the 0 ≤ W < m^/2 case in (<ref>) which is unique.





§ NUMERICAL RESULTS




 §.§ Equilibrium Quantities








In Fig. <ref> we illustrate the unique equilibrium user quantities versus the bandwidth W for two symmetric settings (i.e. when m^= m^) with different market size values . Note due to symmetry in these cases the user quantities for each SP are the same (e.g., x^ = x^). As suggested in (<ref>), both SPs do not enter market  until W is greater than m^/2, which is 0.2 and 0.1 in Fig. <ref> and Fig. <ref>, respectively. The intuition is that SPs avoid overlapping areas to minimize latency costs when bandwidth is limited. However, even with this preference, SPs will still begin serving users in the overlapping area if bandwidth W further increases. Initially, as W increases the SPs serve more users on the non-overlapping sub-markets.  This decreases the delivered price in these areas and thus the marginal revenue gained by adding users, until at some point it is more attractive to add users on the overlapping area (with a higher delivered price).  Note also that when  is larger as in 
Fig. <ref>, this increase in users in the overlapping area results in an initial decrease in users in the non-overlapping area.  This is because increasing traffic in the overlapping area increases the latency cost of SPs on both their overlapping and non-overlapping bands. This in turn reduces the marginal benefit of serving users on non-overlapping band.


Fig. <ref> shows the equilibrium user quantities versus W for two asymmetric cases (>) with different market size values . Note that SP2 has a smaller dedicated market compared to SP1. As in the symmetric case, when the bandwidth is limited, no SP joins the overlapping market. Interestingly, as W keeps increasing, SP2 first enters the overlapping market, and then SP1 enters later.  From SP1's perspective, since it has a relatively larger dedicated market, it does not benefit from competing with SP2 in market  if W is not large enough. However, as W keeps increasing, SP1 will  eventually enter the overlapping market to compete with SP2 directly, but will always serve few customers in that band than SP1 does.  For large , as shown in Fig. <ref>, the SPs once again may decrease the users served in the non-overlapping market, when they increase those served in the overlapping market.  Note here that the decrease for each SP begins at the value of W when that SP begins serving customers in the overlapping market.



Fig. <ref> and Fig. <ref> also suggest that the smaller the overlapping market, the higher the bandwidth required for either SP to join the overlapping market. This observation provides insight for a regulator, showing that providing sufficient bandwidth is the key to encouraging SPs to serve customers in overlapping markets.







 §.§ SP's Revenue

Next, we show some examples of the SP's revenue versus W in
Fig. <ref> and Fig. <ref> for the same settings as in Fig. <ref> and Fig. <ref>, respectively.  On these figures we also include the revenue obtained when the SPs “cooperate" and do not serve any customers in the overlapping band for any value of W.[This is one simple example of how SPs could cooperate.  We leave the consideration of other approaches for future work.]


For a small overlapping market, the equilibrium revenue is an increasing function of W, as shown in Fig. <ref> and Fig. <ref>. For large enough , however, revenue may decrease shortly after either SP enters the overlapping market, as shown in Fig. <ref> and Fig. <ref>. This is because when one SP uses the overlapping market, it increases the latency cost for the other SP on its non-overlapping band and does not account for this externality when determining its quantity.
This can be viewed as a type of Braess's paradox where adding more resources (i.e., increasing W) leads to lower revenue.  Similar effects have been noted in model of markets without geographic separation (e.g. <cit.>). The results here show that such behavior depends on the amount of geographic separation.


 As mentioned earlier, SPs have a preference for their dedicated markets when bandwidth is limited due to the higher latency cost in the overlapping area. When  SPs enter the overlapping market, by comparing their equilibrium revenue to the revenue obtained in the cooperation case, we can see that entering this market actually reduces both SPs revenue in the symmetric case for a range of W (when W is large enough both SPs would benefit from entering the overlapping market).[This is shown in Fig. <ref>; it also occurs for the scenario in Fig. <ref> if W is large enough, though the range of W needed is not shown.]
   This suggests in the symmetric setting the SPs may have an incentive to enter into an agreement to not serve customers in the overlapping region for some rang of W.
  For asymmetric markets as in Fig. <ref>, only the SP with a larger dedicated market (SP1) benefits from cooperation, while SP2's revenue decreases.  However, the total revenue increases for a range of W under cooperation so that it would be profitable for SP1 to compensate SP2 for cooperating. Again, this only holds for a range of W and if W is large enough (not shown), the total revenue would increase if both SPs enter the overlapping market.





 §.§ Consumer Surplus






Next we consider the consumer surplus versus W for the same set of scenarios. 
Fig. <ref> and Fig. <ref> illustrate how consumer surplus varies with W for the two symmetric cases with small and large value of 
, respectively.  Figures <ref> and <ref>, show the consumer surplus for each sub-market, while Fig. <ref> and Fig. <ref> show the surplus for the entire market as well as the surplus obtained by the “cooperation" case discussed previously. 

Focusing first on the individual sub-markets, 
the consumer surplus of market  is zero until SPs enter this market when the bandwidth W is large enough. Notice that after SPs start serving users in the overlapping market, the rate of increase of consumer surplus in their dedicated market slows down when  is small (Fig. <ref>) or even decreases when  is large (Fig. <ref>). Turning to the surplus for the entire market it can be seen that when  is small, this is increasing in W (Fig. <ref>), while when  is large (Fig. <ref>), the consumer surplus can decrease when W increases past the point where both SPs enter the overlapping market. Apparently, in this case the decrease in surplus on the non-overlapping markets is greater than the surplus gained on the overlapping markets.


When the SPs enter the overlapping market, they compete for customers in that market.  Surprisingly, as shown in Fig. <ref> and Fig. <ref>, such competition does not always increase overall consumer surplus compared to the case where the SPs cooperate and stay out of this market. While this competition can benefit users in the overlapping market, it reduces the surplus of customers in the non-overlapping markets compared to the cooperative case. Recall, as shown in Fig. <ref>, the SPs can also improve their revenue by cooperating in this way.  Thus, cooperatively avoiding the overlapping market can improve both the SPs' revenue and the consumer surplus for a range of W. However, it is important to note that this increase in total consumer surplus comes at the expense of consumer surplus in the overlapping area, where no one is served. Consequently, regulators might consider subsidizing users in the overlapping area, using the benefits derived from such cooperation. Note also that, as shown in Fig. <ref> and Fig. <ref>, with sufficiently large W cooperation results in lower surplus compared to the case where both SPs compete in the overlapping region. 





In Fig. <ref> and Fig. <ref> we show similar plots of consumer surplus for asymmetric scenarios with small and large values of , respectively. In both cases, for small values of W, there is no surplus generated in the overlapping market as the SPs do not compete in that market.  Also, note that for small values of W, the smaller SP (SP2) creates more surplus even though it is serving fewer customers.  This is due to the different demand curves in the two markets. SP2's demand curve in  has a steeper slope compared to SP1's demand curve in , meaning that it has to charge a lower delivered price to serve a similar number of customers, which in turn leads to larger welfare. 

In these asymmetric models, When  is small, consumer surplus increases as a function of bandwidth W. However, with a large  (Fig. <ref>), consumer surplus is decreasing in W around the values where the SPs enter the overlapping market. Once again, if the SPs cooperate and do not enter the overlapping market, this can increase the overall surplus for a range of W.



 §.§ Social Welfare


Social welfare is defined as the sum of consumer surplus and the revenue of SPs. Recall that, when  is small, both consumer surplus and revenue increase as a function of bandwidth W, leading to a corresponding increase in social welfare. However, for large values of , increasing bandwidth in the market may not necessarily lead to higher social welfare. We illustrate this in Fig. <ref> for the two large  scenerios. These show that 
 social welfare is not a monotonically increasing function of W. This is expected as we have already shown that both total revenue and consumer surplus may decrease in W when the SPs first enter the overlapping market. We also show the welfare obtained when the SPs cooperate and do not enter this market, which yields a welfare improvement for a range of W.  
 
 
 



§ CONCLUSIONS

We presented a model of a spectrum sharing market with two geographically separated SPs that have partially overlapping coverage areas. In this model, we proved that a unique Nash equilibrium always exists, where SPs avoid entering the overlapping market when bandwidth is limited. With sufficient bandwidth, SPs will enter the overlapping market.  However, the resulting revenue and consumer surplus may decrease as a function of the amount of bandwidth once the SPs enter the overlapping market.  We also showed that the revenue and consumer surplus can both be improved for a range of bandwidth values by allowing the SPs to cooperate and not enter this overlapping market. This suggests that such cooperation may be desirable, but would need to be balanced by considering the fairness to the users within this overlapping market. 


IEEEtran



 Paper Title: A note about high-order semi-implicit differentiation: application to a numerical integration scheme with Taylor-based compensated error.


Fast and scalable finite-element based approach for density functional theory calculations using projector-augmented wave method
    Phani Motamarri
    August 5, 2024
================================================================================================================================





Abstract: In this brief, we discuss the implementation of a  third order semi-implicit differentiator as a complement of the recent work by the author that proposes an interconnected semi-implicit Euler double differentiators algorithm through Taylor expansion refinement. The proposed algorithm is dual to the interconnected approach since it offers alternative flexibility to be tuned and to be implemented in real-time processes. In particular, an application to a numerical integration scheme is presented as the Taylor refinement can be of interest to improve the global convergence. Numerical results are presented to support the rightness of the proposed method.



Preprint - Work in progress



§ INTRODUCTION


Acary et al. <cit.> introduced an implicit discretization technique from which differentiation techniques, for implicit differential inclusion, have emerged recently (see e.g. a survey in <cit.>). The principle of this method was initially to replace the classical sign function by an implicit projector, then it has been combined with explicit (homogeneous) terms to define the semi-implicit differentiation <cit.>; the overall goal is to reduce the effects of chattering. 
Based on the latest work from the author et al.  <cit.> that proposes a semi-implicit cascade differentiator including Taylor refinement to reduce the estimation error, in this brief, an alternative version of this latter algorithm is proposed that could possibly offer more flexibility towards practical implementation.

The brief is organized as followed. The first Section defines the third order semi-implicit differentiator as a natural extension of the previous works <cit.> <cit.> including numerical results, the second Section illustrates an application towards a numerical integration scheme and the third Section gives some concluding remarks and perspectives.
Simulation code can be found in the repository: 



§ SEMI-IMPLICIT DIFFERENTIATION ALGORITHM INCLUDING TAYLOR REFINEMENT




 §.§ Presentation


Considering a real signal x_1(t) to differentiate, denote x_1(k δ) its corresponding discretization where k is an integer and δ the sampling period. Moreover, x_1^+ stands for x_1((k+1) δ).
From the cascaded differentiator  <cit.>, we derive a third-order Semi-Implicit Differentiator (SIHD-3), including correction terms (in bold) that refine the Taylor expansion of the two first estimation lines:


    {[                               z_4^+= z_4+  E_1^+ E_2^+ E_3^+ h ( λ_4 μ^4|e_1|^4 α-3𝒩_4  );                                  z_3^+= z_3+ E_1^+ E_2^+ h (z_4^++λ_3 μ^3|e_1|^3 α-2𝒩_3 );                 z_2^+= z_2+ E_1^+ h (z_3^+ -E_3^+ h 1/2 z_4^+  + λ_2 μ^2|e_1|^2 α-1𝒩_2  ); z_1^+= z_1+ h ( z_2^+ -E_2^+  h/2 z_3^+ + E_3^+ E_2^+ h^2/3! z_4^+  +  λ_1 μ |e_1|^α𝒩_1 );                                                                                           ].

 
 or, equivalently, introducing the SIHD related operator:
 
 
    z_i = D_f ^(i)(z, z^+, x_1),    i = 1...4

 
 where z_i is the estimation associated to the ith line of the differentiator (<ref>).
 
 Denote y = x_1 the signal to differentiate; ẏ = x_2,  ÿ = x_3 and y^(4) = x_4 are respectively the first, second and third order differentiation;  z_2,   z_3 and z_4 are respectively the first, second and third order related estimations; e_1 = x_1 - z_1 is the estimation error.

The projector[The projector behaves either as a sign function, or as a nonlinear "slope" depending on the value of the error e_1 <cit.>.] 𝒩_q for q = 1...4  is defined by:

 
    𝒩_q(ϵ_1):= {[ ϵ_1 ∈ SD →𝒩_q=⌈ϵ_1⌋^q(1-α)λ_q (μ h)^q,  E_q^+=1;                                                ;           ϵ_1 ∉ SD →𝒩_q=   sign(ϵ_1),   E_q^+=0 ].

with the convergence domain[Note that, according to <cit.>, it has been shown that the error e_2, corresponding to the second estimation line of (<ref>), converges asymptotically to e_ 2^+ = e_1^+ / h, hence 'propagating' the errors of estimation through the differentiation orders by 'increment' of h.] SD={ϵ_1   /   |ϵ_1|≤ (λ_1 μ h)^1/q(1-α)}.

In eq. (<ref>), the variable E^+_q allows activating the upper (q+1)th line if the convergence domain has been reached at the qth line. This prevent from getting unexpected behavior between lines like oscillations that could destabilize the differentiator.

The corrections terms have been introduced by the author et al.  in <cit.> with respect to the second order differentiation and a study of the convergence has been proposed in <cit.>. The complete derivation of the refined Taylor expansion in (<ref>)  is described in Appendix.



 §.§ Numerical results


The  differentiator (<ref>) is simulated over 10 sec. considering a sine function signal 


    y(t) = sin(2 π t )
 

under the following initial conditions (that match the derivatives): x_1(0) = 0,   x_2(0) = 1,   x_3(0) = 0 and x_4(0) = -1.  The gains and parameters  are set as follows: λ_1 = 10^3, λ_2 = 10^6, λ_3 = 10^9, λ_4 = 10^9 and α = 0.95.



Figures <ref> and <ref> illustrate respectively the evolution of the estimated z_1, z_2, z_3 and z_4 with respect to the corresponding exact differentiation orders of the sine function without extra noise on y.

Figures <ref> and <ref> illustrate respectively the evolution of the estimated z_1, z_2, z_3 and z_4 with respect to the corresponding exact differentiation orders of the sine function including extra noise of very small amplitude on y (obtained by a random generator[The noise is computed under Matlab® using the instruction  where  is a factor that multiplies the noise.] that is multiplied by η_0 = 10^-6).












Low estimation errors are observed, thanks to the correction terms, without extra noise. In particular, the error is of order < e_1 > = 2.96   10^-12  on the z_1 estimation, of order < e_2 > = 4.69   10^-9 on the estimation of the first derivative z_2 and of order < e_3 > = 4.49   10^-6 on the estimation of the second derivative z_3.
In presence of noise, the accuracy of the estimation remains correct for the two first orders (< e_1 > = 7.55   10^-7 and < e_2 > = 7.47   10^-4) but the noise becomes predominant for the third order, which makes the estimation very noisy. Note that the θ projector <cit.>, which has shown interesting performances with respect to the rejection of the noise, has not been implemented in the current version of the proposed SIHD3 differentiation algorithm. It constitutes an important issue to deal with noisy signals for future investigations.



§ APPLICATION TO NUMERICAL INTEGRATION SCHEME




 §.§ Observer-based numerical scheme



Consider an ordinary differential equation (ODE) such as:

    d   y(t)/d   t = f(y(t), u(t)),      t > t_0,    y(0) = y_0.

The quantities u and y represent respectively the input and the solution of (<ref>). The corresponding usual discrete explicit Euler scheme reads:


    y_k+1 - y_k/h = f(y_k, u_k),      k ∈ℕ,    y(0) = y_0


where k is the sampled time, h is the time-step, and y_k, y_k+1 are respectively the solution of (<ref>) at the discrete instants t_k and t_k+1. The sampled are supposed equally distributed i.e. t_k - t_k-1 is constant for all k.

In the context of solving ODE, an observer-based numerical integration scheme is proposed to provide a better precision of the estimation of the ODE solution. The proposed scheme is close to the Taylor-based numerical schemes <cit.> for which we point out the refined Taylor based estimation of f(y_k, u_k) via our proposed SIHD algorithm. 

Our proposed differentiation method can be  associated to the nonstandard finite-difference (NSFD) methods <cit.>, described as "powerful numerical methods that preserve significant properties of exact solutions of the corresponding differential equations" (a survey can be found e.g. in <cit.>)[Applying a particular differentiator algorithm to the ODE discretization in the context of NFSD was proposed by the author in <cit.>].

The NSFD & SIHD3 based numerical integration scheme reads:


    {[ z_1^+  = z_1 + ψ D_f^(1)(z, z^+, ŷ );                              ŷ = z_1 ], .   k ∈ℕ,   y(0) = y_0   and  ψ = h + O(h^2)


where the SIHD3 differentiator is used to estimate the f part of the ODE and ψ = h + O(h^2) that do satisfy h → 0 according to the NSFD rules[For example, ψ can be chosen linear of h or exponential of h <cit.>.].

In this work, the purpose is to illustrate the convergence of the proposed NSFD &  SIHD3-based algorithm by comparing the evolution of the error with the classical forward Euler scheme and the Runge-Kutta (RK) scheme according to the time.
Note that in this scheme, the semi-implicit differentiator  (<ref>) appears as a closed-loop observer which tends to adjust and propagate the estimation of f through the numerical integration scheme.




 §.§ Numerical examples and discussion


Three examples are presented to illustrate some first convergence properties  through the evolution of the error | y_exact(t) - y_ODE   scheme(t) | with the classical forward Euler scheme and the Runge-Kutta (RK) scheme according to the time. y_exact(t) denotes the exact solution of the ODE (<ref>) and y_ODE   scheme(t) denotes the approximated solution of the ODE (<ref>) using either the classical Euler forward method or the RK method.
 The internal parameters (λ, α) of  SIHD3 are the same as in the first Section (λ_1 = 10^3, λ_2 = 10^6, λ_3 = 10^9, λ_4 = 10^9 and α = 0.95).



  
Example 1 Consider the basic first order equation:


    d   y(t)/d   t = -y(t) + 1,    y(0) = 10^-2,     t > 0


whose exact solution is y_exact = 1 - exp( - t ). Fig. <ref> illustrates the comparaison of the errors between the schemes.




  
Example 2 Consider the logistic equation <cit.>:


    d   y(t)/d   t = 2 y ( 1 - y),     y(0) = 1/5,     t > 10


whose exact solution is y_exact = 1/1 + 4 exp( 2 (10-t  )).  Fig. <ref> illustrates the comparaison of the errors between the schemes.





  
Example 3 Consider the simple model of a water-discharge problem <cit.>:


    d   y(t)/d   t = -10y^2 + 20,     y(0) = 10^-2,     t > 0


whose exact solution is y_exact = √(2)tanh(√(300) t).  Fig. <ref> illustrates the comparaison of the errors between the schemes.


These preliminary numerical experiments show that the proposed NSFD-SIHD3 numerical scheme provides a slightly reduced error compared with the Euler and Runge-Kutta schemes, especially when the solution is converged towards the asymptotic solution. It is not easy to claim that the transient behavior is more precise than the other scheme since SIHD3 performs a dynamic estimation that depends on the (λ, α) parameters, inducing delays during the estimation. Moreover, the NSFD approach appears naturally to adjust the rate of convergence of SIHD3 thanks to the ψ time-step: indeed, ψ can be considered as an additional SIHD3 'external' gain that can 'virtually' increase the speed of convergence of the scheme, hence giving some flexibility between the SIHD3 parameters (λ, α) from the estimation side, and ψ from the numerical scheme side.




§ CONCLUSION AND FUTURE WORKS


We have derived a   differentiator scheme of order three based on semi-implicit techniques that uses Taylor expansion in an implicit way. This differentiator allows reducing the estimation error in comparaison with exact derivatives. 
Future works include the implementation in real-time in the context of robotics applications.
We have also proposed a numerical scheme based on NSFD scheme including the  differentiator as a refined estimator of the f term in the right side of the ODE.
Further investigations concern a complete study of convergence of the proposed scheme, as well as a comparison of the global error with existing algorithms and possibly an extension toward a semi-implicit scheme.




§ APPENDIX


In this appendix, the corrections terms of the differentiator (<ref>) are calculated from the assumption that the discretized signal x_1 can be differentiated until the third order (inducing a constant fourth order derivative).

Assuming that the discretized signal x_1 can be differentiated though the following 'multi-step' Taylor expansion:


    x_1^+   =    x_1+h x_2 + h^2/2! x_3 + h^3/3! x_4 + h^4/4! x_5 
    x_2^+   =    x_2+h x_3 + h^2/2! x_4 + h^3/3! x_5 
    x_3^+   =    x_3+h x_4 + h^2/2! x_5 
    x_4^+   =    x_4 + h x_5 
    x_5^+   =    x_5


and assuming that the fifth order remains constant x_5 = x_5^+, the retro-propagation of the next-step predictions x_q^+, q = 2...4 terms allows refining the first Taylor expansions by substituting each x_q terms by their such corresponding predictions.

To begin with, the third line:

    x_3^+ = x_3+h x_4 + h^2/2! x_5


can be rewritten easily and substituting both x_4 = x_4^+ - h x_5 and x_5^+ = x_5, allows rewriting:

    x_3^+    =    x_3+h ( x_4^+ - h x_5^+ ) + h^2/2! x_5^+


that gives:

    x_3^+ = x_3+h  x_4^+ - h^2/2! x_5^+.


In the same way, substituting x_3 =  x_3^+ -h x_4^+ - h^2/2! x_5 and x_4 = x_4^+ - h x_5 in the second line gives after few calculations:











    x_2^+= x_2 + h   x_3^+ - h^2/2! x_4^+ + R( x_5 )


The term R( x_5 ) denotes the "residual" term associated to the useless fifth order.

Lastly, in the same way, substituting x_3, x_4 and  x_2 into x_1 gives: 


    x_1^+= x_1+h  x_2^+ -  h^2/2!  x_3^+ + h^3/3!x_4^+   + R( x_5^+ )


The bold terms in (<ref>) and (<ref>) correspond indeed to the correction terms that refine the Taylor expansion of the two first lines in (<ref>), taking into account the refinement of the precision of the Taylor terms thanks to the "retro-propagation" of the high order implicit terms.



§ ACKNOWLEDGMENTS

This work was supported partially by the ANR project DigitSlid ANR-18-CE40-0008-01 (https://anr.fr/Project-ANR-18-CE40-0008). Jean-Pierre Barbot is supported with (Région Pays de la Loire) Connect Talent GENYDROGENE project.


unsrt




































 Paper Title: Machine Learning In-Sensors: Computation-enabled Intelligent Sensors For Next Generation of IoT.














Machine Learning In-Sensors: Computation-enabled Intelligent Sensors For Next Generation of IoT  






    
    Andrea Ronco, Lukas Schulthess, David Zehnder, Michele Magno
    Dep. Information Technology and Electrical Engineering 

    ETH Zürich

    andrea.ronco@pbl.ee.ethz.ch
    
  

    August 5, 2024
===========================================================================================================================================================================================



§ ABSTRACT 



Smart sensors are an emerging technology that allows combining the data acquisition with the elaboration directly on the Edge device, very close to the sensors.





To push this concept to the extreme, technology companies are proposing a new generation of sensors allowing to move the intelligence from the edge host device, typically a microcontroller, directly to the ultra-low-power sensor itself, in order to further reduce the miniaturization, cost and energy efficiency.

This paper evaluates the capabilities of a novel and promising solution from STMicroelectronics. The presence of a floating point unit and an accelerator for binary neural networks provide capabilities for in-sensor feature extraction and machine learning.
We propose a comparison of full-precision and binary neural networks for activity recognition with accelerometer data generated by the sensor itself.

Experimental results have demonstrated that the sensor can achieve an inference performance of 10.7 cycles/MAC, comparable to a Cortex-M4-based microcontroller, with full-precision networks, and up to 1.5 cycles/MAC with large binary models for low latency inference, with an average energy consumption of only 90/inference with the core running at 5 MHz. 







Smart Sensors, IoT, Low Power, Edge Computing, Machine Learning





§ INTRODUCTION




Artificial Intelligence and Machine Learning paradigms are increasing their presence in the area of the Internet of Things (IoT) and Smart Sensors to face the challenging task of automatically extracting information from the data generated by "traditional" sensors spread in the physical world <cit.>.
An emerging trend is to bring machine learning on low-power IoT devices, in contrast with the Cloud Computing paradigm, where the data processing is performed in the cloud.
This trend is motivated by the huge amount of data produced by a growing number of IoT devices.
Transferring all the data to the cloud would have several drawbacks including high energy requirements for data transmission, and, more generally, reliability, latency, and privacy concerns <cit.><cit.>. 














Major challenges arise when processing data at the Edge, mostly due to the highly constrained resources available on smart IoT devices<cit.>.
Thus, bringing intelligence to the edge creates fascinating challenges for industrial and academic researchers <cit.>.
In fact, most of today's low-power IoT smart sensors are composed of a low-power microcontroller (MCU), a set of sensors, and an optional but common wireless interface to transmit the findings of the device to a remote gateway or backend <cit.>. 




Many academic and sensor technology companies, including Bosch, TDK, and many others, are proposing a new generation of smart sensors that embed ultra-low power digital signal processors in the same integrated circuit.
In particular, there is interest in combining the sensing part with a small neural network core to process the sensor data on the fly, increasing the overall efficiency and reducing the system latency.




This novel approach is based on the intuition that waking up external processors periodically might have an impact on the power consumption, which is avoidable by moving the intelligence from the host processor to the sensor itself.

This paper presents the evaluation of a novel sensor with an Intelligent Sensor Processing Unit (ISPU) designed and produced by . The ISPU includes both full precision and binary neural network accelerators as well as the capability of DSP for feature extraction of the raw data. The main contribution of this paper is the design of benchmark neural networks (both full precision and binary) to evaluate the new sensor from under different loads to characterize the performance, including the features extraction on the ISPU, and the usability of this solution.
We compare different neural network architectures using both full-precision and binary implementations, with the aim to evaluate the power efficiency of the core in different conditions.
Experimental evaluation in terms of Multiply-Accumulate (MAC) operations per cycle and power in different conditions has been performed to have a fair comparison with a popular microcontroller for smart sensors based on ARM Cortex-M4F.






























§ STRED INTELLIGENT SENSOR PROCESSING UNIT



The inertial module is composed of a low-power, always-on, 6-axis Inertial Measurement Unit (IMU) and the novel Intelligent Sensor Processing Unit (ISPU), as represented in Figure <ref>.
The main novelty of this sensor is the presence of a fully programmable 32 core inside the sensor itself.

The processing core (ISPU) is optimized for power consumption and machine learning, and features 40 of RAM memory (for both program and data), a Floating Point Unit (FPU) for full-precision operations, and an additional Binary Neural Network accelerator with support for convolutions, to enable efficient BNN execution in the sensor.
The sensor offers a high-performance mode and a low-power one, with different sampling frequency ranges, while the ISPU can run at 5 or at 10.

While the processing core allows the implementation of any algorithm, compatibly with the computational resources, the ISPU is designed with a focus on Binary Neural Networks, which are a competitive alternative to full-precision models <cit.>, especially when coupled with ad-hoc accelerators <cit.> <cit.>.

The core has been designed with a wake-up time of only 4 cycles, to allow efficient event-based processing with minimal latency. It wakes up when new sensor data are available, but it is also possible to trigger the execution of algorithms from the external host MCU if needed.

The core can be programmed in C with user-defined algorithms, or with the support of the tools provided by , which allow to deploy Keras <cit.> models and take advantage of the BNN accelerator.







§ EVALUATION SCENARIO AND PROPOSED NEURAL NETWORK

To evaluate the potential of in-sensor intelligence with the ISPU under a realistic real-world setting, we chose an activity recognition task.
Specifically, we opted for an activity classification of different actions on an office chair, as this allows us to use directly the data from the accelerometer hosted in the same die of the ISPU. 
The set of activities we wanted to recognize is the following:

    
  * Idle: no motion is detected on the chair.
    
  * Stand up: a person stands up leaving the chair empty.
    
  * Sit down: a person sits on an empty chair.
    
  * Rotate: the chair spins around the shaft.
    
  * Move: the chair is moved around.



For our demonstration, we consider a prediction pipeline composed of a Feature Extractor followed by a Neural Network, as shown in Figure <ref>.
This well-known combination <cit.> allows us to evaluate the performance of the ISPU for both traditional feature extraction/signal processing tasks and machine learning loads.





 §.§ Feature Extraction

We adopted a feature extraction algorithm based on standard time-domain statistical features, which are suitable for real-time systems <cit.> and are invariant to the sampling rate of the sensor.

Since the ISPU is triggered at every acquisition, we consider 1 acquisition as our time unit to simplify the explanation below.
Every acquisition yields three 16 ADC samples, one per axis. The samples are stored in three dedicated circular buffers of length N=32.

With a periodicity of T=32 (when the buffers are full), we evaluate for each axis the following set of features: Mean, Median, Variance, Maximum, Minimum.
The set of features S_t is stored in a shift buffer which contains the last L=2 sets, from S_t to S_t-(L-1).
Together they compose the input of the neural networks described later.


The configuration we used in our application generates a total of 30 features, which compose the input for the neural networks described below.





 §.§ Neural Networks

We compare different neural network architectures using both full-precision and binary implementations, with the aim to evaluate the power efficiency of the ISPU in different conditions. We generated networks with growing complexity by adding hidden layers with 32 or 64 units.
The full-precision networks have an input size of 30 and the hidden layers have ReLU activation. 
The input of the binary network is padded with two zeros to comply with hardware limitations of the BNN core, which requires the input size of the layers to be multiple of 32. Binary activation is used in the hidden layers. All the models begin with a Batch Normalization layer and terminate with a fully connected layer with SoftMax activation.
The model variations are described in Table <ref>, along with the network complexity measured in Multiply-Accumulate operations.



To train the networks, we acquired a dataset with three different chair types. For each chair, we asked 25 different people to perform the required 5 activities. The models were trained for 100 epochs.
We evaluated the models with standard train/validation/test split, and achieved for all models an accuracy between 96 and 98 for the full-precision ones, and between 93 to 97 for the binary versions, confirming the expectations to experience a drop in accuracy due to the extreme quantization.
The models were deployed on the sensor with the tools provided by STMicroelectronics, while the feature extraction was implemented directly in C.























































































































§ EXPERIMENTAL RESULTS































 
 
 
 
















 §.§ Execution Time and Power Consumption

Since the toolchain does not include an accurate performance profiler, we evaluate the execution time by measuring the average duration of the ISPU phase in the power profile on about 100 samples per configuration.


The feature extraction has an average duration of 6.57, and is common to all the models.

As visible in Figure <ref>, the overhead of the feature extraction is considerable and dominates in the simplest network architectures.
We believe a consistent part of the overhead might be caused by the integer-to-floating-point conversions, which to our knowledge are not optimized.





In Table <ref> we compare the cycles/MAC operation for each network architecture on both the ISPU and an STM32L4R9, an MCU based on the Cortex-M4 core. On the latter, we only evaluate the full-precision models, since the architecture does not have native support for binary networks.

The data show that small networks, in general, perform worst than large ones, as the additional overhead of operations such as Batch Normalization/Softmax has a greater impact.
This is even more relevant in the case of binary networks, because of the aforementioned overhead of data conversion needed at the input and the output of the network.
For the same reason the binary models struggle to provide any speedup for small to medium models, granting only about 25 speedup for the largest model we could compare (Float_(2,64)) on the sensor, as visible in Figure <ref>.

To test this thesis, we also evaluated a single binary model with 4 hidden layers and 256 hidden units per layer to maximise the BNN accelerator utilization. While it was not possible to test the full-precision version of this model on the sensor for memory limitations, the binary one achieved 1.48 cycles/MAC operation, a promising speedup of about 7x on the previously measured full-precision performance.





































§ CONCLUSION




We proposed an evaluation of on-sensor activity classification with the novel sensor with integrated ISPU and analyzed the performance of the core under different conditions, proposing neural networks for realistic application scenarios, and exploiting both the sensing and the on-board processor. Our evaluation shows that the ISPU is suitable for running small full-precision networks, and is mainly limited by the available memory for what concerns this task. The cycles/MAC metric also shows that the performance of the core is comparable with the performance of a Cortex-M4 core with intensive full-precision loads with an inference energy of only 90. Our experiments also show that it is possible to run large binary models with a speedup of around 7x when using the dedicated BNN accelerator.







00
FANN-on-MCU Wang, X., Magno, M., Cavigelli, L., & Benini, L. (2020). "FANN-on-MCU: An open-source toolkit for energy-efficient neural network inference at the edge of the Internet of Things." IEEE Internet of Things Journal, 7(5), 4403-4417..
EdgeComputing G. Premsankar, M. Di Francesco and T. Taleb, "Edge Computing for the Internet of Things: A Case Study," in IEEE Internet of Things Journal, vol. 5, no. 2, pp. 1275-1284, April 2018, doi: 10.1109/JIOT.2018.2805263.
edge_computing_privacy A. Alwarafy, K. A. Al-Thelaya, M. Abdallah, J. Schneider and M. Hamdi, "A Survey on Security and Privacy Issues in Edge-Computing-Assisted Internet of Things," in IEEE Internet of Things Journal, vol. 8, no. 6, pp. 4004-4022, 15 March15, 2021, doi: 10.1109/JIOT.2020.3015432.
Magno2009 Magno, M., Brunelli, D., Thiele, L., Benini, L. (2009, August). Adaptive power control for solar harvesting multimodal wireless smart camera. In 2009 Third ACM/IEEE International Conference on Distributed Smart Cameras (ICDSC) (pp. 1-7). IEEE.
Jelicic2010Jeličić, Vana, Michele Magno, Davide Brunelli, Vedran Bilas, and Luca Benini. "An energy efficient multimodal wireless video sensor network with eZ430–RF2500 modules." In 5th International Conference on Pervasive Computing and Applications, pp. 161-166. IEEE, 2010.
ContiWear Conti, F., Palossi, D., Andri, R., Magno, M., & Benini, L. (2016). Accelerated visual context classification on a low-power smartwatch. IEEE Transactions on Human-Machine Systems, 47(1), 19-30.
EdgeComputing2 W. Shi, J. Cao, Q. Zhang, Y. Li, and L. Xu, “Edge Computing: Vision and Challenges,” IEEE Internet of Things Journal, vol. 3, pp. 637–646, Oct. 2016. Conference Name: IEEE Internet of Things Journal.
bnn_good G. Cerutti, R. Andri, L. Cavigelli, M. Magno, E. Farella, L. Benini, "Sound Event Detection with Binary Neural Networks on Tightly Power-Constrained IoT Devices", 2021.
bnn_yoda R. Andri, L. Cavigelli, D. Rossi and L. Benini, "YodaNN: An Ultra-Low Power Convolutional Neural Network Accelerator Based on Binary Weights," 2016 IEEE Computer Society Annual Symposium on VLSI (ISVLSI), 2016, pp. 236-241, doi: 10.1109/ISVLSI.2016.111.
bnn_xor F. Conti, P. D. Schiavone and L. Benini, "XNOR Neural Engine: A Hardware Accelerator IP for 21.6-fJ/op Binary Neural Network Inference," in IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, vol. 37, no. 11, pp. 2940-2951, Nov. 2018, doi: 10.1109/TCAD.2018.2857019.
keras Keras. The Python Deep Learning API. https://keras.io/. [Online; accessed 1/2/2022]
features_nn Y. Chen and Y. Xue, "A Deep Learning Approach to Human Activity Recognition Based on Single Accelerometer," 2015 IEEE International Conference on Systems, Man, and Cybernetics, 2015, pp. 1488-1492, doi: 10.1109/SMC.2015.263.

features J. M. H. Priyadharshini, S. Kavitha and B. Bharathi, "Classification and analysis of human activities," 2017 International Conference on Communication and Signal Processing (ICCSP), 2017, pp. 1207-1211, doi: 10.1109/ICCSP.2017.8286571.
stm32_datasheet STM32L4F5/7/9 Datasheet:
 www.st.com/resource/en/datasheet/stm32l4r5vi.pdf [Online; accessed 1/2/2022]
































































































































































































































 Paper Title: Hidden Cyber-Physical Contingency Identification, Classification and Evaluation in Modern Power Systems.


Lifelong Person Search
    Jae-Won Yang, Seungbin Hong, and Jae-Young Sim, Member, IEEE

    August 5, 2024
=================================================================

§ ABSTRACT 
This paper introduces an advanced stochastic hybrid system modeling framework for modern power systems (MPS) to identify, classify, and evaluate hidden contingencies, which cannot be detected by normal observation sensors. 
The stochastic hybrid system (SHS) model is designed to capture the dynamics of the internal states of individual nodes, considering their structural properties, and coupling variables under various local and network-level contingencies. Hidden contingencies are identified using a probing approach that measures changes in the eigenvalues of the SHS model and detects deviations from normal operation.
Next, contingencies are categorized into three distinct groups according to their impact on MPS: physical contingencies, control network contingencies, and sensing and measurement network contingencies. This classification enables a proactive evaluation of contingencies.
The practicality and efficacy of the proposed methodology are validated through simulation experiments on the electrical network of two real-world systems. These simulations underscore the approach's capacity to enhance the resilience of power systems against a spectrum of hidden contingencies.


Fully coupled stochastic hybrid systems, Contingency Identification, Cyber-Physical contingency classification, and Evaluation





§ INTRODUCTION


MPS faces a myriad of unpredictable cyber-physical contingencies (CPC), including line outages, equipment malfunctions, cyber attacks, and protection system failures, all of which can jeopardize system reliability. The timely identification and rectification of such contingencies are essential for maintaining MPS reliability. CPC identification within MPS is a critical process that involves the vigilant observation and analysis of system data, such as voltage and current readings. This data is then benchmarked against established norms of system operation to proactively pinpoint potential disruptions before they escalate into full-blown system failures or cause extensive equipment damage.
However, minor contingency events, particularly on the distribution side of the grid such as protection system failures, may not be directly observable under normal operating conditions but become apparent during system disturbances.


Economic and environmental concerns are pushing MPS to operate near stability limits, where hidden contingencies can cause stability issues and potential cascading failures <cit.>. This is especially critical given that distributed energy resources (DERs) with low inertia are highly sensitive to contingencies and prone to instability. For instance, <cit.> demonstrates that a single line outage in a network alters power flow, causing over-current in distant lines and unpredictable relay activation leading to cascading failures. Consequently, developing a systematic approach to promptly detect and mitigate hidden contingencies is essential for ensuring the reliable operation of MPS and preventing cascading failures and blackouts.

The literature predominantly focuses on the detection and management of physical contingencies. Studies demonstrate real-time monitoring capabilities by utilizing PMUs <cit.> and transmission-switching techniques for voltage stability purposes <cit.>. However, the effectiveness of these methods is constrained by the existing infrastructure and placement of PMUs, limiting their ability to detect hidden contingencies. Detection, analysis, and mitigation of sequential hidden N-k line outage based on mixed integer linear programming is studied by <cit.> and <cit.>. Broader stability assessments in <cit.> and <cit.> provide more comprehensive analyses yet remain focused on physical disruptions. Furthermore, the application of machine learning highlights the potential of advanced algorithms to enhance CPC predictions in <cit.>.  However, these methods necessitate extensive training data, and not all CPC signatures are well-defined or measurable. For instance, in <cit.> an autoencoder is designed to detect the existence of anomalies in a distribution network. Similarly, <cit.> applies a penalty method for monitoring and detecting anomalies in the distribution system.
However, to the best of our knowledge, existing research reveals a substantial gap in the detection, identification, and evaluation of hidden contingencies in MPC and the modeling of various forms of contingencies within a united modeling approach.  This approach should encompass a broader spectrum of threat types and effectively tackle the issue of unrecognized contingencies.

In this paper, we aim to develop a versatile dynamic model based on internal and coupling states that have the potential to offer broad applicability for hidden CPC identification across MPS.

Although MPS can change their structures and parameters at any time instant, all management platforms have an interval (internal clock) for data sampling and decision making. Thus, within each time interval, the state space model is assumed to be fixed. The interval depends on sensor, communication, and computer systems. For example, for data acquisition accuracy and anti-aliasing requirements in PMU data, the data rate of the Power Xpert Meter is 1024 samples per cycle in MPS. For CPC management, 160 ms is the IEEE imposed limit for voltage surge. 
Mathematically, the occurrence of contingencies can be treated as a discrete-time stochastic instant where the identification could take place within discrete time intervals rather than a continuous-time process. Hence, the state space model of the system could be represented as a SHS. By this approach, hidden contingencies can be modeled as jumps in the SHS model.

In <cit.>, the dynamics of SHS with random discrete events are modeled as a randomly switched linear system (RSLS) with predefined finite switching scenarios. Next, necessary and sufficient conditions for distinguishing the occurrence of switching in the systems configurations are developed. Accordingly, an algorithm for detecting the active operating scenario and estimating the contiguous states of the system according to the detected operating scenario has been developed. 

In our previous work <cit.>, we categorized nodes within MPS into dynamic and non-dynamic sets, with non-dynamic nodes represented algebraically based on dynamic variables. The accurate estimation of dynamic node states has been devised under a condition of partial observability; which could be the case under contingencies. It is crucial, however, to first determine the specific operating CPC scenario. Building on this, in <cit.> we characterized any CPC scenario as a random switching event and subsequently developed the SHS model of MPS as an RSLS.
In this regard, the list of all normal and contingency operation scenarios must be formed offline and the output of the system for each scenario is calculated for any time interval instant and compared to the measured outputs of the system online to detect the operating scenario accordingly.

Given the diverse origins and characteristics of contingencies, control and mitigation efforts must be managed by different systems within the MPS once they are identified. This paper categorizes contingencies according to the newly proposed SHS model. However, the extensive variety of CPC scenarios introduces significant computational challenges, rendering it impractical to assess every possible scenario. Therefore, our focus is on identifying the critical effects associated with each CPC class. We propose specific evaluation criteria designed to prioritize the assessment of the consequences of various contingencies based on their classification.


The main contributions of this paper are summarized as follows:

    
  * We develop a comprehensive SHS modeling approach for MPS based on coupled real and reactive power dynamics of continuous physical states and stochastic discrete models of contingency events. By selecting local internal variables and network coupling variables (active and reactive powers) as the continuous system states, our model adeptly captures various local and network contingencies through system dynamics.
    
  * We introduce a CPC identification and classification framework using the proposed SHS model, categorizing contingencies into three distinct types: physical, control, and measurement contingencies. 
    
  * We provide detailed mathematical representations for each CPC class, reflect specific characteristics and dynamics of each class of contingency, and evaluate the impacts of the classes on MPS dynamics.  
    
  * We simulate two real-world power systems, the Flores and Sao Miguel islands in the Azores Archipelago, to demonstrate and evaluate the SHS model’s effectiveness in handling different CPC classes. 


The remainder of the paper is structured as follows: Section <ref> provides an overview of the necessary conditions of CPC detection using SHS power system models along with the associated detection algorithm. Section <ref> discusses the coupled hybrid modeling of MPS dynamics. Section <ref> details various CPC classifications and their evaluations. In Section <ref>, we illustrate the analysis of different contingencies using two power system networks on the Flores and Sao Miguel islands. Finally, Section <ref> summarizes the main conclusions of this study.




§ CONTINGENCY IDENTIFICATION BASED ON STOCHASTIC HYBRID SYSTEM MODEL



SHS models, as discussed in <cit.>, offer a framework to represent systems experiencing stochastic discrete events within continuous dynamic operations. 
Typically, dynamics of the systems that are operating around specific equilibrium points, are described using their small signal linearized models. 
As highlighted in Section <ref>, contingencies in MPS are considered stochastic discrete events. Meanwhile, CPC detection takes place within certain time intervals where the system remains fixed within that time interval. Therefore, <cit.> suggests modeling MPS as RSLS. In this context, when the system experiences a CPC, it switches into another predefined operation scenario that captures the effect of that particular CPC. 
In general, RSLS systems are presented as 

    ẋ   = A(α_k) x + B(α_k) u 
    
        y    = C(α_k) x

where (<ref>) and (<ref>) represent the system state space model and outputs, respectively, for t∈ [kτ,(k+1)τ). τ could be chosen as the CPC management time limit in MPS. Moreover, α is the operational scenario of the system selected from a finite set of all normal and CPC operational scenarios denoted by α∈𝒮={1,2,3,…,m} <cit.>. Note that within each time interval, A(α_k), B(α_k), and C(α_k) are fixed matrices <cit.>. If the system suffers from observability, the observable portion of the states is derived based on <cit.> and the measurements are used to estimate the values of the observable portion of the system. Since contingencies change the configuration of a section of the MPS, the state space models of α∈𝒮 share some common eigenvalues. Hence, two different probing input approaches based on input mode and magnitudes are introduced by <cit.> to make different scenarios detectable. In <cit.>, a mode-based input is applied for hidden contingency detection. In this approach, the Laplace transform of the inputs is assumed to be a coprime fractional function U(s)=a(s)/b(s) where a(s) and b(s) do not have any common pole-zero pairs and b(s) has at least one root with any multiplicity q ≥ 1, that is not among the system poles (A(α) eigenvalues), under any contingency scenario being studied.



The contingency detection algorithm outlined in <cit.> operates within two distinct time frames, dividing each interval into two segments. The initial segment, t∈ [kτ, k τ+τ_0) with τ_0≪τ, is focused on identifying the current operational scenario α_k, corresponding to the contingency detection in MPS. Subsequently, the second segment, t∈ [kτ+τ_0, (k+1) τ), is dedicated to state estimation of MPS. The algorithm during the first segment is briefly described below. 

Offline process:

    
  * List all of the normal and contingency scenarios as 𝒮={1,2,…,m}.
    
  * Derive the transfer functions of the MPS under all contingencies by G_i(s)=C(i)(sI-A(i))^-1B(I), ∀ i∈𝒮.
    
  * Design the probing input for distinguishing the contingencies that share common eigenvalues.
    
  * Calculate the zero state response of the system to the probing input for all contingencies based on the sampling period of the measurement systems, t_s, as: y_i^input(l)=(G_i u)(kτ + l t_s), ∀ l=1,2,…, N_0, ∀ i∈𝒮.

Note that all functions described above are deterministic; they can be computed offline and stored for system operation. Subsequently, the outputs computed must be compared online with the measured values to detect α_k ∈𝒮. Owing to the zero initial assumption made in Step 4 of the offline process, it is necessary to estimate the total output value for comparison against the measured data values. Below are the online steps for the CPC detection algorithm.

Online process:

    
  * Collect the measurement data as y(k τ +l t_s) for l=1,2,…, N_0.
    
  * Since measured outputs are not under zero initial conditions, computed outputs cannot be compared to the measured outputs. Thus, the net initial output is derived from the measurement values as y_i^net=y(k τ +l t_s)-y_i^input(l).
    
  * The initial states of the system are driven based on the system response for the net output and Grammian matrices <cit.> of the system scenarios by x̂^i_k= Γ_i^-1 Y_i, i=1,…, m. 
    
  * The total estimated output as a combination of the initial state and probing input is calculated as ŷ_i(l t_s)= C(i)e^A(i)l t_s t_sx̂^i(kτ)+ y^input_i(l t_s).
    
  * Norm of the error between estimated and measured outputs are calculated for different scenarios as ε_i = || ŷ_i(l)-y(kτ+l t_s)||, i=1,…, m.
    
  * Determine α_k:	α̂_k = min_i=1,…, mε_i.

 
The effectiveness of this algorithm for systematic CPC identification relies on a comprehensive model that can accurately capture the effects of various types of contingencies within the state space model.




§ COUPLED HYBRID MODELING OF MPS DYNAMICS



In this section, our goal is to develop an SHS model that utilizes the internal states of power network nodes and their coupling variables. This model aims to enhance the detection of hidden CPC as outlined in Section <ref>, and to analyze the system's stability and reliability under different CPC scenarios. The small signal models in existing literature provide a foundation for approximating the linearized dynamic behavior of an interconnected MPS. It is essential to further develop the small signal model into an RSLS. This advancement will allow us to describe the dynamics of system states accurately, encompassing not just fluctuations in load and generator outputs, but also various types of contingencies.

Different nodes of MPS consist of various types of sources and devices with an associated set of internal states. Accordingly, <cit.> suggests representing the dynamics of different machines based on their electromechanical and electromagnetic parts. The small signal dynamic models of various synchronous and asynchronous equipment such as diesel generators, wind turbines, and hydroelectric turbines are presented in <cit.>. 
Meanwhile, it is crucial to note that parameters indicating the dynamics of different sources are affected by the local contingencies. Thus, we can develop the local SHS model of the MPS devices by deploying the effect of various local CPC scenarios. 
As an illustration, consider the dynamics of the electromechanical equations of a synchronous generator encapsulating a second-order differential equation in terms of the phase angle where δ̇=ω. Consequently, the linearized state space representation can be formulated as

    d/dx[ δ; ω ] =
        [          0          1;          0 -D(α)/M(α) ][ δ; ω ] + [ 0; α ](P_m - P_D)








where, ω is frequency, and M(α) and D(α) are the inertia and damping coefficients within the particular CPC scenario respectively. P_m is the mechanical input to the generator, and P_D is the electrical power output of the generator. Also, (<ref>) represents the SHS model of electromagnetic dynamics of a synchronous generator. 

    d/dx[  V_R; e_fd; e'_q;  V_F ]   =
        [                 -1/T_a -K_a(α) K_f(α)/T_a T_f            -K_a(α)/T_a             K_a(α)/T_a;                  1/T_e   -(K_e(α)+S_e(α))/T_e                      0                      0;                      0                  1/T_d                 -1/T_d                      0;                      0           K_f(α)/T_f^2                      0                 -1/T_f ][  V_R; e_fd; e'_q;  V_F ]
         + [ K_a(α)/T_a;          0;          0;          0 ]V_G^ref+
        [                       0;                       0; -(x_d(α) - x'_d(α))/T_d;                       0 ] i_d

where V_R, e_fd, e^'_q, and V_F are the regulator voltage, field excitation, machine voltage behind the direct transient impedance, and feedback voltage respectively <cit.>. Furthermore, different control gains (K_a, K_e, K_f, etc) and electric parameters (x_d, x'_d) are functions of α.


In general, the SHS model for both the electromechanical and electromagnetic components of MPS nodes can be formulated as follows:

    d x_LC^P/dt =
    A_LC^P(α) x_LC^P + B^P(α) u_P + D^P(α) P_D


    d x_LC^Q/dt =
    A_LC^Q(α) x_LC^Q + B^Q(α) u_Q +D^Q(α) i_d

here, x_LC^P and x_LC^Q represent the internal state variables of the electromechanical and electromagnetic sections of the dynamic devices, respectively. P_D denotes the real power output and i_d represents the reactive current output. Notably, for energy storage devices, P_D may alternate between a positive value (indicating discharging mode) and a negative value (charging mode).
u_P is the control input for real power and u_Q is the control input for reactive power. In addition, A_LC^P(α), A_LC^Q(α), B^P(α), B^Q(α), D^P(α), D^Q(α) are the local system, input, and disturbance matrices of the electromechanical and electromagnetic parts under the CPC scenario α, respectively. 

Contingencies within an MPS network, such as line outages, represent a critical category that the SHS model must accurately capture. Typically, MPS networks are characterized by a set of power flow equations. References <cit.> have developed these equations to describe the power flow from dynamic nodes, taking into account their internal states and disturbances at load buses, as follows:

    [ P_D; i_d ]=
    [ K_PQ_11 K_PQ_12; K_PQ_21 K_PQ_22 ][ δ_D; V_D;     ]+D_PQ[ P_L; i_L ]
  
where K_PQ is the coupling matrix of the interconnected dispatchable nodes derived from the power flow Jacobian matrix <cit.>.

Hence, the dynamics of the coupling variables between dispatchable devices can be written as:

    d/dt[ P_D; i_d ]=	[ K_PQ_11 K_PQ_12; K_PQ_21 K_PQ_22 ][        ω_D; d/dt e_q^';            ]
    +D_PQd/dt[ P_L; i_L ].
 
The dynamics of P_D and i_d from (<ref>) demonstrate a linear relationship, suggesting that selecting P_D alone as the coupling variable is sufficient. The choice of active power P_D to represent the dynamics of the coupling variables in the network is based on its intrinsic role in global frequency control.

Since K_PQ is derived from power flow equations, it enables the SHS model to encompass any CPC impacting power networks. Consequently, integrating the interconnected dynamics of each node in the SHS model through the combination of (<ref>), (<ref>), and (<ref>) leads to:


    [                                                                                                                                                          Id/dt[ x_LC^P; x_LC^Q;    P_D;  ] =
    [        A_LC^P(α)                0           B^P(α);                0        A_LC^Q(α)                0; K_PQ_11(α) S_ω_G                0                0 ][ x_LC^P; x_LC^Q;    P_D ];                                                                       +	
    [ B^P(α)      0;      0 B^Q(α);      0      0 ][ u_P; u_Q ]+ 
    [       0;       0; D_PQ(α) ][  d/dtP_L; d/dt	i_L ] ]


𝒮_ω_G and 𝒮_e'_q vectors specifically defined to include a value of 1 at the index corresponding to the respective internal state variables and 0 at all other indices. The 0's represent vectors of zeros, sized appropriately.  Additionally, I = I - [                 0                 0                 0;                 0                 0                 0;                 0 K_PQ_12(α) S_e'_q                 0;                   ] defines a structurally invertible
matrix. Due to its sparse structure, by applying a Taylor series expansion, we obtain 
I^-1 = I + [                 0                 0                 0;                 0                 0                 0;                 0 K_PQ_12(α) S_e'_q                 0 ].

Finally, we can rewrite (<ref>) as:

    [                                                                                                                                                                 ẋ =[        A_LC^P(α)                0           B^P(α);                0        A_LC^Q(α)                0;           𝒮_1(α) 𝒮_2(α) A_LC^Q(α)              ξ_i ]
    x;                                                                                                                                                                                 + [        B^P(α)             0;             0        B^Q(α);             0 S_2(α) B^Q(α) ]
    u+ 	[       0;       0; D_PQ(α) ][ d/dt	P_L; d/dt	i_L ] ]

where 𝒮_1(α)= K_PQ_11(α) S_ω_G and 𝒮_2(α)= K_PQ_12(α) S_e'_q. 
Note that, ξ_i is added to address the structural singularity caused by the slack bus <cit.> and it is defined as

    ξ_i = 
        1/10M_i   ; if  i  is the slack bus
    
             0    ; otherwise


Remark 1: Power electronic converter-based sources, such as wind turbines and photovoltaic (PV) systems, are increasingly integrated into power system and typically modeled in the d-q frame. These sources are connected to the system through RL filters, necessitating the modeling of inductance current and capacitor voltage as system state variables. Consequently, linearization techniques can be employed to represent the dynamics of the coupling variables P = v_C_d· i_L_d + v_C_q· i_L_d. Integrating these resources into the SHS model will be explored in our future research.

Define the state vector x_i = [ x_LC_i^P x_LC_i^Q    P_D_i ]^T for the ith dynamic node in an MPS consisting of n buses. The corresponding fully coupled SHS model is as follows:

    [                                                                                                             d/dt[ x_1;   ⋮; x_n ]                =
    [ A_11(α_i)         ⋯ A_1n(α_i);         ⋮         ⋱         ⋮; A_n1(α_i)         ⋯ A_nn(α_i) ][ x_1;   ⋮; x_n ];                                                                                                                                                  +
    [ B_11(α_i)         ⋯ B_1n(α_i);         ⋮         ⋱         ⋮; B_n1(α_i)         ⋯ B_nn(α_i) ][ u_1;   ⋮; u_n ];                                                                                                                                   +
    [ D_11(α_i)         ⋯ D_1m(α_i);         ⋮         ⋱         ⋮; D_n1(α_i)         ⋯ D_nm(α_i) ][ d/dt L_1;        ⋮; d/dt L_m ] ]


    [                                                                                                                                                           [ y_1;   ⋮; y_n ] =	
    [ C_1(α_i)        0        …        0;        0 C_2(α_i)        …        0;        ⋮        …        ⋱        ⋮;        0        0        … C_n(α_i) ][ x_1;   ⋮; x_n ].;                                                                                                                                                                             ]

where diagonal terms, A_ii, represent the system matrix of the i^th dynamic node


    A_ii(α)=
    [                          A_LC_i^P(α)                                    0                            B^P_ii(α);                                    0                         A_LC_i^Q (α)                                    0;                 K_PQ_11^ii(α)S_ω_G^i K_PQ_12^ii(α)S_e'_q^i    A_LC_i^Q(α)                                  ξ_i ].
 The off-diagonal terms A_ij represent the coupling matrix between nodes i^th and j^th

    A_ij(α)=
    [                                   0                                   0                                   0;                                   0                                   0                                   0;                K_PQ_11^ij(α)S_ω_G^j K_PQ_12^ij(α)S_e'_q^j  A_LC_j^Q (α)                                   0 ],

here K_PQ^ij means choosing the entries from ith row and jth column of the relative submatrix of K_PQ. 

Additionally, the input signal (u_i) and control matrix (B_i) of the i^th node is defined as follows:

    u_i=[   ω_i^ref; V_D_i^ref ]

and

    B_ij(α)=[                    B^P_ij(α)                            0;                            0                    B^Q_ij(α);                            0 K_PQ_12^ij(α)S_e'_q^j B^Q_ij ].


Equation (<ref>) outlines the SHS model for the measurement and monitoring system of the MPS. While joint observability is crucial for the stable and efficient operation of MPS, the vast number of components and their complexity make it challenging to maintain comprehensive observability. With the increasing integration of DERs, relying solely on redundant measurements becomes impractical. Therefore, it is imperative to identify an efficient set of observations that ensures joint observability with a manageable number of well-structured measurement devices. Further discussions on designing an appropriate measurement and monitoring system for the proposed SHS model are included in Section <ref>.




§ CYBER-PHYSICAL CONTINGENCY CLASSIFICATION AND EVALUATION




In this section, we propose a CPC classification and evaluation based on the fully coupled SHS model, characterize the mathematical underpinnings of each CPC class, and assess their critical impact on the MPS. 
The contingencies are categorized into three distinct groups: physical contingencies (𝒮_p), control network contingencies (𝒮_c), and measurement contingencies (𝒮_m), in addition to scenarios representing normal operations (𝒮_normal). Notably, 𝒮_p∪𝒮_c∪𝒮_m∪𝒮_normal = 𝒮, while 𝒮_p∩𝒮_c∩𝒮_m = ∅. For clarity and simplicity, we define the normal operation scenario as a singleton, denoted by  𝒮_normal = {1}.




 §.§ Physical Contingencies


Physical contingencies are unplanned events that range from network disturbances like transmission line outages to localized impacts such as physical damage to components or electrical failures.
The primary challenge that MPS faces under physical contingencies is balancing supply and demand power to ensure reliability, security, and cost-effectiveness.
These contingencies, particularly line outages, disrupt power flow, leading to various consequences like voltage instability, equipment damage, and potential blackouts. Rapid identification and mitigation are crucial for effective system operation. 

The impact of a physical contingency on the SHS model can manifest either locally or across the network. As per (<ref>), if α represents a local contingency affecting dynamic nodes, then we have:

    A(α)=∑_i=1^m A(i) 1_{α=i},


    B(α)=∑_i=1^m B(i) 1_{α=i},

where 1_α is the indicator function of the CPC: 1_α =1 if α is true; and 1_α =0, otherwise.
Meanwhile, during network contingencies in addition to (<ref>) and (<ref>), we have 

    D(α)=∑_i=1^m D(i) 1_{α=i},

since D(α) is a function of power flow equations.  

Physical contingencies can lead to alterations in power flow, forcing the system to adjust to a new steady state often without immediate notification to operators. Traditional tools, such as contingency analysis, are commonly used to manage risks associated with these events. However, since these methods require solving power flow equations during system operation, they are not suitable for real-time mitigation <cit.>. In contrast, leveraging the SHS model for CPC detection provides operators with sufficient time to implement effective mitigation strategies. Our approach also involves defining specific types and severity of physical CPC scenarios, which inherently improves the accuracy of CPC localization compared to traditional methods.

If, during a particular CPC, the system matrix A(α) is not full-rank, then A(α)x + B(α)u = 0 lacks a unique solution, indicating that the power system does not have a feasible power flow solution under these conditions. Therefore, it is crucial to undertake appropriate control actions, such as integrating new energy sources and load shedding to maintain frequency stability during contingencies.





 §.§ Control Network Contingencies

Control network contingencies involve unexpected disruptions that impair the functionality of control signals within MPS. These disruptions can arise from various sources such as cyber-attacks, system anomalies, human errors, or inaccuracies in signal transmission. Manifestations of these contingencies can include malfunctions, denial of service (DoS) attacks, random operations, or oscillatory behaviors <cit.>, all of which may lead to loss of control, system instability, or even cascading failures <cit.>. Given the significant potential impact of control network contingencies, their rapid and accurate identification is critical. The proposed CPC classification and evaluation offers an effective method for addressing these challenges. 

Mathematically, control network contingencies impact the fully coupled SHS model in the following manner:

    B(α)=∑_i=1^m B(i) 1_{α=i}.

Specifically, disruptions within the control system, or failures at a specific local node i, result in alterations to the diagonal entries, denoted as B_ii(α). Similarly, interruptions in wireless control signals between nodes i and j lead to changes in the off-diagonal entries represented as B_ij(α).

A significant challenge posed by this class of contingencies is the loss of controllability, where the ability to stabilize the system during CPC may be compromised. Consequently, analyzing the controllability matrix, 

    𝒞(α) = [     B(α)   A(α) B(α)  … A(α)^n-1B(α)     ],
is crucial for developing emergency response strategies for unstable scenarios. This analysis is essential for effective MPS planning and preparedness




 §.§ Measurement Network Contingencies

The measurement network is crucial for state estimation and operational management in MPS. Therefore, measurement network contingencies can significantly undermine the system's ability to accurately estimate states during operation. This accuracy is vital for monitoring the system to ensure the stability and reliability of MPS. A measurement contingency scenario can lead to the system's observer receiving inaccurate measurements, potentially triggering unexpected behaviors and leading to severe consequences. It is therefore critical to identify measurement network contingencies promptly and mitigate their impact on the system's observability.


To address this issue, <cit.> proposes an observer design for each system scenario, where the estimation of the observable portion of the system is based on the available measurements. Measurement network contingency mathematically affects the following segments of the SHS model:

    C(α)=∑_i=1^m C(i) 1_{α=i};

The observability matrix, defined as  

    𝒪(α) = [           C(α)^T     (C(α)A(α))^T                … (C(α)A(α)^n-1)^T ]^T
is critical in designing the measurement system and implementing backup sensors. This matrix serves as a key criterion for ensuring comprehensive system observability despite potential measurement network contingencies.

A significant distinction between measurement network contingencies and other types of contingencies is that measurement network contingencies can make certain system states unobservable, even if the system was initially fully observable. Therefore, it is crucial to identify which parts of the system remain observable under each measurement network contingency scenario <cit.>. Detection of measurement network contingency scenarios must be based on the observable portion of the network specific to that failure scenario. Additionally, only the observable states of the system are available for state estimation and contingency detection in subsequent time intervals until the failure is resolved.
The comprehensive framework for CPC classification and evaluation is outlined in Algorithm 1.

  




§ SIMULATION


To evaluate the effectiveness of the proposed fully coupled SHS model in identifying and evaluating the impacts of various CPC classes, we apply it to two real-world systems. Firstly, we study the Flores Island network, part of the Azores Archipelago in Portugal. This network features a 15 kV radial electric power distribution system encompassing 46 buses, which collectively handle a power demand of 2MW. This system is analyzed under various CPC scenarios covering all three categories defined in the CPC classification and evaluation framework. Next, we extend our examination to Sao Miguel, the largest Island in the Azores Archipelago. Detailed information about these networks is provided in <cit.>. The analysis involves assessing the system's response to a small probing input across different scenarios. To address the conditions necessary for optimal control, the pole placement technique is employed to strategically shift the system's poles.



 §.§ Flores Island

The electrical demand of Flores Island is met by three modest-scale power facilities, which include diesel (node 1), wind (node 19), and hydroelectric (node 46) power plants. These facilities contribute approximately 50%, 15%, and 35% of the total demand, respectively. Assuming that the load nodes are non-dynamic, the fully coupled SHS model for the entire network can be constructed based on the dynamics of these three power-generating nodes.
 
For the fully coupled SHS model, the diesel plant, equipped with a PI governor control system, is designated as the slack bus. Meanwhile, the hydro plant features a proportional governor control (GC), and the wind plant employs synchronous generators. Consequently, the states of each dynamic node are defined based on their respective control systems and roles within the network.


    x_D    = [  ω_G_1    m_B    P_c  V_R_1 e_fd_1 e'_q_1  V_F_1    P_1 ]^T 
     
        x_W    = [  ω_G_19 e'_q_19    P_19 ]^T 
    
        x_H    = [  ω_G_46       q       v       a  V_R_46 e_fd_46 e'_q_46  V_F_46    P_46 ]^T.

The detailed parameters for the SHS model of the power plants during normal operation are documented in <cit.>. Consequently, the fully coupled SHS  model is constructed for the state vector x = [ x_D x_W x_H ]^T by integrating (<ref>), (<ref>), and (<ref>) into the RSLS model (<ref>). During the normal operation, A(1) ∈ℝ^20 × 20 represents the coupled dynamics of the entire network and is full-rank. Additionally, under the standard assumption of control inputs for angular frequency ω and transient voltage e'_q for all three power resources, the matrix B(1) ∈ℝ^20 × 6 ensures that the network is controllable (full-rank 𝒞(1)). Similarly, with the assumption of PMU and voltage measurements at these resources, observability (full-rank 𝒪(1)) is achieved with C(1) ∈ℝ^6 × 20.



    The wind plant is considered a synchronous machine connected to the grid, and its electromechanical parts could be presented as
    
    d ω_G/dt   = 1/2H_WP_m-D_W/2H_Wω_G-1/2H_WP_G 
    
            P_m    = -K_m_Wω_G

    The mechanical power is denoted by P_m, while H_W and D_W represent the inertia and damping coefficients, respectively. Additionally, K_m_W is the proportional gain of the pitch control system. By adjusting the value of K_m_W, the dynamics of the system can be significantly influenced.  Based on the data in <cit.>, (<ref>) is derived as
    
    ω̇_G= -4.294 ω_G -2.146 P_G.

    The electromagnetic component of the synchronous generator is described by the differential equation
    
    d V_G/dt   = -1/T_d_WV_G -(X_d_W-X'_d_W)/T_d_WV^ref_G 
         = -1.513 V_G -37.986 V^ref_G.

    Furthermore, the local electromechanical states of the diesel generator and hydro plant are defined by x^PLC_D and x^PLC_H, according to equations (<ref>) and (<ref>) respectively. The local electromagnetic states of both plants, x^Q_LC_D and x^Q_LC_H, are detailed in (<ref>). Based on these definitions, the local system matrices for the diesel and hydro plants of the Flores generators during normal operation are computed as follows:
    
    A^P_LC_D   =
            [  -0.0110   0.4413        0; -2222.22   -1.667   -66.67;      -10        0        0 ]
    
            A^Q_LC_D   =
            [      -5 -192.31    -125     125;  1.5281 -1.6886       0       0;       0  0.4255 -0.4255       0;       0 -2.3669       0 -1.5385 ]
      
            A^P_LC_H   =
            [ -0.0232  0.6438       0  -0.352; -0.2778 -1.3889       0    0.25;       0       0    -0.5    0.03;  -16.67       0   16.67 -117.67 ]
     
            A^Q_LC_H   =
            [      -20 -8888.89    -8000     8000;  -1.1111    -1.15        0        0;        0   0.2857  -0.2857        0;        0    1.235        0   -1.111 ].

     Also, B_P_D= [       0 -0.4433       0 ]^T,  B_Q_D=[       0       0 -3.2154       0 ]^T,
    
    
        B_P_H= [ -0.2316       0       0       0 ]^T, and  B_Q_H=[       0       0 -0.5823       0 ]^T.
         Additionally, the Jacobian matrix can be derived from the power flow equations of the network. Although the load nodes are assumed to be non-dynamic, eliminating the need for the matrix D_PQ, the matrix K_PQ, which pertains to three generator nodes, has been derived from the Jacobian matrix using the data provided in <cit.> as follows:
    
    K_PQ_11   =
            [  126.39   -1.41 -124.98;   -1.35    1.35       0; -125.02       0  125.02 ]
    
             K_PQ_12   =
            [  127.51 -2.4883 -125.02;   -2.52    2.49       0; -124.98       0 -125.02 ]
    
             K_PQ_21   =
            [ -127.496     2.49   125.02;     2.51    -2.51        0;   124.98        0  -124.98 ]
    
            K_PQ_22   =
            [   126.36    -1.41  -124.98;    -1.35     1.41        0;  -125.02        0 124.9825 ]

    where the first, second, and third rows and columns represent network dynamics of the diesel, wind, and hydro generators respectively.


Line outages represent a ubiquitous physical contingency that MPS encounter. Given the impracticality of measuring every line in an MPS, especially distribution networks, identifying and localizing such contingencies poses a formidable challenge for the operators. We consider 𝒮_P to include all, N-1, N-2, and N-3 line outage scenarios. First, the rank of A(α);  ∀α∈𝒮_P must be calculated. We realize that in scenarios involving 1-17, 17-18, and 1-46 for any α∈𝒮_P contingencies, A(α) is not full-rank, resulting in an infeasible power flow. For instance, the frequency instability of the wind turbine is observed in Fig. <ref> due to imbalanced generation and demand in Scenario 1 (line 1-17 outage).

Although other scenarios do not lead to instability, they are aimed to be detected by their response to the probing inputs. 
Therefore, the probing input as a step function U(t) is implemented for 0.5 seconds by the inputs of the system to the network. 
For example, in Scenario 2, we examine the line outage between nodes 18 and 35 as a hidden contingency scenario.
The frequency and voltage values at three generation nodes for Scenario 1 and 2 are illustrated in Fig <ref> and <ref> respectively. 
Even though the difference in response compared to normal operation appears to be minor for Scenario 2, we illustrate the eigenvalues of the system matrices for these scenarios in Fig. <ref> which shows that while some eigenvalues are common between the Scenarios, differences between others are noticeable, making them detectable through precise measurements.
Notably, the eigenvalue associated with Scenario 1 is located at the origin, as observed in Fig. <ref> which supports the instability conclusion of the system discussed earlier.





Control contingencies are the other class of contingencies in Section <ref> which affects the system’s control inputs. A common example is a DoS attack, where control over one or more inputs is compromised. Denoting all single DoS scenarios as 𝒮_C, our analysis shows that loss of control over the excitation of any generator renders the system uncontrollable. Conversely, losing frequency control from individual sources might allow other sources to compensate and maintain control of the system.  




Let us consider two control contingency scenarios: Scenario 3 (ω_H DoS) and Scenario 4 (e'_q_W DoS). Scenario 3 leads to frequency instability. Therefore, it is a major CPC that must be mitigated immediately. With rapid identification of this CPC, we define the desired poles, Λ^*(α), with a constant shift of current poles, Λ(α), (i.e. Λ^*(α)=Λ(α)-20) and use the pole placement technique to stabilize the system in Fig. <ref>.


In contrast to the earlier scenario involving control contingencies, pole placement feedback cannot be applied to Scenario 4 (e'q_W DoS) because of controllability limitations. Fig. <ref> demonstrates the frequency dynamics of Scenario 4. Although the system maintains stability, the lack of controllability prevents any enhancement of the operational conditions.












The final category of CPC involves measurement network contingencies, where the sensor measurements (y(α)) diverge from the actual values. Given that both state estimation and control decisions rely on these measurements, it is crucial to determine whether any measurement system is failing. Moreover, if a measurement network contingency results in a loss of observability, full state estimation across all 20 states becomes infeasible. We have therefore assessed the impact of measurement failures on the system's observability, as detailed in Table <ref>. Our analysis reveals that only failure of the ω_H sensor leads to a loss of observability. Consequently, implementing redundant measurement systems at this node is beneficial.

























 §.§ Sao Miguel Island

Sao Miguel is modeled as 15 prosumers in <cit.> where each prosumer acts as a dynamic node. The prosumer-based network of the island is shown in Fig. <ref>.
The system's response is captured by the center of inertia (CoI), represented by the following equation: 

    Δ f_CoI = 1/∑_i=1^N M_i∑_i=1^N M_i Δ f_i

This formula effectively aggregates the overall behavior of the system under various CPC scenarios, providing a comprehensive integrated measure of the system performance.
Note that the RSLS output in  (<ref>) can be defined as:

    C(α)=1/∑_i=1^N M_i(α)[ M_1(α) 𝒮_ω_G^1              … M_1(α) 𝒮_ω_G^N ]

where N=15.
Similar to the previous scenario, Sao Miguel features a full-rank system matrix. Additionally, the implementation of frequency control and the measurement of frequency across every prosumer contribute to a controllable and observable operational state.



We begin by analyzing the rank of matrix A(α) for all N-1 line outage scenarios. The results show that A(α) remains full rank under N-1 line outage scenarios. Fig. <ref> shows the frequency dynamics of CoI under normal operations, Scenario 1 (line 1-4), Scenario 2 (line 10-11), and Scenario 3 (line 14-15). While the system maintains stability, the variations in the system's model are distinguishable by the dynamics of CoI.

 

Next, we explore control network contingency scenarios, specifically focusing on ω_G_i DoS events for i=1,2,…,15. Our findings indicate that N-1 and N-2 control contingencies do not compromise the system's controllability. However, scenarios involving N-3 control contingencies lead to a loss of controllability. Table <ref> lists sets of N-3 control contingencies that could endanger system controllability in the event of a DoS. Considering the likelihood of such contingencies arising from anomalies or cyber-attacks, the risk of multiple frequency control DoS contingencies is a major concern.



We examine two control DoS scenarios: Scenario 4 (ω_1, ω_2, ω_4 DoS) resulting in the loss of controllability, and Scenario 5 (ω_1, ω_2, ω_5 DoS) controllability maintained. Although neither scenarios cause instability, in Scenario 5 system performance can be improved using the pole placement algorithm. In other words, adjusting the poles Λ_i^*=Λ_i-20 can improve the frequency response. This adjustment, however, is not feasible in Scenario 4 due to the loss of controllability. Fig. <ref> shows the frequency response of CoI for these scenarios.


Next, we study measurement network contingencies for Sao Miguel. 
Our analysis shows that the system is resilient for N-1 contingencies, while (ω_2, ω_14) is the only N-2 measurement failure contingency that results in a loss of observability. Moreover, among all of the N-3 failure scenarios, 127 scenarios lead to the loss of observability. Due to space limitations, we do not list all of these contingencies here, however, we realize that from all of the N-3 contingencies that lead to unobservability, at least one of the measurements is either ω_G_1, ω_G_2 or ω_G_3. This indicates the necessity of backup measurement systems for these three nodes for reassurance of the system observability.

Figure <ref> demonstrates the CoI's frequency value under three measurement network contingency scenarios: Scenario 6 (ω_2, ω_14 failures),  Scenario 7 (ω_1, ω_2, ω_6 failures), and Scenario 8 (ω_3, ω_6, ω_11 failures). It is observable that although the system is operating under similar conditions, the measurements of CoI are different for various measurement network contingencies making the contingencies detectable. 





§ CONCLUSION



This paper introduces a new fully coupled SHS modeling approach to detect hidden CPC in MPS by combining dynamics of local internal electromechanical and electromagnetic states of dynamic nodes and dynamics of their coupling variables integrated with discrete contingency states. This forms a SHS model. 
We have categorized CPC into three distinct classes: physical contingencies, control network disruptions, and measurement network contingencies.

We study the mathematical representation of different CPC classes in the SHS model and how various types of hidden CPC affect MPS. Accordingly, the impacts of each class of CPC are evaluated for MPS stability, controllability, and observability.  
Two real-world system networks of Flores and Sao Miguel Islands are used to illustrate theoretical results.

The findings of this paper aim to bolster the resilience and robustness of MPS, ensuring effective identification and state estimation against CPC. 
Looking forward, future research will concentrate on integrating machine learning techniques to better identify CPC and assess their uncertain effects on the SHS model. 



ieeetr




 Paper Title: Joint Vehicle Connection and Beamforming Optimization in Digital Twin Assisted Integrated Sensing and Communication Vehicular Networks.








Joint Vehicle Connection and Beamforming Optimziation in Digital Twin Assisted Integrated Sensing and Communication Vehicular Networks
    Weihang Ding Student Member, IEEE, Zhaohui Yang Member, IEEE, Mingzhe Chen Member, IEEE,
 Yuchen Liu Member, IEEE and Mohammad Shikh-Bahaei Senior Member, IEEE

W. Ding and M. Shikh-Bahaei are with Department of Engineering, King's College London, London, UK. (email: weihang.ding@kcl.ac.uk; m.sbahaei@kcl.ac.uk)
Z. Yang is with the College of Information Science and Electronic Engineering, Zhejiang University, Hangzhou, Zhejiang 310027, China, and Zhejiang Provincial Key Lab of Information Processing, Communication and Networking (IPCAN), Hangzhou, Zhejiang, 310007, China (Email: yang_zhaohui@zju.edu.cn). 
M. Chen is with the Department of Electrical and Computer Engineering and Frost Institute for Data Science and Computing, University of Miami, Coral Gables, FL 33146 USA (Email: < mingzhe.chen@miami.edu)>. 
Y. Liu is with the Department of Computer Science, North Carolina State University, Raleigh, NC, 27695, USA (Email: yuchen.liu@ncsu.edu).



This work is supported in part by National Key R&D Program of China (Grant No. 2023YFB2904804), Young Elite Scientists Sponsorship Program by CAST 2023QNRC001,  Zhejiang Key R&D Program under Grant 2023C01021, the Fundamental Research Funds for the Central Universities, K2023QA0AL02


This work is supported in part by the U.S. National Science Foundation under Grants CNS-2312138, CNS-2312139, and CNS-2332834.


    
===================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================



§ ABSTRACT 


This paper introduces an approach to harness digital twin (DT) technology in the realm of integrated sensing and communications (ISAC) in the sixth-generation (6G) Internet-of-everything (IoE) applications. We consider moving targets in a vehicular network and use DT to track and predict the motion of the vehicles. After predicting the location of the vehicle at the next time slot, the DT designs the assignment and beamforming for each vehicle. 
The real time sensing information is then utilized to update and refine the DT, enabling further processing and decision-making. In the DT, an extended Kalman filter (EKF) is used for precise motion prediction. This model incorporates a dynamic Kalman gain, which is updated at each time slot based on the received echo signals. The state representation encompasses both vehicle motion information and the error matrix, with the posterior Cramér-Rao bound (PCRB) employed to assess sensing accuracy. We consider a network with two roadside units (RSUs), and the vehicles need to be allocated to one of them. To optimize the overall transmission rate while maintaining an acceptable sensing accuracy, an optimization problem is formulated. Since it is generally hard to solve the original problem, Lagrange multipliers and fractional programming are employed to simplify this optimization problem. To solve the simplified problem, this paper introduces both greedy and heuristic algorithms through optimizing both vehicle assignments and predictive beamforming. The optimized results are then transferred back to the real space for ISAC applications. Recognizing the computational complexity of the greedy and heuristic algorithms, a bidirectional long short-term memory (LSTM)-based recurrent neural network (RNN) is proposed for efficient beamforming design within the DT. Simulation results demonstrate the effectiveness of the DT-based ISAC network. Notably, the LSTM-based RNN method achieves similar transmission rates as the heuristic algorithm but with significantly reduced computational complexity.







Integrated sensing and communication, vehicular network, digital twin.





§ INTRODUCTION


Digital twins (DTs) that serve as virtual representations of physical objects, systems, or processes, offering detailed and dynamic simulations of their real-world counterparts, has gained significant attention from both academia and industry. 
Specifically, through constantly learning and updating from the real space, DTs accurately characterize the working conditions and locations of physical entities, enabling precise predictions of future events. Therefore, DTs have been applied for widespread applications in edge computing <cit.>, the Internet of things (IoT) <cit.>, cyber-physical systems <cit.>, and vehicle networks.



DT offers a dynamic virtual representation of vehicular network systems, mirroring the physical state, processes, and real-time systems of vehicles or networks. With various sensors installed on the vehicle and network infrastructure, as well as broader contextual information such as traffic conditions and environmental factors, DTs are capable of simulating, predicting with data, and optimizing the performance and maintenance of the physical counterparts <cit.>. DT has been demonstrated to be effective in traffic management and optimization <cit.>. By simulating traffic flows and vehicle interactions within a DT of a vehicular network, cities and organizations can optimize traffic patterns, reduce congestion, and enhance safety. Additionally, by simulating various crash scenarios and cyber-attack simulations, DT contributes to improving vehicle safety features and enhancing resilience against attacks on vehicular networks <cit.>. Furthermore, DT can play a crucial role in monitoring the real-time health and performance of vehicles, predicting maintenance needs <cit.>, and forecasting battery charging requirements <cit.>. Given these applications and the rising trend of developing autonomous vehicles, the collaboration between DTs and integrated sensing and communication (ISAC) in the sixth-generation (6G) IoE holds great promise <cit.>.





The integration of sensor technologies with advanced communication (ISAC) systems <cit.> has been attracted great attention in recent years. To realize ISAC, the authors in <cit.> proposed a millimeter wave (mmWave) system to support both positioning and downlink broadband services and analyzed the trade-off between positioning accuracy and communication rate. The authors in <cit.> explored this trade-off in a similar system, by splitting the overall time for beam alignment and data transmission respectively. The optimal overhead in a multi-user scenario was designed in <cit.>, balancing the communication and sensing performances. The above works <cit.> all focused on a scenario where the mobility of the users is limited. For high-speed vehicles, for example, vehicles that are driven on highways, the delay induced by the overhead is intolerable since it will lead to outdated motion prediction. To reduce the delay, dual-functional radars are proposed, that is, both radar sensing and communication are achieved with one dual-functional device and the same signal <cit.>. In <cit.>, a single radar waveform has been used for both sensing and carrying information. This scheme is a low-data-rate scheme because the information bits are transmitted by selecting either the down-chirp waveform or the up-chirp waveform. Due to inherent correlations, the sensing information can be further used for channel estimation <cit.>.



The innovation of the fifth-generation (5G) technology, which exploits both massive multiple-input multiple-output (mMIMO) antenna arrays and the mmWave spectrum, provides the opportunity for reliable ISAC systems <cit.>. Compared to MIMO, mMIMO deploys a much larger array. It has been shown in <cit.> that if the transmitter uses an infinite number of antennas to serve a limited number of users, fast-fading tends to vanish and the channels tend to be orthogonal. In other words, the system can be viewed as interference-free when the vehicular network is relatively sparse. The mMIMO technique can be used to compensate for the excessive path-loss of the mmWave signals and formulate “pencil-like" beams to concentrate the signal power in the desired direction <cit.>. The beam-squint effect, induced by the increasing number of antennas, is harnessed in conjunction with the beam-split effect for Joint Channel Assignment and Scheduling (JCAS) in the mmWave/terahertz (THz) band <cit.>. Therefore, mMIMO and mmWave are an excellent combination to achieve high resolution and high rate and can support ISAC in the sixth-generation (6G) network <cit.>. In addition, it is also possible to apply positioning reference signals (PRSs) in 5G for sensing, positioning, and communication <cit.>.



Previous research primarily focused on improving the performance of ISAC systems by designing dual-functional transmit signal waveforms. In <cit.>, the authors designed adaptive integrated waveforms that improved both the mutual information of the impulse response and the data rate. In <cit.>, the authors proposed a scheme for embedding digital information into radar signals and designed a low-complexity receiver to recover the information. In <cit.>, the authors designed a data-embedded multi-subband quasi-perfect waveform for ISAC in mmWave and low THz band. Besides, phase coding has been applied for ISAC signal design in <cit.> to reduce the effect of noise. More recent works have focused on beamforming design to enhance performance. Extended Kalman filter (EKF) and learning algorithms were respectively applied in <cit.> and <cit.> to jointly optimize sensing and communication performance by exploiting the Cramer-Rao lower bound (CRB). In <cit.>, the authors proposed a message-passing algorithm for estimating vehicle states and applied a Bayesian approach to analyze information content propagating on the factor graph. In <cit.>, the transmitter utilizes the jointly precoded communication and radar waveforms to design dual-functional transmit beamforming, which optimizes both functions simultaneously. Furthermore, the security challenges in ISAC were studied in <cit.>.






Several works have jointly considered DT and ISAC for IoE applications. In <cit.>, the authors focused on resource allocation for ISAC in DT without designing an explicit DT model. Cui et al. <cit.> proposed an ISAC waveform design method without accounting for the movement of objects. In <cit.>, the authors used DT to make intelligent offloading decisions in an unmanned aerial vehicle (UAV)-assisted ISAC network but primarily concentrated on stationary ground targets. Mu et al. <cit.> proposed a Federated Learning (FL)-empowered DT-based communication-assisted sensing network with synthetic aperture radar (SAR) distributed on aircraft. In <cit.>, the authors proposed a smart building DT by sensing power consumption and acting according to environmental factors. However, <cit.> lacks mathematical analysis. Liu et al. <cit.> proposed a DT-based method to intelligently predict the state and trajectory of moving targets, but the targets are maneuvering, and communication is not involved. This paper involves the design of a comprehensive DT for modeling and predicting vehicle movements, then collaboratively assigning vehicles to two roadside units (RSUs) and designing predictive beamforming for the next time slot. A major advantage of applying DT in this work is that it provides a centralized framework for aggregating data from multiple RSUs. This centralized approach to data handling enables more comprehensive analytics, better decision-making, and a unified view of the entire system, as opposed to the isolated sensing information provided by individual RSUs. The DT can make real-time decisions and perform real-time simulations in response to the real-time information collected from the physical environment. Furthermore, considering the allocation of vehicles to different RSUs, it is essential that the information collected from various RSUs be managed jointly. Therefore, decisions must be made using DT rather than by each RSU independently. In our work, we only consider the scenario with two RSUs, but there might be multiple RSUs in reality. However, due to path loss effects, the performance gains obtained from RSU selection are only significant when the distances between the vehicle and the RSUs are similar. Since the coverages of the RSUs can be seen as circles, we only consider the vehicles located near the line connecting the intersection points of two circles. In a system with multiple RSUs, we can simplify the problem by decomposing it into problems between each pair of RSUs. EKF is applied to track the movement of the vehicles in DT. The real-time Kalman gain is updated based on the difference between the received and expected echo signals. Subsequently, the predicted states are corrected using the Kalman gain and the echo signal. Our main focus is on maximizing the transmission rate, with sensing accuracy serving as a constraint. To address this optimization problem, we introduce both a greedy algorithm and a heuristic algorithm. Furthermore, we propose a bidirectional long short-term memory (LSTM)-based recurrent neural network (RNN) to predict the beamforming of the vehicles. The key contributions of this paper are outlined as follows:


    
  * A DT is employed for real-time modeling of vehicles in an ISAC-based vehicular network. The motion model is based on an EKF, and the Kalman gain is updated at each time slot based on the received echo signal to accurately track the real-time location and movement of vehicles. The state of the vehicle includes not only the motion information of the vehicle but also the error matrix, and the posterior Cramér-Rao Bound (PCRB) is used to evaluate the sensing accuracy of the vehicles.
    
  * After the DT obtains the predicted locations of the vehicles, we formulate an optimization problem to optimize the communication performance of the system. Sensing accuracy is considered as a constraint to ensure an accurate DT for the real network. We consider a vehicular network with two RSUs, so the vehicles need to be assigned to one of them. This optimization problem is simplified using Lagrange multipliers and fractional programming. To address this challenge, we propose both a greedy algorithm and a heuristic algorithm. These algorithms are designed to jointly optimize vehicle assignments and predictive beamforming, aiming to maximize the overall transmission rate.
    
  * Due to the high computational complexity of both the greedy algorithm and the heuristic algorithm, we introduce an efficient approach using a bidirectional LSTM-based method to design beamforming for each vehicle in the DT. The bidirectional LSTM is employed to account for the high correlation among the beamformings of adjacent vehicles. Simulation results demonstrate that the LSTM-based method achieves a transmission rate only slightly lower than the heuristic algorithm while significantly reducing computational complexity.


The subsequent sections of this paper are organized as follows. In Section II, we present the system model, including the radar sensing model and the communication model. Section III introduces the proposed DT framework. The optimization problem is formulated in Section IV and solved in Section V. An alternative method utilizing LSTM networks is discussed in Section VI. Section VII presents all relevant simulation results. Finally, Section VIII concludes the whole paper.




§ SYSTEM MODEL






We consider a vehicular network that consists of two RSUs equipped with mMIMO antennas[One can easily extend the considered model with two RSUs to a network with multiple RSUs since we have considered the overlap caused by two RSUs.] serving K vehicles, as shown in Fig. <ref>. At each time slot, the RSUs simultaneously transmit information to the vehicles and sense the position of these vehicles. The sensed information is then forwarded to the DT where a two-dimensional projection of the physical network is constructed. Using matched filtering and EKF, the DT continuously tracks the vehicles, predicts their movements, and optimizes both vehicle assignments and beamforming for the next time slot to achieve high-capacity ISAC. Next, we first introduce the radar sensing model. Then, we explain the communication model including the channel gain and signal-to-interference-plus-noise
ratio (SINR) expression.  



















 §.§ Radar Sensing



Both RSUs are equipped with a mmWave-band mMIMO uniform linear array (ULA) consisting of N_t transmit antennas and N_r receive antennas. Each vehicle is equipped with only one antenna. Since there are K vehicles that need to be served in the system, at the n-th time slot, the i-th RSU transmits a multi-dimensional multi-beam ISAC signal. Based on the information measured from the echo signal received at the (n-1)-th time slot, the beamforming matrix of the i-th RSU at time slot n can be expressed as

    𝐅_i,n = [𝐟_[i,1],nξ_[i,1],n,  …, 𝐟_[i, K],nξ_[i,K],n] ∈ℂ^N_t× K,

where 𝐟_[i,k],n∈𝐂^N_t× 1 is the beamforming vector for the k-th vehicle, and ξ_[i,k],n∈{0,1} indicates whether vehicle k is connected to RSU i at time slot n. Thus, the transmitted signal at RSU i can be expressed as

    𝐬̃_i,n(t) = 𝐅_i,n𝐬_i,n(t)∈ℂ^N_t × 1,

where 𝐬_i,n(t)=[s_[i,1],n(t), s_[i,2],n(t), …, s_[i,K],n(t)]^T∈ℂ^K × 1 is the ISAC signal.

Since the signal is omnidirectional, the signal received by each RSU contains the echoes reflected by all vehicles within the coverage of the RSU, regardless of whether they are connected to the RSU or not. Thus, the reflected echo signal received at the i-th RSU at time slot n can be expressed as

    𝐫_i,n(t)     = κ∑_k=1^Kβ_[i,k],ne^j2πμ_[i,k],nt𝐛(φ_[i,k],n)𝐚^H(φ_[i,k],n)
       ·s̃_i,n(t-ν_[i,k],n)+𝐳_i,n(t),

where κ=√(N_tN_r) is the antenna gain, μ_[i,k],n and ν_[i,k],n are respectively the Doppler frequency and the time delay with respect to vehicle k and RSU i at time slot n, 𝐚(φ_[i,k],n)∈ℂ^N_t× 1 and 𝐛(φ_[i,k],n)∈ℂ^N_r× 1 are respectively the transmit and receive steering vectors, with φ_[i,k],n=cosθ_[i,k],n, 𝐳_i,n(t)∈ℂ^N_r× 1 denotes the noise vector at RSU i with σ_e^2 being the variance of each element of 𝐳_i,n(t), and β_[i,k],n=ϱ/2d_[i,k],n is the reflection coefficient with d_[i,k],n being the distance between vehicle k and RSU i at time slot n and ϱ represents the complex fading coefficient that depends on the radar cross-section. The transmit and receive steering vectors 𝐚(φ_[i,k],n) and 𝐛(φ_[i,k],n) can be respectively expressed as

    𝐚(φ_[i,k],n) = √(1/N_t)[1, e^-jπφ_[i,k],n, …, e^-jπ (N_t-1)φ_[i,k],n]^T,


    𝐛(φ_[i,k],n) = √(1/N_r)[1, e^-jπφ_[i,k],n, …, e^-jπ (N_r-1)φ_[i,k],n]^T.


The steering vectors corresponding to different vehicles in the network are asymptotically orthogonal. However, we cannot assume that |𝐚^H(φ_[i,k],n)𝐚(φ_[i,k'],n)|=0 for all k≠ k' in a dense-traffic vehicular network, as it is more likely that multiple vehicles are at similar angles relative to the RSU. Since steering vectors are asymptotically orthogonal to each other, only echoes from vehicles at similar angles will interfere with each other. At the i-th RSU, the echoes from the k-th vehicle can be extracted using the following equation:

    𝐫_[i,k],n(t)    =κβ_[i,k],ne^j2πμ_[i,k],nt𝐛(φ_[i,k],n)𝐚^H(φ_[i,k],n)
       ·𝐟_[i,k],nξ_[i,k],ns_[i,k],n(t-ν_[i,k],n)+𝐳_[i,k],n(t),

where 𝐳_[i,k],n(t)∈ℂ^N_r× 1 is the summation of the noise and interference caused by the echoes of other vehicles. With the existence of inter-beam interference, 𝐳_[i,k],n(t) can be written as

    𝐳_[i,k],n(t)    = κ∑_m k^Kβ_[i,m],ne^j2πμ_[i,m],nt𝐛(φ_[i,k],n)𝐚^H(φ_[i,k],n) 
       ·𝐟_[i,m],nξ_[i,k],ns_[i,k],n(t-ν_[i,m],n)+𝐳_i,n(t).






 §.§ Communication Model

Different from previous works <cit.>, in this work, the RSUs transmit control information, DT-related data, and personal data traffic to the vehicles. The vehicles are equipped with a single antenna, thus the communication between the RSU and the vehicles forms a multiple-input and single-output (MISO) system. At time slot n, the signal received by vehicle k from RSU i can be expressed as

    c_[i,k],n(t) =    κ'√(α_[i,k],n)e^j2πμ_[i,k],nt𝐚^H(φ_[i,k],n)𝐬̃_i,n(t)+z_c(t),

where κ'=√(N_t) represents the transmit antenna gain, and z_c(t)∼𝒩(0,σ_c^2) denotes the communication noise. Finally, α_[i,k],n is the path-loss coefficient, which can be computed by

    α_[i,k],n = α̃ d_[i,k],n^-2,

where α̃ is the channel power gain at the reference distance d_0 = 1m. Ideally, the beamforming matrices of different vehicles are asymptotically orthogonal, which yields 𝐚^H(φ_[i,k],n)𝐬̃_i,n(t) ≈ s_[i,k],n(t). However, interference between vehicles is often unavoidable in a dense network, especially for vehicles that are overlapping or at similar angles relative to a RSU. The SINR

at vehicle k can be written as

    SINR_[i,k],n = 
       κ'^2|α_[i,k],n||𝐚^H(φ_[i,k],n)𝐟_[i,k],n|^2ξ_[i,k],n/∑_m≠ k^Kκ'^2|α_[i,k],n||𝐚^H(φ_[i,k],n)𝐟_[i,m],n|^2ξ_[i,m],n+σ_c^2.




§ DT-BASED STATE TRANSITION AND PREDICTION

Next, we introduce the construction of the DT with the sensed information.

Sensing information of the physical network is forwarded to the DT layer for data processing. After matched filtering, the state of the vehicle at the next time slot can be predicted. Additionally, optimal vehicle assignment and beamforming can be designed based on the predicted states of the vehicles. For vehicle k with respect to RSU i at time slot n, we define 𝐱_[i,k],n=[φ_[i,k],n, d_[i,k],n, v̇_[i,k],n]^T ∈ℂ^3 × 1 as its state, and 𝐲_[i,k],n=[𝐫̃_[i,k],n, ν̃_[i,k],n, μ̃_[i,k],n]^T ∈ℂ^(N_t+2) × 1 as the measured parameters. Here, v̇_[i,k],n represents the radial velocity of the k-th vehicle to the i-th RSU at time slot n. ν̃_[i,k],n and μ̃_[i,k],n are the measurements of ν_[i,k],n and μ_[i,k],n, respectively. 𝐫̃_[i,k],n is the matched filtering output, which will be discussed in the next subsection. The vector 𝐲_[i,k],n can be obtained through matched filtering and further used to correct the prediction of 𝐱_[i,k],n and predict 𝐱_[i,k],n+1. The block diagram of the DT-based ISAC network is shown in Fig. <ref>. To avoid extra overhead, the matched-filtering and Kalman filter are operated at the RSUs and only the states of the vehicle are needed to be transmitted to the processor for decision-making. The above processes can be operated either on a local DT or through edge computing, depending on the scale of the applications. In this work, we assume that the vehicle assignment and beamforming design are performed locally because there are stringent delay limits.






 §.§ Matched Filtering



The sensing information of the vehicles can be extracted using matched filtering. For the rest of this subsection, we only consider the connected links between the vehicles and the RSUs. Using matched filtering method, ν_[i,k],n and μ_[i,k],n can be estimated by the following equation:

    {ν̃_[i,k],n, μ̃_[i,k],n} = 
       max_ν,μ|∫_0^T_s𝐫_[i,k],n(t) s^*_[i,k],n(t-ν_[i,k],n)e^-j2πμ_[i,k],n tdt|^2,

where T_s is the duration of the ISAC signal, which should be lower than the slot duration T. The matched filter output of 𝐫_[i,k],n(t) can be written as

    𝐫̃_[i,k],n=   ∫_0^T_s𝐫_[i,k],n(t) s^*_[i,k],n(t-ν̃_[i,k],n)e^-j2πμ̃_[i,k],n tdt
    
        =   κβ_[i,k],n𝐛(φ_[i,k],n) 𝐚^H(φ_[i,k],n)𝐟_[i,k],n
       ·∫_0^T_ss_[i,k],n(t-ν_[i,k],n)s^*_k,n(t-ν̃_[i,k],n)
       · e^-j2π(μ̃_[i,k],n-μ_[i,k],n)tdt
       +∫_0^T_s𝐳_[i,k],n(t)s^*_[i,k],n(t-ν̃_[i,k],n)e^-j2πμ̃_[i,k],ntdt
    
        =   κβ_[i,k],nG 𝐛(φ_[i,k],n)𝐚^H(φ_[i,k],n)𝐟_[i,k],n+𝐳̃_[i,k],n,

where 𝐳̃_[i,k],n∼𝒞𝒩 (0,σ_𝐫,[i,k],n^2𝐈_N_R) denotes the measurement noise after matched filtering with 𝐈_N_R representing a N_R× N_R identity matrix, and G=∫_0^T_ss_k,n(t-ν_[i,k],n)s^*_k,n(t-ν̃_[i,k],n)e^-j2π(μ̃_[i,k],n-μ_[i,k],n)tdt represents the matched filtering gain.







Note that d_[i,k],n and v̇_[i,k],n can be reflected by ν̃_[i,k],n and μ̃_[i,k],n, which yields:

    ν̃_[i,k],n = 2d_[i,k],n/c+ε_[i,k],n,


    μ̃_[i,k],n = 2v̇_[i,k],nf_c/c+ϱ_[i,k],n,

where f_c is the carrier frequency, c is the speed of light, ε_[i,k],n∼𝒩(0,σ_ν,[i,k],n^2) and ϱ_[i,k],n∼𝒩(0,σ_μ,[i,k],n^2) are the estimation errors of ν̃_[i,k],n and μ̃_[i,k],n, with noise variance being σ_ν,[i,k],n^2 and σ_μ,[i,k],n^2, respectively.

After matched filtering, the desired signal is amplified and the interference is filtered. The measurement noise can be seen as inversely proportional to the signal-to-noise ratio (SNR) at the receive antenna <cit.>, which yields:

    σ_𝐫,[i,k],n^2 = ρ_𝐫^2σ_e^2/G,


    σ_ν,[i,k],n^2 = ρ_ν^2σ_e^2/Gκ^2 |β_[i,k],n|^2|η_[i,k],n|^2,


    σ_μ,[i,k],n^2 = ρ_μ^2σ_e^2/Gκ^2 |β_[i,k],n|^2|η_[i,k],n|^2,

where η_[i,k],n=𝐚^H(φ_[i,k],n)𝐟_[i,k],n represent the beamforming gain factor, ρ_𝐫, ρ_ν and ρ_μ are constant indexes determined by the system configuration. 






 §.§ State Evolution









At each time slot, the state estimation is based on the observation of radar echoes. The kinematic model of a moving vehicle in the network relative to the RSU is shown in Fig. <ref>.
The prediction of 𝐱_[i,k],n, denoted by 𝐱̂_[i,k],n, is obtained based on the measurement at time slot n-1, i.e., 𝐱̃_[i,k],n-1. 𝐱_[i,k],n is a function of 𝐱_[i,k],n-1, which yields:

    𝐱_[i,k],n = 𝐠(𝐱_[i,k],n-1)+ϵ_x,

where function 𝐠(·) can be written as (<ref>), as shown at the top of the next page, and ϵ_x=[ϵ_φ,ϵ_d,ϵ_v̇]^T is the prediction noise, with ϵ_φ∼𝒩(0,σ_φ^2), ϵ_d∼𝒩(0,σ_d^2), ϵ_v̇∼𝒩(0,σ_v̇^2).

Since the vehicles are driven on a highway and the duration of each time slot is relatively short, it is reasonable to assume that the direction of the movement is along the highway and the velocities of the same vehicle at adjacent time slots are equal, i.e., v̇_[i,k],n-1/φ_[i,k],n-1 = v̇_[i,k],n/φ_[i,k],n.

mytempeqncnt




 §.§ Extended Kalman Filter


As the movement of the vehicles is continuous, the RSUs need to continuously track the vehicles for a certain period. Without sufficient information, sensing performance is limited. However, accuracy is expected to improve over time based on previous measurements. Therefore, this work applies the Kalman filter to track the vehicles. Since the observation function is non-linear, the standard Kalman filter is not applicable in this work. Instead, an EKF is utilized to estimate sensing errors. 𝐆_[i,k],n is defined as the Jacobian matrix of 𝐠(𝐱̃_[i,k],n-1), which can be expressed as

    𝐆_[i,k],n = 
    [   ∂φ_[i,k],n/∂φ̃_[i,k],n-1   ∂φ_[i,k],n/∂d̃_[i,k],n-1  ∂φ_[i,k],n/∂ṽ̇̃_[i,k],n-1;  ∂ d_[i,k],n/∂φ̃_[i,k],n-1  ∂ d_[i,k],n/∂d̃_[i,k],n-1 ∂ d_[i,k],n/∂ṽ̇̃_[i,k],n-1;  ∂v̇_[i,k],n/∂φ̃_[i,k],n-1  ∂v̇_[i,k],n/∂d̃_[i,k],n-1 ∂v̇_[i,k],n/∂ṽ̇̃_[i,k],n-1 ].


Since 𝐲_[i,k],n is a function of 𝐱_[i,k],n-1, we have:

    𝐲_[i,k],n = 𝐡(𝐱_[i,k],n)+χ_[i,k],n,

where χ_[i,k],n∈ℂ^(N_t+2)× 1 is the measurement noise. The Jacobian matrix of 𝐡(𝐱_[i,k],n), i.e., 𝐇_[i,k],n, can be expressed as

    𝐇_[i,k],n   =∂𝐡(𝐱_[i,k],n)/∂𝐱_[i,k],n
       =
    [ ∂𝐫̃_[i,k],n/∂φ_[i,k],n                      0                      0;                      0                    2/c                      0;                      0                      0                 2f_c/c ].




By taking the first-order Taylor expansion, the state transition function and the observation function can be approximately linearized by

    𝐱_[i,k],n≈    𝐆_[i,k],n𝐱_[i,k],n-1 + 𝐠(𝐱̃_[i,k],n-1)
       -𝐆_[i,k],n-1𝐱̃_[i,k],n-1+ϵ_x,


    𝐲_[i,k],n≈    𝐇_[i,k],n𝐱_[i,k],n + 𝐡(𝐱̂_[i,k],n)
       - 𝐇_[i,k],n𝐱̂_[i,k],n+ χ_[i,k],n.


The state prediction MSE matrix 𝐌̂_[i,k],n can be expressed as

    𝐌̂_[i,k],n  =   𝔼[(𝐱_[i,k],n-𝐱̂_[i,k],n)(𝐱_[i,k],n-𝐱̂_[i,k],n)^H]
    
        =   𝐆_[i,k],n𝐌̃_[i,k],n-1𝐆_[i,k],n^H + 𝐄_[i,k],n,

where 𝐄_[i,k],n=diag(σ_φ^2,σ_d^2,σ_v̇^2) is the covariance matrix of the state prediction noise, and 𝐌̃_[i,k],n-1 is the state measurement MSE matrix at the (n-1)-th time slot.


The Kalman gain 𝐊_[i,k],n can be expressed as

    𝐊_[i,k],n =    𝐌̂_[i,k],n𝐇_[i,k],n^H 
       (𝐐_[i,k],n+𝐇_[i,k],n𝐌̂_[i,k],n𝐇_[i,k],n^H)^-1.


We define 𝐞_[i,k],n = 𝐲_[i,k],n-𝐡(𝐱̂_[i,k],n) as the difference between the actual received echo signal and the anticipated echo signal. To enhance the accuracy of the vehicle tracking, 𝐞_[i,k],n is measured after each time slot and subsequently updated in the Kalman gain. Through analyzing the error of the state prediction, we can obtain the following lemma.



The error of the state prediction, i.e., 𝐱_[i,k],n - 𝐱̂_[i,k],n equals to 𝐇_L,[i,k],n^-1(𝐞_[i,k],n-χ_[i,k],n), where 𝐇_L,[i,k],n^-1=(𝐇_[i,k],n^H𝐇_[i,k],n)^-1𝐇_[i,k],n^H is the left inverse of 𝐇_[i,k],n.


𝐞_[i,k],n denotes the difference between 𝐲_[i,k],n and 𝐡(𝐱̂_[i,k],n) and can be expressed as

    𝐞_[i,k],n =    𝐇_[i,k],n𝐱_[i,k],n - 𝐇_[i,k],n𝐱̂_[i,k],n  + χ_[i,k],n
    
        =   𝐇_[i,k],n (𝐱_[i,k],n-𝐱̂_[i,k],n) + χ_[i,k],n.

Then, we have:

    𝐱_[i,k],n-𝐱̂_[i,k],n = 𝐇_L,[i,k],n^-1(𝐞_[i,k],n-χ_[i,k],n).


Hence, Lemma 1 is proved.


The updated error matrix can be expressed as


    𝔼[(𝐱_[i,k],n-𝐱̂_[i,k],n)(𝐱_[i,k],n-𝐱̂_[i,k],n)^H]
        = 𝔼[𝐇_L,[i,k],n^-1(𝐞_[i,k],n-χ_[i,k],n) 
             (𝐞_[i,k],n-χ_[i,k],n)^H(𝐇_L,[i,k],n^-1)^H]
        = 𝐇_L,[i,k],n^-1(𝐞_[i,k],n𝐞_[i,k],n^H - 𝔼[𝐞_[i,k],nχ_[i,k],n^H]
             - 𝔼[χ_[i,k],n𝐞_[i,k],n^H]+ 𝔼[χ_[i,k],nχ_[i,k],n^H] ) (𝐇_L,[i,k],n^-1)^H
        = 𝐇_L,[i,k],n^-1(𝐞_[i,k],n𝐞_[i,k],n^H-𝐐_[i,k],n)(𝐇_L,[i,k],n^-1)^H,

where 𝐐_[i,k],n=diag(σ_𝐫,[i,k],n^21_N_r^T,σ_ν,[i,k],n^2,σ_μ,[i,k],n^2) with 1_N_r representing a size-N_r all one column vector.




Substituting (<ref>) into (<ref>), the updated Kalman gain can be written as







    𝐊_[i,k],n = 𝐇_L,[i,k],n^-1(𝐞_[i,k],n𝐞_[i,k],n^H-𝐐_[i,k],n)
       (𝐇_[i,k],n𝐇_L,[i,k],n^-1)^H(𝐐_[i,k],n+𝐇_[i,k],n𝐇_L,[i,k],n^-1
       (𝐞_[i,k],n𝐞_[i,k],n^H-𝐐_[i,k],n)(𝐇_[i,k],n𝐇_L,[i,k],n^-1)^H )^-1.


This updated Kalman gain can be used for fix some of the prediction errors at the last time slot, which yields:

    𝐱̃_[i,k],n = 𝐱̂_[i,k],n + 𝐊_[i,k],n𝐞_[i,k],n.


Finally, the MSE matrix of the state measurement can be computed by

    𝐌̃_[i,k],n =   𝔼 [(𝐱_[i,k],n - 𝐱̃_[i,k],n)(𝐱_[i,k],n - 𝐱̃_[i,k],n)^H]
    
        =   𝔼 [(𝐱_[i,k],n -𝐱̂_[i,k],n - 𝐊_[i,k],n𝐞_[i,k],n)
       × (𝐱_[i,k],n - 𝐱̂_[i,k],n - 𝐊_[i,k],n𝐞_[i,k],n)^H]
    
        =   𝔼[(𝐱_[i,k],n-𝐱̂_[i,k],n)(𝐱_[i,k],n-𝐱̂_[i,k],n)^H] 
       - 𝔼[𝐊_[i,k],n𝐞_[i,k],n  (𝐱_[i,k],n -𝐱̂_[i,k],n)^H]
    
        =   𝐊_[i,k],n𝐐_[i,k],n(𝐇_L,[i,k],n^-1)^H.




The DT then uses the predicted states and the MSE matrix to design the predictive beamforming and assign vehicles to different RSUs. The overall algorithm for state prediction with EKF is summarized in Algorithm <ref>.



§ OPTIMIZATION PROBLEM FOR BEAMFORMING DESIGN AND VEHICLE ASSIGNMENT


Our goal is to optimize the overall transmission rate within the vehicular network by strategically assigning vehicles to different RSUs and designing predictive beamforming for both sensing and communication purposes. Once the DT predicts the location of vehicles at the next time slot, we start the beamforming design and vehicle allocation process. Ideally, we would aim to find the optimal 𝐅_i,n and ξ_n={ξ_[i,k],n}, ∀ i, ∀ k that maximize both the communication rate and the sensing accuracy. However, there exists a clear trade-off between these objectives. Greedily designing the predictive beamforming that is highly suitable for communication results in a maximum communication rate in the next time slot. However, this design could lead to a decline in the CRB of the next time slot which will unavoidably affect the accuracy of the DT for the subsequent time slot, subsequently affecting the potential communication rate in later time slots. The challenge of establishing a direct relationship between sensing accuracy and communication rate makes the problem more complex to model. 





 §.§ Posterior Cramér-Rao Lower Bound


CRB is a fundamental concept in estimation theory and statistics. It establishes the minimum variance level for unbiased estimators of a deterministic parameter that is fixed but unknown. Specifically, the variance of any such estimator cannot be lower than the reciprocal of the Fisher information. However, in this work, the estimation errors of vehicle states depend on not only the measured parameters but also the errors inherited from the previous time slots. Therefore, the PCRB is used for finding the MSE lower bound of the vehicle state.

The conditional probability density function (PDF) of 𝐲_[i,k],n and 𝐱_[i,k],n given 𝐱̂_[i,k],n can be expressed as

    p(𝐱_[i,k],n,𝐲_[i,k],n   |𝐱̂_[i,k],n) = 
       p(𝐲_[i,k],n|𝐱_[i,k],n,𝐱̂_[i,k],n)p(𝐱_[i,k],n|𝐱̂_[i,k],n),

where p(𝐲_[i,k],n|𝐱_[i,k],n,𝐱̂_[i,k],n) is the conditional PDF of 𝐲_[i,k],n given 𝐱_[i,k],n and 𝐱̂_[i,k],n, and p(𝐱_[i,k],n|𝐱̂_[i,k],n) is the conditional PDF of 𝐱_[i,k],n given 𝐱̂_[i,k],n. According to the measurement model, p(𝐲_[i,k],n|𝐱_[i,k],n,𝐱̂_[i,k],n) can be computed by

    p(𝐲_[i,k],n|𝐱_[i,k],n   ,𝐱̂_[i,k],n) = 1/π ^N_r+2 (𝐐_[i,k],n)
       exp((𝐲_[i,k],n - 𝐡(𝐱_[i,k],n))^H𝐐_[i,k],n^-1
       (𝐲_[i,k],n - 𝐡(𝐱_[i,k],n))).


The posterior Fisher information matrix 𝐅_[i,k],n can be computed by

    𝐅_[i,k],n
       =-𝔼[∂^2ln p(𝐲_[i,k],n|𝐱_[i,k],n,𝐱̂_[i,k],n)p(𝐱_[i,k],n|𝐱̂_[i,k],n)/∂𝐱_[i,k],n^2]
       = -𝔼[∂^2ln p(𝐲_[i,k],n|𝐱_[i,k],n,𝐱̂_[i,k],n)/∂𝐱̂_[i,k],n^2]
              -𝔼[∂^2ln p(𝐱_[i,k],n|𝐱̂_[i,k],n)/∂𝐱_[i,k],n^2]
       = 𝐇_[i,k],n𝐐_[i,k],n^-1𝐇_[i,k],n^H+ 𝐌̂_[i,k],n^-1.


The PCRB matrix is equivalent to the state measurement MSE matrix, hence we have:

    𝐌̃_[i,k],n = 
       (𝐇_[i,k],n𝐐_[i,k],n^-1𝐇_[i,k],n^H+ (𝐆_[i,k],n𝐌̃_[i,k],n-1𝐆_[i,k],n^H)^-1)^-1,

which shows the direct relationship between 𝐌̃_[i,k],n and 𝐌̃_[i,k],n-1.



 §.§ Problem Formulation


While our primary focus is on communication, ensuring a high level of sensing accuracy is crucial for constructing the DT, and, consequently, leads to reliable communication. In this work, although our goal is to maximize the overall throughput at the current time slot, we also impose a constraint on the PCRB to ensure that the sensing error does not increase over time, thereby enabling the construction of an accurate DT. Therefore, we set m̃_[i,k],n^(11)≤m̃_[i,k],n-1^(11), where m_[i,k],n^(ij) denotes the (i,j)-th entry of 𝐌̃_[i,k],n.

𝐌̃_[i,k],n is a 3× 3 matrix, hence we can easily find its inverse matrix. Since we assume that the current sensing performance is good, η_[i,k],n≈ 1. With all the other entries being constant, we have:

    (Π_[i,k],n𝐟_[i,k],n)^H (Π_[i,k],n𝐟_[i,k],n) ≥Λ_[i,k],n,

where Λ_[i,k],n is a constant value achieved by using (<ref>) to find the inverse of the 3×3 matrix, and Π_[i,k],n = ∂𝐛(φ_[i,k],n)𝐚^H(φ_[i,k],n)/∂φ_[i,k],n. The calculation of Λ_[i,k],n is detailed in the Appendix. Then, we can formulate the optimization problem as (<ref>), shown at the top of the next page. 






§ ALGORITHM DESIGN


Problem (<ref>) is difficult to solve because it is non-convex. To solve Problem (<ref>), we use an auxiliary variable γ_[i,k],n to replace SINR_[i,k],n. This allows us to formulate a new optimization problem as Problem (<ref>), shown at the top of the next page below Problem (<ref>).



Problem (<ref>) can be divided into an inner optimization problem over γ_n, and an outer optimization problem over ξ_n and 𝐅_n. The inner optimization problem is convex over γ_n, hence, strong duality holds, and we can formulate the Lagrangian function as (<ref>), shown at the top of this page.




Problem (<ref>) is equivalent to the following dual problem:

    min_λ_n≥0max_γ_n L(γ_n,λ_n).



The saddle point (γ_n^*,λ_n^*) can be found by setting ∂ L(γ_n,λ_n)/∂γ_[i,k],n=0, which yields:


    λ_[i,k],n^*    = 1/1+γ_[i,k],n
    
        =   ∑_m≠ k^Kκ'^2|α_[i,k],n||𝐚^H(φ_[i,k],n)𝐟_[i,m],n|^2ξ_[i,m],n+σ_c^2/∑_m=1^Kκ'^2|α_[i,k],n||𝐚^H(φ_[i,k],n)𝐟_[i,m],n|^2ξ_[i,m],n+σ_c^2.


Since (<ref>) is convex with respect to γ_n, while the other parameters are fixed, the optimal γ_n, i.e., γ_n^* can be calculated by


    γ_[i,k],n^*    =
       κ'^2|α_[i,k],n||𝐚^H(φ_[i,k],n)𝐟_[i,k],n|^2ξ_[i,k],n/∑_m≠1^K κ'^2|α_[i,k],n||𝐚^H(φ_[i,k],n)𝐟_[i,m],n|^2ξ_[i,k],n+σ_c^2.



Once γ_n is fixed, we use fractional programming <cit.> to eliminate the fractional term in the objective function. We introduce a set of auxiliary variables y_[i,k],n, which enables us to express the transformed objective function as a new function denoted by f_q(ξ_n,𝐅_n,γ_n,𝐘_n), as shown by (<ref>) at the top of the next page, where 𝐘_n is the set of {y_[i,k],n}. Problem (<ref>) is equivalent to maximizing f_q(ξ_n,𝐅_n,γ_n^*,𝐘_n), hence we can formulate an optimization problem on f_q(ξ_n,𝐅_n,γ_n^*,𝐘_n), as shown by (<ref>) at the top of the next page below (<ref>).








To solve Problem (<ref>), we use a heuristic algorithm that iteratively fixes the other parameters and optimizes one parameter at a time. As f_q(ξ_n,𝐅_n,γ_n^*,𝐘_n) is a convex function of y_[i,k],n, the optimal y_[i,k],n, i.e., y_[i,k],n^* can be achieved by setting ∂ f_q(ξ_n,𝐅_n,γ_n,𝐘_n)/∂ y_[i,k],n=0. Then we have:

    y_[i,k],n^*    =
       κ|β_[i,k],n||𝐚^H(φ_[i,k],n)𝐟_[i,k],n|ξ_[i,k],n/∑_m≠ k^Kκ^2|β_[i,m],n|^2|𝐚^H(φ_[i,k],n)𝐟_[i,m],n|^2ξ_[i,m],n+σ_z^2.






With y_[i,k],n being fixed too, the objective function and all the constraints of Problem (<ref>) are convex functions with respect to 𝐅_n. Hence, we can use the projected gradient descent (PGD) algorithm to find the solution.



The optimization of vehicle assignments is a binary assignment problem. We present two methods to optimize the matching between RSUs and vehicles. Firstly, a greedy algorithm is proposed that sequentially allocates vehicles for optimal performance. Secondly, a heuristic algorithm is proposed for assigning the vehicles to the RSUs. We define an index ζ_[i,k],n as (<ref>), as shown in the next page below (<ref>), representing the comprehensive utility and penalty arising from the allocation of vehicle k to RSU i during time slot n.









  §.§.§ Greedy Algorithm



In this algorithm, vehicles are allocated to RSUs one by one. The algorithm starts with the vehicle that exhibits the largest difference in distances between itself and both RSUs, considering this vehicle suffers from the greatest impact of path loss. Let 𝒱 represent the set of vehicles in the system. The greedy algorithm is shown in Algorithm <ref>. Through this approach, a reasonable assignment strategy can be established by assigning the vehicles one by one.




  §.§.§ Heuristic Algorithm



We formulate the following optimization problem to optimize ξ_n: 


    max_ξ_n   ∑_i=1^I∑_k=1^K ξ_[i,k],nζ_[i,k],n
    subject to     ξ_[i,k],n∈{0,1},∀ k,∀ i
       ∑_i=1^Iξ_[i,k],n=1,∀ k.






This optimization problem is challenging to solve as every single change in ξ_n results in a series of changes in ζ_[i,k],n. Here, we employ a heuristic algorithm to find the optimal matching between the vehicles and the RSUs, as shown by Algorithm <ref>. In each iteration, we select the vehicle that is most suitable for the other RSU and assign it to that RSU. The suitability of a vehicle for a particular RSU can be reflected by ζ_[i,k],n. This algorithm terminates when all vehicles are assigned to the most suitable RSU.


f_q(ξ_n,𝐅_n,γ_n,𝐘_n) converges to a stationary point.


The aim of (<ref>) and (<ref>) is to find the parameters that maximizes f_q(ξ_n,𝐅_n,γ_n,𝐘_n), hence f_q(ξ_n,𝐅_n,γ_n,𝐘_n) is non-decreasing after each iteration.




§ LEARNING-BASED BEAMFORMING DESIGN AND VEHICLE ASSIGNMENT


Solving the optimization problem proves to be time-consuming, because of the heuristic algorithm employed and the high computational complexity associated with each iteration. This work also introduces an RNN-based technique for beamforming design and vehicle assignment.

RNNs are a class of neural networks that are well-suited for modeling sequential data. As vehicle allocations and beamforming matrices are significantly influenced by the adjacent vehicles, they have to be optimized jointly. The impact of neighboring vehicles is managed by the internal states iterated within the RNN layer. Additionally, the number of vehicles to be allocated within the system varies over time, leading to time-varying input dimensions to the learning algorithm. RNNs can leverage their internal states to handle variable-length sequences of inputs, making them ideal for our requirements.




 §.§ LSTM Network



LSTM networks, introduced by Hochreiter and Schmidhuber <cit.>, extend RNNs by providing short-term memory mechanisms to preserve internal states. A standard LSTM cell includes one input gate, one output gate, and one forget gate. The output of the forget gate can be calculated by

    f_t = σ (W_f· [h_t-1,x_t]+b_f),

where f_t lies in the range (0,1), W_f is the weight of the forget gate, b_f is the bias of the forget gate, x_t is the current input, and h_t-1 is the previous output value.

Similarly, the output of the input gate and the output gate can be respectively written as

    i_t = σ (W_i · [h_t-1,x_t]+b_i),


    o_t = σ (W_o  · [h_t-1,x_t]+b_o),

where W_i and W_o are the weights of the input gate and output gate respectively, b_i and b_o are the bias of the input gate and output gate respectively. While LSTM networks are commonly employed in time series contexts, we have found them effective in our model due to the robust correlation among adjacent users after sorting the vehicles based on their locations. The key advantage of LSTMs over traditional RNNs lies in their ability to learn and remember long-term dependencies in sequential data, due to their gated architecture and memory cells. This makes them well-suited for our work where the beamforming matrices of multiple vehicles need to be jointly considered.




 §.§ Bi-LSTM-based Vehicle Assignment and Beamforming Design


Since a vehicle is affected by the vehicles on both sides, we employ a bi-directional LSTM framework, as depicted in Fig. <ref>, which encompasses K sets of input features. Organizing information about the vehicles based on their positions before inputting into the learning algorithm is crucial. Considering the real and imaginary parts individually, the input feature of the neural network includes Re{𝐫_[i,k],n}, Im{𝐫_[i,k],n}, Re{𝐟_[i,k],n-1}, Im{𝐟_[i,k],n-1}, ν̃_[i,k],n, and μ̃_[i,k],n. Given the multi-dimensional nature of the input, a flatten layer follows each input layer to generate a one-dimensional tensor. 




The LSTM output is then forwarded to the adjacent LSTM to leverage the correlation between the beamforming of adjacent vehicles. Subsequently, fully connected dense layers, with Rectified Linear Unit (ReLU) serving as the activation function, follow the LSTM layer to generate the output. The output includes both the real and imaginary parts of the beamforming, i.e., [Re{𝐟_[i,k],n}, Im{𝐟_[i,k],n}] ∈ℂ^2N_t× 1, used to formulate the predictive beamforming. MSE is employed as the loss function for evaluating the predicted beamforming. Notably, the overfitting of this network is negligible, hence drop-out layers are not needed.





 §.§ Complexity Analysis


The overall complexity of the proposed algorithm is the summation of the complexity of two separate learning algorithms, both are bi-directional LSTM networks. LSTM is local in both space and time, which means that the storage requirements of the network are irrelevant to the actual input size. For each time slot, the time complexity of each weight of the LSTM network equals 𝒪(1).

The weight of the LSTM network can be computed by W=4[h(h+e)+h] with h and e being the number of hidden units and the embedding dimension of input. Since bi-directional LSTM is applied in both algorithms, the total time complexity of the proposed LSTM framework can be written as

    C_1 = 𝒪(8[h_1(h_1+e_1)+h_1])= 𝒪(8[h_1(h_1+e_1)]),

where h_l and e_l are the number of hidden units and the embedding dimension of input. 

The complexity of each iteration in the greedy algorithm C_g and the heuristic algorithm C_h can be expressed by

    C_g = C_h = 𝒪(K^2 N_t + IK^2N_t^2/ϵ)=𝒪( IK^2N_t^2/ϵ),

where ϵ denotes the accuracy. Normally, the heuristic algorithm is faster when K is low and the greedy algorithm is faster when K is high. The complexity of the LSTM network has lower degrees than the heuristic algorithm, hence the LSTM is generally less complex especially when the number of vehicles is high.





§ NUMERICAL RESULTS






We conducted various simulations in this section to evaluate the performance of our proposed methods. The default settings of the simulation parameters are shown in Table <ref>. We first set N_r=N_t=32 and train the LSTM networks with a training set of 10,000 samples and a validation set of 3,000 samples. The training results in terms of the accuracy rate for the vehicle assignment problem and the MSE for the beamforming optimization problem are shown in Fig. <ref> and Fig. <ref>, respectively. We can achieve an accuracy rate of above 96% after tens of training epochs for vehicle assignment. The LSTM network for beamforming optimization requires more epochs for training. The optimal performance can be achieved after roughly 200 training epochs unless the number of vehicles in the system is too large. Note that as the number of vehicles increases, the accuracy rate decreases, while the MSE of the predicted beamforming also decreases. This reduction in MSE occurs because, with more vehicles in the system, distribution tends to be more even, reducing randomness. Conversely, when fewer vehicles are present, the randomness of the system increases significantly, which complicates beamforming prediction. Furthermore, with only a few vehicles, there typically is one optimal assignment, with other assignments being significantly inferior. However, as the number of vehicles increases, the system accommodates more sub-optimal solutions, which helps maintain relatively good performance despite the decrease in accuracy rate.










With the number of antennas being constant, we change the number of vehicles in the system. We compare the performance of the optimal matching and beamforming obtained from the heuristic algorithm, the LSTM-based method, the Greedy method, and the conventional distance-based vehicle-RSU matching. Fig. <ref> presents the relationship between the number of vehicles in the system and the average throughput of each vehicle. Generally, the average throughput of all four schemes decreases sharply as the number of vehicles increases. This decrease occurs because more vehicles introduce more interference and complicate the assignment. Our proposed DT-based vehicle assignment and beamforming design significantly outperforms the traditional distance-based method, especially when more vehicles are served simultaneously. However, the greedy algorithm performs even worse than the distance-based method when the number of vehicles is low but approaches the performance of Algorithm <ref> when the number of vehicles is high. This behavior is due to the tendency of the greedy algorithm to produce sub-optimal matchings. When there are only a few vehicles in the system, the optimal matching between vehicles and RSUs is unique, and sub-optimal matching is far worse than the optimal one. However, as the number of vehicles increases, this performance gap becomes smaller, and there may even be multiple optimal matching schemes. Furthermore, we can see that with the LSTM network, vehicles can achieve throughput only slightly lower than it obtained from Algorithm <ref>, while significantly reducing processing time.




Apart from the number of vehicles, the performance of the system is also influenced by the number of antennas at the RSUs. We keep the number of vehicles in the system constant and vary the number of antennas, as shown in Fig. <ref>. It is assumed that the number of transmit antennas and the number of receive antennas are equal at the RSUs and there are K=50 vehicles. Clearly, the average throughput increases when there are more antennas. The results in the figure demonstrate that Algorithm <ref> achieves the best performance, with the LSTM slightly behind, followed by the greedy algorithm and the distance-based method.

In all the previous simulations, we assume a constant number of vehicles. However, in reality, vehicles continuously enter and exit the system, leading to fluctuations in the number of vehicles. To see how the communication performance and sensing performance varies over time in a dynamic environment for a continuous period, we compare these schemes while assuming that vehicle arrivals follow a Poisson process. Given that the vehicles are on a highway, we assume that their instantaneous speeds follow a Gaussian distribution with a mean of 30 meters per second, and we ignore any congestion on the highway. To estimate the average number of vehicles when the system is in a stationary state, we apply Little's law. Since the arrival of vehicles is random, the results of each simulation vary. Therefore, we simulate the system for lots of times and used one of the most common cases as an example.




Initially, we set the average arrival rates in both directions to 5 vehicles per second, resulting in an average of 20 vehicles in the system. Starting from an initial state where the system is vacant, we continuously monitor performance for 40 seconds and present the simulation results in Fig. <ref>. The upper half of Fig. <ref> displays the total throughput comparison of all four schemes. It shows that the total throughput increases initially and then fluctuates around a stationary level. It is challenging to point out the superior scheme on the upper half of Fig. <ref>. To provide more clarity, we present the throughput gain of the other three schemes relative to the distance-based matching on the lower half of Fig. <ref>, which corresponds to the upper half exactly. In the first three seconds, the throughput gain of the greedy algorithm remains negative, which also demonstrates the results in Fig. <ref>. As the number of vehicles increases, the greedy algorithm generally outperforms the distance-based matching, but it exhibits occasional negative throughput gains, indicating its high instability. The matching based on Algorithm <ref> consistently demonstrates superiority, except for scenarios with very few vehicles, where it achieves zero gain. The LSTM-based method, while slightly inferior to Algorithm <ref>, still outperforms the other schemes in most cases.



Fig. <ref> displays the results when we increase the arrival rate to 12.5 vehicles per second, resulting in an average of 50 vehicles when the system is stationary. Similar to the previous scenario, the total throughput initially increases and then fluctuates around a stationary value, but the initial increase is sharper. As the number of vehicles rapidly rises to a relatively high level, the throughput gain of the greedy algorithm is negative for only one to two seconds and remains positive for the remainder of the time. The overall performance ranking remains consistent: Algorithm <ref> is the best, followed by the LSTM-based method and the greedy algorithm, all of which outperform the distance-based matching. Additionally, we observe that the stability of the greedy algorithm significantly improves as the average number of vehicles in the system increases. Furthermore, in both Fig. <ref> and Fig. <ref>, the total throughput increases initially and remains high as the system stabilizes, indicating that the sensing performance also remains high. This consistent level of throughput suggests that the sensing accuracy of the system is effectively supporting the communication processes, corroborating our approach that links high communication rates with robust sensing performance.



In Fig. <ref>, we compare the root of CRB (RCRB) for scenarios with 16, 32, and 64 transmit antennas. Initially setting RCRB to 0.1 meters, our results demonstrate a sharp decrease in RCRB at the beginning due to the design of beamforming techniques that enhance both communication and sensing performances. Subsequently, RCRB reaches a steady state where it converges to a constant value suitable for high-quality communication purposes. Notably, as the number of antennas increases, the RCRB bound decreases.


According to the simulation results, our proposed vehicle assignment and beamforming optimization algorithm outperforms all the other schemes no matter whether there are more or fewer vehicles in the system. The LSTM-based algorithm brings us slightly lower throughput but significantly reduces the processing time. The greedy algorithm performs the worst when the number of vehicles in the system is low, but approaches the optimal method when the number of vehicles increases.   



§ CONCLUSION



In this work, we introduce a design for DT-based ISAC in vehicular networks. The physical system in the real world comprises two RSUs and multiple vehicles. The DT is used to create the motion model of the vehicles to track them and perform intricate computations aimed at optimizing the overall communication rate. The received sensing information from the vehicles is then forwarded to the DT for beamforming design, vehicle assignments, and potentially other applications. While the optimization problem can be solved iteratively, we also leverage bi-directional LSTM networks to expedite decision-making. Our results demonstrate a substantial improvement in overall throughput. Furthermore, this work has the potential for extension to scenarios involving multiple RSUs and applications in facilitating handovers.



























§ FIND R


First, we define Ω_[i,k],n as

    Ω_[i,k],n =    𝐇_[i,k],n𝐐_[i,k],n^-1𝐇_[i,k],n^H
       + (𝐆_[i,k],n𝐌̃_[i,k],n-1𝐆_[i,k],n^H)^-1.

 
Then, m̃_[i,k],n^(11)≤m̃_[i,k],n-1^(11) is equivalent to

    (ω_(2,2)ω_(3,3)-ω_(2,3)ω_(3,2))/(ω_(1,1)(ω_(2,2)ω_(3,3)
       -ω_(2,3)ω_(3,2))-ω_(1,2)(ω_(2,1)ω_(3,3)-ω_(2,3)ω_(3,1))
       +ω_(1,3)(ω_(2,1)ω_(3,2)-ω_(2,2)ω_(3,1)))≤m̃_[i,k],n-1^(11),

where ω^(i,j) is the (i,j)-th entry of Ω_[i,k],n. Because Ω_[i,k],n is positive-definite, we have

    (ω_(1,2)(ω_(2,1)ω_(3,3)-ω_(2,3)ω_(3,1)) + ω_(1,3)(ω_(2,1)ω_(3,2)
       -ω_(2,2)ω_(3,1)) + (ω_(2,2)ω_(3,3)-ω_(2,3)ω_(3,2))/m̃_[i,k],n-1^(11))
       /(ω_(2,2)ω_(3,3)-ω_(2,3)ω_(3,2))≤ω_(1,1).


We define m̂_[i,k],n^(1,1) as the (1,1)-th entry of 𝐌̂_[i,k],n, then ω_(1,1) can be written as

    ω_(1,1) = 
       κ^2 β_[i,k],n^2 G^2/σ_𝐫,[i,k],n^2(Π_[i,k],n𝐟_[i,k],n)^H (Π_[i,k],n𝐟_[i,k],n) + m̂_[i,k],n^(1,1).


Based on (<ref>) and (<ref>), Λ_[i,k],n in (<ref>) can be calculated by

    Λ_[i,k],n   = ((ω_(1,2)(ω_(2,1)ω_(3,3)-ω_(2,3)ω_(3,1)) 
       + ω_(1,3)(ω_(2,1)ω_(3,2)-ω_(2,2)ω_(3,1)) + (ω_(2,2)ω_(3,3)
       -ω_(2,3)ω_(3,2))/m̃_[i,k],n-1^(11))/(ω_(2,2)ω_(3,3)
       -ω_(2,3)ω_(3,2))-m̂_[i,k],n^(1,1))σ_𝐫,[i,k],n^2/κ^2 β_[i,k],n^2 G^2.


IEEEtran



[
    < g r a p h i c s >
]Weihang Ding
received the B.Sc. degree in telecommunication engineering from Beijing Jiaotong University, Beijing, China, in 2018, the M.Sc. degree in telecommunications from King's College London, London, United Kingdom, in 2019. He is currently pursuing the Ph.D. degree in telecommunications at King's College London too. His main research interests include UAV communication, physical layer resource allocation, hybrid automatic repeat requests, channel coding, and integrated sensing and communication.


[
    < g r a p h i c s >
]Zhaohui Yang
(Member, IEEE) is currently a ZJU Young Professor with the Zhejiang Key Laboratory of Information Processing Communication and Networking, College of Information Science and Electronic Engineering, Zhejiang University. He received the Ph.D. degree from Southeast University, Nanjing, China, in 2018. From 2018 to 2020, he was a Post-Doctoral Research Associate at the Center for Telecommunications Research, Department of Informatics, King’s College London, U.K. From 2020 to 2022, he was a Research Fellow at the Department of Electronic and Electrical Engineering, University College London, U.K. His research interests include joint communication, sensing, and computation, federated learning, and semantic communication. He received IEEE Communications Society Leonard G. Abraham Prize Award in 2024, IEEE Marconi Prize Paper Award in 2023, IEEE Katherine Johnson Young Author Paper Award in 2023. He currently serves as an Associate Editor of IEEE TGCN, IEEE CL, IEEE TMLCN. He has served as a Guest Editor for several journals including IEEE Journal on Selected Areas in Communications.


[
    < g r a p h i c s >
]Mingzhe Chen
is currently an Assistant Professor with the Department of Electrical and Computer Engineering and Frost Institute of Data Science and Computing at University of Miami. His research interests include federated learning, reinforcement learning, virtual reality, unmanned aerial vehicles, and Internet of Things. He has received four IEEE Communication Society journal paper awards including the IEEE Marconi Prize Paper Award in Wireless Communications in 2023, the Young Author Best Paper Award in 2021 and 2023, and the Fred W. Ellersick Prize Award in 2022, and four conference best paper awards at ICCCN in 2023, IEEE WCNC in 2021, IEEE ICC in 2020, and IEEE GLOBECOM in 2020. He currently serves as an Associate Editor of IEEE Transactions on Mobile Computing, IEEE Transactions on Communications, IEEE Wireless Communications Letters, IEEE Transactions on Green Communications and Networking, and IEEE Transactions on Machine Learning in Communications and Networking.


[
    < g r a p h i c s >
]Yuchen LiuYuchen Liu is currently an Assistant Professor with the Department of Computer Science at North Carolina State University, USA. He got his Ph.D. degree at the Georgia Institute of Technology, USA. His research interests include wireless networking, digital twins, generative AI, distributed learning, mobile computing, and software simulation. He has received several best paper awards at IEEE and ACM conferences. He currently serves as Associate Editors of IEEE Transactions on Green Communications and Networking, IEEE Transactions on Machine Learning in Communications and Networking, and Elsevier Computer Networks.



[
    < g r a p h i c s >
]Mohammad Shikh-Bahaei
(Senior Member, IEEE) received the B.Sc. degree from the University of Tehran, Tehran, Iran, in 1992, the M.Sc. degree from the Sharif University of Technology, Tehran, in 1994, and the Ph.D. degree from King’s College London, London, U.K., in 2000.,He worked for two start-up companies, and National Semiconductor Corporation, Santa Clara, CA, USA (now part of Texas Instruments Inc.), on the design of third-generation mobile handsets, for which he has been awarded three U.S. patents as the Inventor and the Co-Inventor, respectively. In 2002, he joined King’s College London, as a Lecturer, where he is currently a Full Professor of Telecommunications with the Center for Telecommunications Research, Department of Engineering. He has authored/coauthored numerous journals and conference papers. He has been engaged in research in the area of wireless communications and signal processing for 25 years both in academic and industrial organizations. His research interests include learning-based resource allocation for multimedia applications over heterogeneous communication networks, full-duplex and UAV communications, and secure communication over wireless networks.,Dr. Shikh-Bahaei was a recipient of the Overall King’s College London Excellence in Supervisory Award in 2014. He was the Founder and the Chair of the Wireless Advanced (formerly, SPWC) Annual International Conference from 2003 to 2012.





 Paper Title: RF Power Transmission for Self-sustaining Miniaturized IoT Devices.












Journal of  Class Files, Vol. 14, No. 8, August 2015
Shell et al.: Bare Demo of IEEEtran.cls for IEEE Journals

RF Power Transmission for Self-sustaining Miniaturized IoT Devices
    

Lukas Schulthess1, 
Federico Villani1, 
Philipp Mayer1, 
Michele Magno1
1Dept. of Information Technology and Electrical Engineering, ETH Zürich, Switzerland 


    August 5, 2024
=====================================================================================================================================================================



§ ABSTRACT 
Radio Frequency (RF) wireless power transfer is a promising technology that has the potential to constantly power small Internet of Things (IoT) devices, enabling even battery-less systems and reducing their maintenance requirements. However, to achieve this ambitious goal, carefully designed RF energy harvesting (EH) systems are needed to minimize the conversion losses and the conversion efficiency of the limited power. For intelligent internet of things sensors and devices, which often have non-constant power requirements, an additional power management stage with energy storage is needed to temporarily provide a higher power output than the power being harvested. 
This paper proposes an RF wireless power energy conversion system for miniaturized IoT composed of an impedance matching network, a rectifier, and power management with energy storage. The proposed sub-system has been experimentally validated and achieved an overall power conversion efficiency (PCE) of over 30 for an input power of -10dBm and a peak efficiency of 57 at 3dBm.











Wireless energy transfer, RF energy harvesting, battery-less devices, self-sustaining sensors.




§ INTRODUCTION



Wireless power transfer (WPT) is an emerging technology set to supply wireless devices in the future generation of self-sustaining and battery-free Internet of Things (IoT) and wireless sensor networks (WSN) <cit.>. 
RF transmission is widely used for wireless data transfer and wireless communications. 
Additionally, RF waves can also carry energy together with information. 
Due to the recent improvement in power consumption of electronics, leading to miniaturization and reduction of power consumption of IoT devices, WPT is increasingly emerging as a potential and feasible solution as power supply technology for them <cit.>.

Wireless power transfer has several advantages over environmental energy harvesting for the supply of IoT devices <cit.>. 
The main advantage is the controllable nature of wireless power transfer versus the unpredictable nature of environmental energy harvesting <cit.>. 
In fact, instead of harvesting energy for environmental sources, WPT technology is designed to exploit a stable and controllable wireless power supply by deploying a dedicated power transmitter that periodically or continuously sends energy
<cit.>. 
Moreover, RF WPT allows long-distance energy delivery to a multitude of IoT nodes and enables a small form factor at the receiver with compact receiving antennas, especially when compared to other technologies such as inductive coupling or magnetic resonance.
For these reasons, the interest for RF WPT has increased in research and industry in recent years <cit.>.
The vision for this emerging technology is to permanently power low-power devices, eliminating the need for batteries or increasing their lifespans and reducing maintenance without loss of performance <cit.>. 
This can be especially attractive in situations where a periodical change of batteries is difficult or not desired. For these reasons, powering the sensors through wireless power sources provides an effective alternative to other energy harvesting solutions such as solar, thermal, or vibration <cit.>.


WPT is mainly divided into near- and far-field WPT depending on the spatial range of the transfer. 
Near-field inductive WPT is used to wirelessly power many commercially available near-field communication (NFC) devices such as credit cards and smartphones and is widely adopted <cit.>.
However, its range is limited to few a centimeters, due to the mainly inductive transfer of energy and data in the near-field. 
On  the contrary, far-field WPT is based on sending and receiving electromagnetic (EM) waves using dipoles, which allows for power transmission at longer ranges of over 1 meter <cit.>. 

This makes far-field WPT more flexible and particularly suitable for power multi-casting, allowing the transmitter/receiver to move around, even in Non-Line-of-Sight (NLoS) environments <cit.>. 
Support of longer ranges is crucial to supply energy to IoT devices in a wide range of applications <cit.>. 
As a result, there has been a fair amount of research dedicated to the design of dedicated WPT transmitters for these sensor systems <cit.>. 
Despite this, far-field RF-WPT over longer ranges is still very challenging. One of the major challenges is the efficient transformation of the input EM power levels smaller than -10dBm, to the voltage and current demands of the load <cit.>. This is exacerbated by the need to use a miniaturized antenna and compact designs for small and unobtrusive  IoT devices <cit.>.


RF WPT is also increasing its popularity due to the availability of novel commercial products. For instance, Powercast allows power transfer up to 200 using a single narrow-band sub-GHz channel over several meters. 
Having power available at this range, combined with the possibility to supply several receivers from a single transmitter, makes RF WPT a promising power source for low-power IoT devices <cit.>. 
However, it must be taken into account that the effective power available at the IoT nodes is strongly affected by the distance, antenna size, RF matching, and rectification efficiency, and will therefore be orders of magnitude below the transmitted power of 200. Therefore, to efficiently extract power from EM waves, a carefully designed RF EH system is of utmost importance.




This paper presents the electronic blocks of an energy-efficient RF WPT sub-system for miniaturized IoT and focuses on critical design considerations for each part. The design and evaluation of an optimized RF power rectifier and matching circuit for 915 for input power levels ranging from -12dBm to 9dBm is presented. 
Experimental results demonstrate that the optimized design increases the conversion efficiency by up to 20 compared to off-the-shelf circuitry. Finally, the proposed sub-system, but also other commercially available RF rectifiers and power management systems are exploited to maximize the PCE. 
Experimental results show that is possible to achieve an overall PCE of over 30 for an input power of -10dBm which is above the state of the art for such systems.










































































§ ARCHITECTURE
 


The goal of a WPT sub-system is to extract power from electromagnetic waves received by the antenna as efficiently as possible to achieve power transfer over longer distances. Eventually, the captured electromagnetic energy is converted into a usable direct current (DC) voltage.
To achieve this, each of the three main blocks of the EH system (ref. Fig. <ref> (a)), 
impedance matching network, rectifier, and power management with energy storage requires a carefully considered design.




 §.§ Impedance Matching Network

To achieve maximal power transfer between the source (antenna) and the load (rectifier), a carefully designed impedance matching network (IMN) is essential. By transforming the low impedance of the antenna to the system impedance, power reflections can be mitigated. Choosing discrete components with a high-quality factor minimizes power dissipation and further improves the IMN's conversion efficiency <cit.>. 


 §.§ Rectifier

When designing a high efficient RF-to-DC rectifier, selecting appropriate diodes is one of the most important factors since they are the main source of loss and their characteristic determines the overall performance of the rectifier.
Zero-bias Schottky diodes provide very low threshold voltage and junction capacitance in comparison to common diodes, making them well-suited for RF rectifiers and allowing efficient operation <cit.>.



 §.§ Power Management with Energy Storage

After rectification, the harvested RF power is available as DC power and can be already used to drive a load. However, if the load's power needs cannot be guaranteed at all times, which is especially true for miniaturized IoT devices, power management with energy storage is needed <cit.>. By applying strict power-saving techniques to the IoT devices, the system can still achieve self-sustainability <cit.>.


The proposed WPT sub-system, illustrated in Fig. <ref> (b), is optimized for a single narrow-band RF power transmitter with a center frequency of 915 and is realized using off-the-shelf components. 
High-Q capacitors and inductors of the series GJM1555 and LQW18AN keep the conversion losses at a minimum. 
C1 is a DC block to allow the rectifying circuits' DC bias to settle at mid-rail voltage. 
C2 and L1, together with the diode capacitances, build a single band matching network in π-configuration. The rectifier structure itself is based on a voltage doubler using BAT15-04W zero-bias RF Schottky diodes for efficient RF-to-DC conversion.

High-quality RF matching requires a well-defined printed circuit board (PCB) stackup. Thus, the proposed circuit has been realized on a 2-layer PCB using RO4003c as the core dielectric with a thickness of 0.813. RO4003C provides a low and controlled dielectric constant of 3.38 over a wide frequency range and a dissipation factor of 0.0027 at 10, making it well suited for power-sensitive high-frequency designs.

For power management and energy storage, the e-peas AEM30940 energy management circuit has been selected [ref. Fig.<ref> (a)]. It can start operating with empty storage elements from an input voltage as low as 380 and an input power of 3. Once active, power from input voltages as low as 50 can be converted and temporarily stored in a capacitor before generating a constant output voltage level with the internal buck converter and low drop out (LDO) regulator.











§ EXPERIMENTAL RESULTS


To evaluate the performance of the proposed sub-system it has been tuned for 915 which is a commonly used carrier frequency for WPT <cit.>. The individual functional blocks have been evaluated in terms of functionality and efficiency. Aside from the proposed narrow-band rectification circuit of section <ref>, measurements are conducted on the e-peas AEM30940 evaluation board to facilitate reproducibility.

The evaluation of the matching and rectifier characteristics, including an optimal maximal power point configuration analysis, is given in subsection <ref>. Subsection <ref> evaluates the end-to-end harvesting efficiency of the e-peas harvesting integrated circuits (IC) when combined with different RF frontends. Finally, subsection <ref> analyzes the cold-start sequence of the harvesting IC for low input power.



 §.§ Matching Circuit and Rectifier

Ideally, RF harvesting circuits comprise a broadband antenna and non-frequency selective matching circuits, allowing them to harvest from a wide range of carrier signals and input powers. If the application, the carrier frequency, and the typical input power is known, the RF path can be matched precisely to the input impedance of the rectifying circuit. 

<ref> visualizes the characterization of the matching and rectification circuitry. For the evaluation, the two selectable input frontends on the AEM30940 evaluation board, e-peas HP and e-peas LP, respectively optimized for power levels above -10dBm  and below -10dBm, are compared against the narrow band implantation of section <ref> "custom". The circuit matching is analyzed with an HP7853 network analyzer at a constant source power of 0dBm and a matched resistive load on the rectifier output replacing the subsequent circuitry. <ref> (a) shows the S11 parameter of the three circuits for frequencies of up to 2. The 
high-quality factors of the selected components allow a narrow matching to the 915 target frequency for the custom rectifier. In contrast, the e-peas HP circuit allows additional harvesting from the 868 band.

To precisely analyze the rectification efficiency of the previously shown matching and rectifying circuits, an R&S SML03 signal generator configured to 915MHz is connected to the circuit input. The peak output power and the corresponding maximal power point (MPP) configuration are measured with a source measurement unit (SMU) Keysight B2901A emulating an adaptive load. <ref> (b) shows the resulting RF path efficiency as a function of the input power for optimally matched loads. The results suggest that for input signals higher than -5dBm, the e-peas HP outperforms the e-peas LP configuration. In contrast, the custom implementation shows superior performance for input power levels between -12dBm and 9dBm with up to 20 efficiency gain as compared to the commercial circuit's efficiency. The corresponding MPP configuration is shown in <ref> (c). The e-peas LP configuration shows a linear shift with 0.4/dBm over the analyzed power range. The circuits designed for higher input powers are approximately stable until  4dBm, with a significantly rising  MPP ratio above. The inset visualizes the output power for a varying load at 0dBm.























 §.§ Wireless Power Conversion Efficiency

Correctly controlling the output voltage of the RF path is essential to efficiently supply subsequent circuitry. The circuit behavior, when combined with an epeas AEM30940 harvesting IC, is shown in <ref> (b). For the measurement, the previously tested RF frontends have been  attached to the IC's harvesting input. To allow a measurement of the end-to-end harvesting efficiency, shown in <ref> (a) an SMU configured to 3.5 has been connected in parallel to an intermittent energy storage capacitor C_boost. This allows monitoring of the harvested current at a stable and repeatable storage configuration. The embedded maximal power point tracking (MPPT) has been configured to 50 of the open circuit voltage. 
The e-peas LP frontend allows energy-positive operation of the harvesting circuitry down to -15dBm. Combined with the high power optimized frontends, the circuit reaches -16dBm and -15dBm for custom and e-peas HP, respectively. The custom circuit combined with the AEM30940 IC shows increased harvesting efficiency for input power levels between -11dBm and 5dBm with a peak efficiency of 57 at 3dBm. 




 §.§ Cold Start

A critical feature of intermittent operation devices is the capability to power up despite totally discharged energy storage. <ref> (b) shows this cold-start behavior for the AEM30940 IC connected to the RF frontend presented in section <ref>. The IC's overcharge protection has been configured to V_ovch=2.7 and the under-voltage lockout level to V_ovdis=2.2. The regulator output voltage is V_lv=1.2. 

At t=0, the harvesting circuit is supplied with a 915MHz RF input signal with a power of -15dBm. After 35, the initial wake-up phase completes as the intermittent storage capacitor C_boost charges to the programmed maximum. The activation of the output regulator V_lv and the corresponding inrush currents combined with the limited input power results in consecutive under-voltage lockouts until normal operation is reached after 56. Finally, after 93, C_boost is fully charged, and the circuit operates in overcharge protection mode. 







§ CONCLUSION

The paper proposed and evaluated a WPT sub-system, composed of an impedance matching network, rectifier, and power management with energy storage, optimized to a single narrow-band channel at 915. The experimental evaluation showed an overall PCE of over 30 at an input power of -10dBm and a peak efficiency of 57 at 3dBm. In comparison to commercial solutions, the proposed RF frontend showed an increased PCE for input power levels between -11dBm and 5dBm.
The commercial IC AEM30940 from epeas allows to temporarily provide more power than is being harvested, acting as a buffer and keeping the voltage supply constant, which is especially important for miniaturized IoT devices. Using the proposed RF frontend, the AEM30940 is able to exit cold start after 56 from an input power as low as -15dBm and switches to normal operation. 












 Paper Title: Long-Term Energy Management for Microgrid with Hybrid Hydrogen-Battery Energy Storage: A Prediction-Free Coordinated Optimization Framework.


1
.001




N.Qi et al. Long-Term Energy Management of Microgrid    



N. Qi et al.


mode = title]Long-Term Energy Management for Microgrid with Hybrid Hydrogen-Battery Energy Storage: A Prediction-Free Coordinated Optimization Framework  





















































































1]Ning Qi
[1]
nq21767@columbia.edu 
Conceptualization, Modeling, Methodology, Software, Writing

2]Kaidi Huang
Methodology, Proof

1]Zhiyuan Fan
Modeling, Revision




1]Bolun Xu
Supervision, Revision, Funding Support






[1]Department of Earth and Environmental Engineering, Columbia University, New York, NY 10027, USA
[2]Department of Electrical Engineering, Tsinghua University, Beijing 100084, China


[1]Corresponding author 



§ ABSTRACT 

This paper studies the long-term energy management of a microgrid coordinating hybrid hydrogen-battery energy storage. We develop an approximate semi-empirical hydrogen storage model to accurately capture the power-dependent efficiency of hydrogen storage. We introduce a prediction-free two-stage coordinated optimization framework, which generates the annual state-of-charge (SoC) reference for hydrogen storage offline. During online operation, it updates the SoC reference online using kernel regression and makes operation decisions based on the proposed adaptive virtual-queue-based online convex optimization (OCO) algorithm. We innovatively incorporate penalty terms for long-term pattern tracking and expert-tracking for step size updates. We provide theoretical proof to show that the proposed OCO algorithm achieves a sublinear bound of dynamic regret without using prediction information. Numerical studies based on the Elia and North China datasets show that the proposed framework significantly outperforms the existing online optimization approaches by reducing the operational costs and loss of load by around 30% and 80%, respectively. These benefits can be further enhanced with optimized settings for the penalty coefficient and step size of OCO, as well as more historical references.













  * Long-term energy management of microgrid considering seasonal uncertainties and seasonal storage

  * A prediction-free two-stage coordinated optimization framework

  * SoC reference of hydrogen storage generated from kernel regression and historical and AI-generated scenarios

  * A virtual-queue-based online convex optimization algorithm with expert-tracking

  * Numerical studies on Elia and North China with ground-truth datasets spanning 10 years





Long-Term Energy Management Hydrogen Hybrid Energy Storage Online Convex Optimization Microgrid 

[
    [
    July 31, 2024
=================






§ INTRODUCTION




 §.§ Background and motivation


A microgrid is a self-contained electrical network with resources including energy storage (ES), renewable energy sources (RES), and controllable loads, operated in either grid-connected or island mode <cit.>. Microgrids enhance energy resilience, promote decarbonization, and reduce transmission system investments, but the volatility of RES poses challenges to short-term supply-demand balances <cit.>. Besides, seasonal variations in RES availability <cit.> and extreme weather events <cit.> have highlighted the significance of the long-term energy management of microgrids. 

Hybrid energy storage system (HESS) <cit.> offers a promising way to guarantee both the short-term and long-term supply-demand balance of microgrids.  HESS is composed of two or more ES units with different but complementing characteristics, such as duration and efficiency. 

In day-ahead or intra-day operations, batteries can effectively address the uncertainties introduced by RES and demand. For long-term operation, hydrogen storage consisting of electrolyzer and fuel cell can provide efficient solutions to seasonal energy shifting <cit.>. In this paper, we focus on a typical application: hybrid hydrogen-battery energy storage (H-BES). Given the differences in storage properties and unanticipated seasonal uncertainties, designing an effective long-term energy management framework for microgrids with H-BES is significant but challenging.



 §.§ Literature review


Previous research mainly focuses on the short-term energy management of microgrids with H-BES. Two-stage robust optimization is proposed in <cit.> for the market operation of H-BES, where the uncertainties from RES are modeled by uncertainty sets. A two-stage distributionally robust optimization-based coordinated scheduling of an integrated energy system with H-BES is introduced in <cit.>, where an ambiguity set is employed to model the uncertainties from RES and integrated energy loads. Two-stage stochastic energy management of H-BES is proposed in <cit.>, where the uncertainties from RES, load, and prices are modeled by typical scenarios. However, these works rely solely on offline optimization methods with predefined uncertainty modeling, which may face optimality or feasibility issues in real-time operation. This motivates the research on real-time energy management with online optimization methods, such as the rolling-horizon method. Model predictive control (MPC) is the widely used rolling-horizon method and multi-level MPC controllers are developed for microgrids with hydrogen or H-BES in <cit.>. An actor-critic deep reinforcement learning method is proposed in <cit.> to address multi-timescale coordinated dispatch of microgrid with hybrid battery and supercapacitor. MPC and approximate dynamic programming approach are jointly utilized for multi-stage coordinated dispatch <cit.>, which achieves robust real-time performance through continuously updated forecasts. However, the limitations in the aforementioned works mainly lie in (i) The short-term energy management methods may face infeasibility issues in the long-term operation when considering seasonal variations RES and load.  (ii) The performance of these techniques strongly depends on the accuracy of the prediction of uncertainties. However, the predictions are practically
[A]i, pIndices for parallel learning sequence and hydrogen storage efficiency segment, respectively
[A]s, tIndices for historical scenarios and time period, respectively
[A]Ω_I, Ω_PSets for parallel learning sequence and hydrogen storage efficiency segment, respectively
[A]Ω_S, Ω_TSets for historical scenarios and time period, respectively
[A]x, ξSets for decision variables and stochastic parameters, respectively
[B]ESEnergy storage
[B]HESSHybrid energy storage system
[B]H-BESHybrid hydrogen-battery energy storage
[B]MPCModel predictive control
[B]OCOOnline convex optimization
[B]RESRenewable energy sources
[B]RMSERoot mean square error
[B]RegRegret for OCO algorithm
[B]SDPStochastic dynamic programming
[B]SoCState of charge
[C]A_p ,  B_pSlope and the intercept of piecewise linear charging segments, respectively
[C]C_p ,  D_pSlope and the intercept of piecewise linear discharging segments, respectively
[C]P^B, P^HUpper power bounds of battery and hydrogen storage, respectively
[C]P^BLower power bound of battery
[C]E^B, E^HUpper SoC bounds of battery and hydrogen storage, respectively
[C]E^B, E^HLower SoC bounds of battery and hydrogen storage, respectively
[C]η^B,c, η^B,dCharging and discharging efficiency of battery storage, respectively
[C]η^H,c, η^H,dCharging and discharging efficiency of hydrogen storage, respectively
[C]εSelf-discharge rate of battery storage
[C]P^D, P^DLower and upper power bounds of diesel generator, respectively
[C]RD^D, RU^DDownward and upward ramping rates of  diesel generator, respectively
[C]c^B, c^H, c^D, c^LMarginal discharge costs of battery and hydrogen storage, fuel price diesel generator, load curtailment price, respectively
[C]ω_s,t, φDynamic weights for historical scenarios and penalty coefficient for reference tracking, respectively
[C]α_i,t, β_i,t, γstep sizes for OCO algorithm, respectively
[C]Q_i,tVirtual queue for OCO algorithm
[D]E_t^B, E_t^HState of charge of battery and hydrogen storage, respectively
[D]P_t^B,c, P_t^B,dCharging and discharging power of battery storage, respectively
[D]P_t^H,c, P_t^H,dCharging and discharging power of hydrogen storage, respectively
[D]z_t^c, z_t^dBinary variables for charging and discharging segments of hydrogen storage, respectively
[D]h_t^c, h_t^dHydrogen production and consumption, respectively
[D]P_t^D, P_t^L, P_t^RDiesel generator power, loss of load power and dispatched renewable power, respectively
[
    

    
    

    
    2
    
    
]


unavailable or unreliable for microgrid operators.

To address the first limitation, recent studies have started to explore the long-term energy management of microgrids, which aims to solve the multi-time-period dispatch with non-anticipativity. Stochastic dynamic programming is technically sound, which can decompose the multi-period dispatch problem into sequential single-period dispatch problems through value function. And it is applied in <cit.> by learning the value function of H-BES. However, it becomes computationally intractable to train the value function if the storage duration spans multiple months. A continuous spectrum splitting approach is proposed in <cit.> to assign low-frequency uncertainty scenarios to hydrogen and high-frequency uncertainty scenarios to batteries for power balance, but this approach is designed for the planning of H-BES. A data-driven coordinated dispatch framework is proposed in <cit.>, where the state of charge (SoC) reference for hydrogen storage is generated based on historical simulations. This reference is then updated and embedded into MPC for real-time operation. However, the use of MPC makes the entire framework dependent on forecasting. 

Additionally, prediction-free online optimization methods are gaining increased attention. Lyapunov optimization
and online convex optimization (OCO) are effective representatives <cit.>. Lyapunov optimization adopts a “1-lookahead” pattern, where uncertainties are observed first, followed by solving the Lyapunov drift problem <cit.>. It has wide applications in demand response <cit.>, electric vehicle charging <cit.>, microgrid <cit.>, etc. The long-term operational cost minimization of hydrogen-based building energy systems is transformed into several single-slot subproblems using Lyapunov optimization <cit.>. A joint energy scheduling and trading algorithm based on Lyapunov optimization and a double-auction mechanism is designed in <cit.> to optimize the long-term energy cost of each microgrid. However, in some cases, the uncertainties can not be observed before decision-making and 
Lyapunov optimization becomes inapplicable. For instance, storage participants bid with unknown future prices, and the prices are cleared by the market after the bidding process <cit.>.

Instead, OCO adopts a “0-lookahead” pattern, where the decision is made before the observation of uncertainties. 
And OCO has been utilized in demand side management <cit.> and ancillary services <cit.> due to its completely prediction-free and fast response nature. However, to the best of our knowledge, no research has addressed the long-term energy management of microgrids with H-BES within the OCO framework. The application of OCO in the focused topic may face the following challenges: (i) OCO is problem-dependent without a predefined mathematical formulation, and there is no prior experience available as a reference for designing OCO for microgrids with H-BES. (ii) Although recent works <cit.> have embedded inter-temporal constraints into the OCO framework, OCO still risks falling into local optima due to its myopic nature. (iii) OCO aims to achieve regret (Reg) that grows sublinearly with time horizon T. However, most of the existing OCO algorithms fail to address the sublinear bounds for dynamic Reg <cit.> or require prediction information to improve the performance <cit.>. Please see Table <ref> for a comprehensive comparison. 




 §.§ Research gap


Existing literature is summarized in Table <ref>. Although some works achieve good results in the long-term energy management of microgrids with H-BES, there are still several research gaps that have not been adequately addressed.

(1) Most existing studies employ a simplified operational model for hydrogen storage, using a constant energy conversion efficiency regardless of whether the storage operates at full capacity. However, the efficiency of hydrogen storage varies with the charge/discharge power and follows a nonlinear function <cit.>. Using a simplified model can result in sub-optimal or even infeasible solutions <cit.>. Therefore, it is crucial to incorporate this nonlinearity into the microgrid energy management with H-BES.

(2) Current microgrid energy management approaches either employ offline optimization methods (e.g., robust optimization <cit.>, frequency-domain method <cit.>) or prediction-dependent online optimization methods (e.g., MPC <cit.>, stochastic dynamic programming <cit.>). However, the distribution and prediction information is often inaccurate or unavailable in practical microgrid operations. Thus, designing a prediction-free optimization framework for microgrid energy management with H-BES is necessary.

(3) OCO is a promising “0-lookahead” online optimization method originating from the fields of machine learning and control <cit.>-<cit.>. However, OCO lacks a global view of long-term operations and adaptability to the high volatility of microgrids. Hence, it is important to extend traditional OCO methods to incorporate long-term operational patterns and time-varying properties. 





 §.§ Contributions


Motivated by the research gaps, this paper proposes a prediction-free coordinated optimization framework for long-term energy management of microgrid with H-BES while incorporating the nonlinearity of hydrogen storage and seasonal uncertainties from RES and load. Specifically, our contributions are threefold:

(1) Modeling: We propose an approximate semi-empirical hydrogen storage model using piecewise linear relaxation, which accurately captures the power-dependent efficiency of hydrogen storage. Simulations demonstrate that, compared to the constant efficiency model, the proposed approximation model avoids both overly optimistic and overly conservative strategies. This results in a reduction of the practical yearly operational cost by 10% or 36%, and a decrease in yearly loss of load by 1.94 MWh or 3.85 MWh.

(2) Solution Methodology: We introduce a prediction-free two-stage coordinated optimization framework. In the offline stage, the ex-post SoC references for hydrogen storage are generated by deterministic mixed-integer linear programming with historical and AI-generated data on RES and load. These references help to avoid myopic online decision-making and are incrementally updated by kernel regression with newly observed data. Subsequently, we develop an adaptive virtual-queue-based OCO algorithm for prediction-free online decision-making. Compared to the traditional OCO algorithm <cit.>, the proposed method innovatively incorporates a penalty term for long-term pattern tracking and expert-tracking for step size updates. The proposed OCO algorithm is proven to achieve a sublinear bound for dynamic regret. 


(3) Numerical Study: We demonstrate the effectiveness of the proposed framework using ground-truth data from Elia <cit.> and North China <cit.>. Simulations show that introducing the reference significantly reduces operational costs and loss of load by 40%-57% and 60%-90%, respectively. Furthermore, compared to the prediction-dependent MPC method, the prediction-free OCO method further decreases operational costs and loss of load by 24%-29% and 73%-89%, respectively. These benefits can be further enhanced with optimized settings for the penalty coefficient and step size of OCO, as well as more historical references.










 §.§ Paper Organization


We organize the remainder of the paper as follows. Section <ref> presents an approximate semi-empirical modeling of hydrogen storage. Section <ref> provides the problem formulation for long-term energy management of the microgrid with H-BES. Section <ref> introduces the prediction-free two-stage coordinated optimization framework and the proof of OCO performance. Section <ref> describes numerical case studies to verify the effectiveness of the proposed framework. Finally, we conclude this paper in Section <ref>.





§ APPROXIMATE SEMI-EMPIRICAL HYDROGEN ENERGY
 STORAGE MODEL
 


 §.§ Structure of hydrogen storage system


A hydrogen storage system is composed of several key components, such as electrolyzers, hydrogen storage tanks, fuel cells, compressors, and other auxiliary equipment, as illustrated in Fig. <ref>. Electrolyzers convert electrical energy into chemical energy by producing hydrogen and oxygen. This paper considers the most mature and commonly used alkaline water electrolyzer. Hydrogen storage tanks are used to store the produced hydrogen. Fuel cells convert the stored hydrogen back into electricity, and we consider the typical type, proton exchange membrane fuel cell (PEMFC). Other auxiliary equipment, including the compressor, cooling system, and control system, is excluded from the modeling.





 §.§ Alkaline water electrolyzer model


(1) Polarization curve

The polarization curve describes the electrochemical behavior of an electrolyzer, modeling the relationship between current and voltage. To account for the impact of temperature and pressure on the thermodynamics and electrochemical process within the electrolyzer, we combine the most used model proposed by Ulleberg <cit.> and the modified model proposed by Sanchez <cit.>:

    U_cell^E   = U_rev  + [(r_1 + d_1) + r_2 ·θ + d_2 · P] ·iA
        + s ·log[(t_1 + t_2/θ + t_3/θ^2) ·iA + 1]

where the reversible voltage and cell voltage of the electrolyzer are defined as U_rev and U_cell^E. Temperature and pressure are given by θ and P. The current and effective area of the electrode is defined as i and A. Parameters r_1, r_2, d_1, d_2, t_1, t_2, t_3, s are the constants which can be learned from the experimental data.

(2) Faraday efficiency

Faraday efficiency is defined as the ratio of measured hydrogen production to the theoretical value. For an alkaline electrolyzer, the Faraday efficiency typically ranges from 85% to 95% and is affected by temperature. We adopt the four-parameter Faraday efficiency model as (<ref>). 

    η_F=((i/A)^2/f_1+f_2 ·θ+(i/A)^2) ·(f_3+f_4 ·θ)

where Faraday efficiency is defined as η_F. Parameters f_1, f_2, f_3, f_4 are the constants which can be learned from the experimental data.

(3) Approximate charging efficiency

According to Faraday's law, the hydrogen production rate is defined as (<ref>). The charging efficiency is given by (<ref>).

    
    h^c=3600 ·η_F· M · i· N2 F
       η^H,c=h^c·LHVP_Stack=3600 ·η_F·  M·LHV2 F · U_cell


where h^c is the hydrogen production rate of electrolyzer. M is the molar mass of hydrogen. F is the Faraday's constant, i.e., 96485 C/mol. N is the number of cells of the stack. LHV is the lower heat value of hydrogen, i.e., 33.33 kWh/kg.

As illustrated in Fig. <ref>, the blue curves from the semi-empirical model are non-linear and power-dependent, including a peak in efficiency at around 20% of the rated power. Therefore, a constant conversion efficiency cannot capture the variations in efficiency. To facilitate dispatch optimization, we adopt a piecewise linear approximation for hydrogen production, depicted by red dashed lines.




 §.§ PEMFC model


(1) Polarization curve

The polarization curve of PEMFC is typically modeled using the equivalent circuit model proposed by Amphlett <cit.>. The cell voltage U_cell^F is given by (<ref>), which equals the open circuit voltage E_Nernst dropped by three types of irreversible losses: activation losses U_act, ohmic losses U_ohmic, and concentration losses U_con.

    
    U_cell^F=E_Nernst -U_act -U_ohmic -U_con 
        E_Nernst =1/2 F[Δ G-Δ S (θ-θ_ref )
       +R ·θ(log( P_H_2)+log (P_O_2)/2)]
        U_act =a_1+a_2 ·θ+a_3 ·θ·log (C_O_2)+a_4 ·θ·log(i)
        U_ohmic =i · R_ohmic =i(r_M· l/A+R_c)
        U_con =B ·log(1-JJ_max), J=iA


where Δ G is the Gibbs free energy. Δ S is the entropy change. R is the gas constant (8.314 J/(K·mol)). P_H_2 and P_O_2 are the partial pressures of hydrogen and oxygen respectively. T_ref is the reference temperature (298.15 K).  C_O_2 is the oxygen concentration at the surface of the cathode catalyst. r_M is the resistivity of the electrolyte membrane. l is the thickness of the electrolyte membrane. B is the concentration overpotential coefficient. J and J_max are the current density and its maximum value. a_1, a_2, a_3, a_4 are constants that can be learned from the experimental data.

(2) Approximate discharging efficiency

According to Faraday's law, the hydrogen consumption rate is defined as (<ref>). The discharging efficiency is given by (<ref>).

    
    h^d=3600 · M · i· N2 F
       η^H,d=P_Stackh^d·HHV= 2 F · U_cell3600 M·HHV


where h^d is the hydrogen consumption rate of PEMFC. HHV is the higher heat value of hydrogen, i.e., 39.4 kWh/kg.

The blue curves from the semi-empirical model in Fig. <ref> are non-linear and power-dependent. We also adopt a piecewise linear approximation for hydrogen consumption, depicted by red dashed lines.





 §.§ Equivalent hydrogen storage model


The equivalent hydrogen storage model is presented in (<ref>). Constraint (<ref>) defines the relationship between SoC, charge power, and discharge power. Constraints (<ref>) limit the SoC of hydrogen storage within the bounds. Constraint (<ref>) guarantees ensures a sustainable energy state for hydrogen storage over cycles. Constraints (<ref>)-(<ref>) describe the tractable formulation of piecewise linear charging and discharging functions. Constraints (<ref>) limit hydrogen storage's charging and discharging power. 

 Constraints: ∀ t∈Ω_T ∀ p∈Ω_P


    E_t+1^H=E_t^H+Δ t(h_t^c-h_t^d)-E_t^H,L
       E^H≤ E_t^H≤E^H
        E_T^H≥ E_0^H
        h_t^c=∑_p (A_p P_p,t^H,c+B_p z_p,t^c), h_t^d=∑_p (C_p P_p,t^H,d+D_p z_p,t^d)
        P_t^H,c=∑_p P_p,t^H,c, P_t^H,d=∑_p P_p,t^H,d
       ∑_p z_p,t^c=1, ∑_p z_p,t^d=1
       P_p^Hz_p,t^c≤ P_p,t^H,c≤P_p^Hz_p,t^c, 0≤ P_p,t^H,d≤P_p^Hz_p,t^d




where Ω_T and Ω_S are the set of time and parameter segments, respectively. P_t^H,c, P_t^H,d, and E_t^H are decision variables for the charge power, discharge power, and SoC of hydrogen storage. The SoC of hydrogen storage can be measured by the hydrogen mass or as a ratio of the rated capacity. E_t^H,L is the hydrogen load for industrial production processes, such as fertilizer manufacturing and steel-making. E^H and E^H are the lower and upper bounds of SoC. P^H and P^H are the lower and upper bounds of power. The lower charging power bound is set by the minimum operating power of the electrolyzer, typically 15%-20% of the nominal power. A_p and B_p are the slope and the intercept of piecewise linear charging segments. C_p and D_p are the slope and the intercept of piecewise linear discharging segments. z_p,t^c and z_p,t^d are binary variables for piecewise linear function.




§ LONG-TERM ENERGY MANAGEMENT OF MICROGRID
 



 §.§ Microgrid structure


In this paper, we only consider the island mode of the microgrid, and the microgrid structure is illustrated in Fig. <ref>. The microgrid consists of renewable generators (wind and solar), diesel generators, H-BES and local loads.




 §.§ Problem formulation

The objective defined in (<ref>) aims to minimize the system cost. This cost comprises the production costs of the diesel generator, penalties for load curtailment (island mode), and operational costs of H-BES. Constraints (<ref>) and (<ref>) define the power bounds and ramping bounds of the diesel generator. Constraints (<ref>) define the constraints for battery, which are similar in formulation to those for hydrogen storage (<ref>), as both types of storage involve constraints on charging and discharging rates, SoC, etc. However, it is important to note that the battery efficiency is considered to be constant, there is no minimum charging power limit in (<ref>), and the self-discharge rate should be considered in (<ref>). Constraints (<ref>) limit the load curtailment and dispatchable RES. Constraint (<ref>) limits the power import from the main grid. Power balance constraint is defined as (<ref>). The complementary constraints for charging and discharging of battery and hydrogen storage are relaxed and have been removed from the model since sufficient conditions are satisfied <cit.>, i.e., discharging price (“+”) is greater than the charging price (“0”). Moreover, the power flow constraints are overlooked within the dispatch model since the microgrid network is generally designed with high reliability and large redundancy <cit.>.

Objective Function:


    xmin G(x, ξ)=∑_t∈Ω_T(C_t^L+C_t^D+C_t^B+C_t^H)
       
       C_t^L=c^LP_t^LΔ t, C_t^D=c^DP_t^DΔ t, C_t^B/H=c^B/HP_t^B/H,dΔ t


where c^L and P_t^L are the load curtailment price and load curtailment power. c^D and P_t^D are the fuel price and power of diesel generator. c^B and c^H are marginal discharge costs of battery and hydrogen storage. Δ t is the time interval. 

Constraints: ∀ t∈Ω_T


    P^D≤P_t^D≤P^D         

    -RD^D≤P_t+1^D-P_t^D≤RU^D        






    E_t+1^B=(1-εΔt)E_t^B+Δt(η^B,cP_t^B,c-P_t^B,d/η^B,d)        
 
    E^B≤E_t^B≤E^B        

    E_T^B≥E_0^B        

    0≤P_t^B,c≤P^B        

    0≤P_t^B,d≤P^B        






    0≤P_t^L≤ξ_t^L        

    0≤P_t^R≤ξ_t^R        





    0≤P_t^G≤P_t^G       




    P_t^G+P_t^R+(P_t^B,d-P_t^B,c)+(P_t^H,d-P_t^H,c)+P_t^L= ξ_t^L       

where P^D and P^D are the lower and upper power bounds of diesel generator. RD^D and RU^D are downward and upward ramping rates of  diesel generator. P_t^B,c, P_t^B,d, and E_t^B are decision variables for the charge power, discharge power, and SoC of battery. η^B,c and η^B,d are the charge and discharge efficiency of battery. ε is the self-discharge rate of battery. E^B and E^B are the lower and upper SoC bounds of battery. P^B is the upper power bound of battery. ξ_t^L and ξ_t^R are the load power and available RES power with uncertainties. P_t^R is the dispatched RES power. The set of stochastic parameters is given by ξ={ξ_t^L,ξ_t^R}. The set of decision variables is given by x={P_t^L,P_t^D,P_t^R,P_t^B,c/d,P_t^H,c/d,E_t^B,E_t^H,h_t^c/d,z_t^c/d}.

The multi-time-period economic dispatch of microgrid with H-BES (P_1) is summarized in (<ref>). Next, we present the methodology for solving this problem. 

  (P_1) xmin      G(x,ξ)
 
 s.t.       (<ref>), (<ref>)-(<ref>)



§ PREDICTION-FREE COORDINATED OPTIMIZATION FRAMEWORK
 



 §.§ Motivations

Solving the problem (<ref>) has the following challenges:

(1) Non-anticipatively : The long-term energy management of the microgrid typically spans more than one month or one season. Nevertheless, the forecast accuracy is acceptable only for several hours ahead. Hence, the load power and available RES power are unanticipated in the long-term optimization. And online optimization methods should be adopted to decompose the long-term optimization problem into several short-term optimization problems.

(2) Storage Dispatch Priority: Batteries with lower marginal discharge costs will be given priority over hydrogen storage with higher marginal discharge costs. We defer the complete proof to Appendix <ref>. The battery-prioritized strategy is feasible and economical for short-term operation. However, this approach does not account for seasonal variations in RES and load, which will result in a lack of pre-stored hydrogen and load losses in long-term operations. Therefore, it is necessary to design a “reference” with a global view to help guide hydrogen storage actions.

(3) Convexity: The piecewise linearization will introduce nonconvexity to the optimization, which contradicts the overall logic of most convex optimization approaches. However, introducing a global “reference” can mitigate this challenge by pre-determining the efficiency.




 §.§ Two-stage coordinated optimization framework

We propose a two-stage coordinated optimization framework as illustrated in Fig <ref>. The proposed framework consists of both online and offline stage optimization. The offline stage aims to generate the ex-post SoC references for hydrogen storage using historical data on RES and load. These references can help avoid myopic decision-making and will be incrementally updated by kernel regression with newly observed data. Subsequently, online decisions are made using an adaptive virtual-queue-based OCO algorithm.



 §.§ Offline-stage optimization

Firstly, sequential sequences of scenarios, denoted as ξ_s={ξ_s,t^L,ξ_s,t^R},t∈Ω_T={1,2,⋯,T}, s∈Ω_S={1,2,⋯,N}, are generated from historical data of the past few years. Additionally, to account for climate change and enhance the diversity of references, we can also collect references from different months and seasons. For instance, if we focus on a seasonal dispatch problem and have historical data for 5 years, then T=1 season and N=5 × 4. To enhance adaptability to extreme weather conditions, we add extreme scenarios into the historical data using Generative Adversarial Networks <cit.>. Afterward, we can solve the deterministic 
mixed-integer linear programming (MILP) as (<ref>) to generate the SoC references of hydrogen storage, i.e., E_s^H,*={E_s,t^H,*},t∈Ω_T, s∈Ω_S.

  (P_2) x_smin      G(x_s,ξ_s)
 
 s.t.       (<ref>), (<ref>)-(<ref>)




 §.§ Online-stage optimization


(1) Data-Driven Reference Tracking

Inspired by <cit.>, we propose a data-driven reference tracking method to combine both the 'lookback' pattern from historical data and the 'lookahead' pattern from newly observed data. Firstly, we define ξ_[t] as the observed sequence for uncertainties from the first time slot to the current time slot t in (<ref>). Additionally, ξ_s,[t] defined in (<ref>) represents the corresponding historical sequence for uncertainties in scenario s. Subsequently, by checking the similarity between ξ_[t] and ξ_s,[t], dynamic weights ω_s,t are assigned to each historical scenario based on the Gaussian kernel function and Euclidean distance, as outlined in (<ref>). To account for the temporal dynamics, the Gaussian kernel function is modified with a scaling factor t. And the optimal bandwidth σ can be found through heuristic methods such as the bisection method. Additionally, the weights are updated in real-time dispatch instead of using average or heuristic values. Finally, the SoC reference of hydrogen storage is updated as (<ref>). This updated reference also determines the efficiency segment of hydrogen storage, eliminating the nonconvexity issue that arises when using convex optimization approaches.


    ξ_[t]={ξ_1^G,ξ_1^L,ξ_1^R,⋯,ξ_t^G,ξ_t^L,ξ_t^R}
       ξ_s,[t]={ξ_s,1^G,ξ_s,1^L,ξ_s,1^R,⋯,ξ_s,t^G,ξ_s,t^L,ξ_s,t^R}
       ω_s,t=K_t(ξ_[t],ξ_s,[t])∑_s^'=1^NK_t(ξ_[t],ξ_s^',[t]), K_t(x,y)=e^-(x-y_2)^2/tσ^2
       E_[t]^H,R=∑_s=1^Nω_s,tE_s,[t]^H,*



(2) Real-Time Corrective Dispatch

Real-time corrective dispatch (P_3) is formulated in (<ref>), which aims to minimize the instant operational cost while tracking the SoC reference of hydrogen storage. φ is the penalty coefficient to control the SoC deviation from the reference. P_3 admits a compact form in (<ref>). f_t and g_t represent the time-varying objective function and time-varying constraints due to hydrogen storage SoC reference E_t^H,R and uncertainties ξ, respectively. By leveraging the Lagrangian Relaxation, we can obtain the optimum by (<ref>). λ_t is the dual variables of the constraints g_t. ⟨ x,y⟩ denotes the standard inner product. However, without prior knowledge of uncertainties ξ, f_t and g_t are unknown to the online decision-maker. Hence, we next design a VQB-OCO algorithm to solve this issue.

  (P_3) x_tmin      G(x_t,ξ_t)+φ(E_t^H-E_t^H,R)^2
 
 s.t.       (<ref>), (<ref>)-(<ref>)




    
    min_x f_t(x_t)  s.t. g_t(x_t)≤0
        x_t=min_x{f_t(x)+⟨λ_t, g_t(x)⟩}



(3) VQB-OCO Algorithm

The key idea of VQB-OCO is to use information from past time to approximate the current situation. The virtual queue is employed as the substitution of unknown dual variables. Hence, we design the update policy for the virtual queue, decisions, and weights in (<ref>)-(<ref>). Finally, we can obtain the weighted average value of dispatch decision as (<ref>). The VQB-OCO algorithm and the overall two-stage coordinated optimization framework are summarized in Algorithm <ref>.


    Q_i,t-1=Q_i,t-2+β_t-1[g_t-1(x_t-1)]_+


    x_i,t   =min_x{α_i,t-1⟨∂ f_t-1(x_t-1), x⟩+
       α_i,t-1β_i,t-1⟨ Q_i,t-1, [g_t-1(x)]_+⟩+x-x_i,t-1^2}


    ℓ_i,t-1=⟨∂ f_t-1(x_t-1), x_i,t-1-x_t-1⟩, ρ_i,t=ρ_i,t-1e^-γℓ_i,t-1∑_i=1^Nρ_i,t-1e^-γℓ_i,t-1


    x_t=∑_t=1^Nρ_i,tx_i,t



 f_t(x) is approximated using the first-order Taylor expansion ⟨∂ f_t-1(x_t-1), x⟩. The term λ_t is substituted by a virtual queue Q_i,t-1. The constraint function g_t(x) is replaced by the clipped constraint function [g_t-1(x)]_+. A regularization term |x-x_i,t-1|^2 is added to ensure the convexity of the optimization problem and to enhance the convergence of the algorithm.


Determining the learning rate (step size) is important yet challenging. We assign different learning rates to the first two terms, α_i,t-1 and β_i,t-1. Rather than utilizing fixed or adaptive learning rates, we employ the expert-tracking algorithm proposed by <cit.>, which computes x_t in parallel with various learning rates as described in equation (<ref>). The weights for each expert ρ_i,t are updated based on their empirical performance using an exponential function, as shown in equation (<ref>).



Based on our previous work <cit.>, the dual variables of the long-term constraints remain fixed when the optimum does not reach the constraint bounds. However, when the optimum reaches these bounds, the dual variables increase, representing a penalty. The update of the virtual queue follows the same pattern as described in equation (<ref>) to limit constraint violations.




(4) Performance of VQB-OCO

OCO focuses on the performance of regret (Reg), as defined in (<ref>), where y_t is the global optimum. Various OCO algorithms ensure that Reg is a sublinear function of T by designing parameters and update policy, as it implies that the algorithm performs as well as the global optimum in hindsight as T approaches infinity. Next, we provide parameter settings and a proof to achieve strictly sublinear dynamic regret.




    Reg=∑_t=1^T[f_t(x_t)-f_t(y_t)]


Assumption 1. The functions f_t and g_t are convex. The feasible set X is convex and closed, and it has a bounded diameter d(X), i.e.,

    ∥ x-y∥≤ d(X), ∀ x,y∈X


Assumption 2. There exists a positive constant 
F such that

    | f_t(x)-f_t(y)|≤ F, ∥ g_t(x)∥≤ F, ∀ t∈Ω_T, ∀ x,y∈X



Assumption 3. The subgradients ∂ f_t(x) and ∂ g_t(x) exist. And there exists a positive constant G such that

    ∥∂ f_t(x)∥≤ G, ∥∂ g_t(x)∥≤ G, ∀ t∈Ω_T, ∀ x,y∈X



Given the assumptions 1–3, and parameters setting as (<ref>), κ∈ [0, c], c ∈ (0, 1), α_0 > 0, β_0 > 0, and γ_0 ∈ (0, 1/(√(2G))) are constants. Then, we have the performance of Reg and Vio as (<ref>). 


    M=⌊κlog_2(1+T)⌋+1, α_i,t=α_02^i-1t^c, β_i,t=β_0√(α_i,t),  γ=γ_0T^c


    Reg=𝒪(T^c (1 + P_x)^1 - κ + T^1 - c (1 + P_x)^κ)



Proof: The performance of the proposed OCO algorithm achieves a similar performance with <cit.> which achieves 𝒪(T^max{1-a-c, c}) for dynamic regret with the help of prediction data. And it outperforms the performance of <cit.> and <cit.>, achieving dynamic regret with a linear function of P_x, which is not satisfactory. Moreover, by setting κ=c=0.5, the proposed OCO algorithm achieve the performance of 𝒪(√(T(1 + P_x))), which aligns with the performance of <cit.> where
long term constraints are not considered. Hence, the proposed OCO algorithm is no worse than the existing versions. We defer the complete proof to Appendix <ref>.





§ CASE STUDIES
 



 §.§ Set-up


The main parameters and configurations are listed in Table <ref>. Specifically, the capacities of the battery and hydrogen storage are half of the load capacity. The storage durations of the battery and hydrogen are 2 hours and 400 hours, respectively. The installed capacity of renewables is 200 kW, comprising an equal share of solar and wind. The cost coefficients can be found in <cit.>.

We demonstrate the effectiveness of the proposed method based on two datasets: (1) We use the 15-minute historical data on solar, wind, and load from 2014 to 2023 obtained from Belgium's transmission system operator (Elia) <cit.> for the baseline case study. (2) We also use the hourly historical data of wind and load from 1981 to 2020 in North China <cit.> to demonstrate the impact of data resolution and data quantity.

The optimization is coded in MatLab with Yalmip interface and solved by Gurobi 11.0 solver. The programming environment is Intel Core i9-13900HX @ 2.30GHz with RAM 32 GB.






 §.§ Offline-stage optimization


(1) Data visualization

We first show the monthly average available renewable and load power of Elia from 2014 to 2023 in Figure <ref>. It is observed that all of them exhibit seasonal patterns. Wind power is abundant in spring and winter but scarce in summer, while solar power is relatively high in summer and extremely low in winter. Load power peaks in winter. Correspondingly, the net load also peaks in winter and hits a low in summer. Therefore, it indicates the critical role of hydrogen storage to address the seasonal variations in renewables and load, as well as to maintain the long-term energy balance of the microgrid.


(2) Impact of hydrogen storage efficiency model

Next, we compare the offline energy management performance in 2023 with different hydrogen models, including:

(E1): Piecewise linear model as proposed in (<ref>), and the parameters are fitted based on the experimental data as shown in Figure <ref> and <ref>.

(E2): Constant efficiency model with both the highest charging and discharging efficiencies of 63%.


(E3): Constant efficiency model with the lowest charging and discharging efficiencies, i.e., 53% and 45%, respectively.

The hydrogen storage SoC is shown in Figure <ref>. Other results are also summarized in Table <ref>. It is observed that using the highest constant efficiency model results in the most optimistic performance, with the lowest operational cost ($97,188), whereas the lowest constant efficiency model yields the highest operational cost ($132,933) due to the significant increase in costs of diesel generation and loss of load. However, using a constant efficiency model can lead to feasibility issues in practical operation, resulting in losses in either charging or discharging power. Additionally, the optimistic strategy generated by the highest efficiency model will introduce an additional loss of load cost of 1.94 MWh. Considering the practical consequences, E2 and E3 will increase the total system costs by 10% and 36%, respectively, compared to E1. This result demonstrates that the proposed model can capture the characteristics of power-dependent efficiency and achieve more reliable and economical performance in practice.






 §.§ Online-stage optimization


(1) Reference tracking

We test the reference tracking performance in 2023 with different methods, including:

(R1): Global optimal reference generated by deterministic multi-period optimization with perfect knowledge of uncertainty realizations.

(R2): The proposed data-driven reference tracking, trained with 2014-2022 historical data.

(R3): The proposed data-driven reference tracking, trained with 2014-2022 historical data and AI-generated data. The AI-generated data is produced by randomly reducing the historical solar power but increasing the wind power by 10%-50% each quarter.

(R4-R6): Similar to the (R3) method, the AI-generated data is produced by:
- (R4): Randomly reducing both the historical solar and wind power by 10%-50% each quarter.
- (R5): Increasing the historical solar and wind power by 10%-50% each quarter.
- (R6): Using all the AI-generated data from (R3) to (R5).

(R7): The reference generated using the average historical performance.

The hydrogen SoC references are compared in Figure <ref>, and the tracking performance is summarized in Table <ref>. The root mean square error (RMSE) is calculated as the average difference between the generated reference and the global optimal reference. The optimal choice of σ obtained through the bisection method is 0.098. It is observed that the references generated by the proposed methods R2-R6 can better track the seasonal variations of RES and load, resulting in lower RMSE compared to the reference generated by R7. This is because the proposed methods employ kernel regression to update the weights of historical references instead of using fixed and average values. Additionally, additional generated data inputs will increase the tracking performance as they create new potential extreme scenarios. However, as is shown in the performance of R6, an excessive amount of generated data can lead to overfitting in the regression model, thereby reducing tracking accuracy. The average computation time for a single time interval is around 2 ms, which is acceptable even for minute-level scheduling and control.



Furthermore, we test the reference tracking performance on the North China dataset. The data visualization and reference tracking performance are shown in Figure <ref> in Appendix <ref>. Compared with the Elia dataset, both the wind and load data exhibit less variation across seasons and years. The maximum variations across years are 0.22 and 0.09 for wind and load, respectively, while for the Elia dataset, they are 0.25 and 0.15 for wind and load, respectively. Specifically, the load data in North China maintains the same shape across the years. The optimal choice of σ obtained through the bisection method is 50. The proposed tracking method performs the same as the averaged method, with an RMSE of 0.046. This is because the historical data and historical references show significant similarity across years, causing the proposed method to select the average value when updating weights. The above results demonstrate the benefit of using a data-driven reference tracking method when historical uncertainties exhibit significant variations across years. Additionally, an appropriate amount of AI-generated data can improve adaptability to extreme weather scenarios.



(2) Online decision-making

We test the online decision-making in 2023 with different dispatch methods, including:

(M0): Deterministic optimization with perfect knowledge of uncertainty realizations for the whole year results in the global optimum. This method is optimistic and not applicable in practice. Hence, it only serves as a baseline.

(M1): The proposed prediction-free coordinated approach, which utilizes OCO for online optimization and R5 for reference tracking.

(M2): The scheduling-correction method proposed by <cit.>, which utilizes MPC for online optimization and R5 for reference tracking.

(M3): Online optimization by OCO without reference tracking. 

(M4): Online optimization by MPC without reference tracking. 

The operational performance is summarized in Table <ref>. It is observed that the proposed method M1 outperforms the others in terms of cost-effectiveness, achieving an optimality gap of 27% compared with M0. This is due to its smallest loss of load and RMSE compared to the reference. Additionally, M1 and M2 obtain much better economic performance than M3 and M4. This can be explained by the hydrogen SoC as shown in Figure <ref>. M3 and M4 generate myopic decisions by continuously discharging the hydrogen storage to reduce short-term operational costs during the winter peak. They fail to charge the hydrogen storage during the renewable-rich spring and summer, resulting in an extremely low SoC after winter. Consequently, these myopic decisions prevent hydrogen storage from effectively shifting energy seasonally, leading to a substantial loss of load and low utilization of RES in practice. In contrast, M1 and M2 follow the pattern of reference while M1 has the better reference following performance (lower RMSE) since OCO utilizes the real-time observed data. This result demonstrates the benefit of introducing a global reference for the online optimization method. We also apply SDP algorithm
in <cit.> to this problem. However, the SDP cannot converge within 24 hr due to the “curse of dimensionality”. Therefore, it is infeasible to use SDP for long-term energy management of microgrid with H-BES.



Moreover, we compare the power dispatch strategies of H-BES and DG using M1 and M2, as shown in Figure <ref>. It is observed that M1 can better track the net load curve using only hydrogen storage actions. In contrast, M2 keeps charging hydrogen storage and uses DG when renewables are insufficient. This is because the OCO-based method simultaneously tracks the previous decisions and the reference, updating the strategy based on newly observed data, which is more adaptive to the time-varying environment. While the MPC-based method only tracks the reference and updates the strategy based on forecast data. Therefore, if the reference or forecast is not accurate, the MPC-based method may struggle to achieve good performance. This can also be explained by SoC gaps as shown in Figure <ref>. Furthermore, due to prediction errors, MPC-based online optimization may encounter infeasibility issues, resulting in additional loss of load and penalty costs. In contrast, the OCO-based method makes decisions based on observed data, thereby avoiding infeasibility issues. Regarding computational efficiency, it can be seen that the OCO method has better performance than MPC. Both methods achieve single-step optimization in tens of ms, which is acceptable for most online optimization scenarios.






Additional tests on the North China dataset align with the above results, showing that optimization with reference outperforms optimization without reference, and OCO-based methods outperform MPC-based methods. The results are summarized in Table <ref> and Figure <ref> in Appendix <ref>. Compared with the case results in Elia, both M1 and M2 achieve SoC strategies with smaller gaps from M0, due to better reference tracking performance. However, in terms of cost and reliability performance, the North China case shows worse results. This is because renewable energy in the North China case is solely supplied by wind power, leading to insufficient generation and higher load curtailment.






 §.§ Sensitivity analysis


In this subsection, we further investigate the key impact factor of the proposed optimization framework.

(1) Penalty Coefficient of Reference Tracking. The penalty coefficient represents the tradeoff between instant operational cost and reference tracking performance. However, since this reference is estimated from historical data, it may not be optimal for the current year. We compare the cost and tracking performance in Figure <ref> when scaling up the penalty coefficient φ. It is observed that RMSE is monotonically decreased with the penalty coefficient, while the operational cost initially decreases to a minimum value at φ = 90000 but then gradually increases with the penalty coefficient. This suggests that improving tracking performance does not always lead to lower operational costs.






(2) Reference and step size of OCO Algorithm. As illustrated in Section 5.3, the reference has a critical impact on operational performance. Table <ref> summarizes the performance using M1 with different references (i.e., fixed reference generated by R7 and updated reference generated by R5) and different step sizes (i.e., fixed step size proposed by <cit.> and step size generated by the proposed expert-tracking algorithm). It is observed that compared to fixed reference, using the proposed updated reference reduces operational cost by 1.09%-1.70% and the loss of load by 5.68%-8.52%. This highlights the importance of finding the right reference for hydrogen storage. Additionally, the proposed step size setting decreases the operational cost by 0.67%-1.29% and the loss of load by 0.06%-3.16%. This is because the step size generated by expert-tracking can better adapt to the changing cost function and avoid heuristic settings.




(3) Sizing of Renewables. We further compare the reliability performance of M1 and M2 in Figure <ref> when scaling up renewable capacity. It is observed that the loss of load decreases with increased renewable capacity. Moreover, M1 achieves an acceptable reliability level with twice the renewable capacity, while M2 requires at least 4 times the renewable capacity to meet reliability requirements. This demonstrates the benefit of the proposed method in reducing renewable planning costs. 





§ CONCLUSION
 

This paper proposes a prediction-free coordinated optimization framework for long-term energy management of microgrid with H-BES. To accurately captures the power-dependent efficiency of hydrogen storage, we propose an approximate semi-empirical hydrogen storage model using piecewise linear relaxation. Moreover, to address the long-term operational patterns of renewables and load and to eliminate dependence on predictions, we introduce a prediction-free, two-stage coordinated optimization framework. The key idea is to generate and track the SoC reference of hydrogen storage using historical scenarios and kernel regression to avoid myopic online decisions. And online decisions are made based on the proposed VQB-OCO algorithm by leveraging feedback control policy and newly observed data. Case studies on Elia and North China verify that:

(1) Compared to the constant efficiency model, the proposed approximation model avoids both overly optimistic and overly conservative strategies.

(2) The proposed optimization framework outperforms existing online optimization methods and achieves an acceptable gap compared to deterministic optimization with perfect foresight of uncertainties.

(3) The SoC reference of hydrogen storage is critical to overall performance. Therefore, more reliable historical or AI-generated scenarios, along with sophisticated techniques for setting penalty coefficients, are highly required.

(4) The OCO algorithm typically lacks a global view and is sensitive to step size settings. Thus, it is beneficial to incorporate a penalty term for long-term pattern tracking and expert-tracking for step size updates.

Further work will focus on addressing constraints violations in the OCO algorithm and extending the proposed framework to operations in a market environment.
























































     Appendix







§ PROOF OF STORAGE PRIORITY


The marginal discharge cost of battery is $10/MWh-$30/MWh, while it is $1/kg-$2/kg ($30/MWh-$60/MWh) for hydrogen storage, which is much higher than battery. Assume we have an optimal discharge power from H-BES, which is entirely supplied by the battery, i.e., x^1=P_t^B,d. Considering another optimum where the discharge power is a mix from both the battery and hydrogen storage, i.e., x^2=ρ P_t^B,d+(1-ρ) P_t^H,d. From the H-BES cost of the two optima (<ref>) and the marginal cost & efficiency of two types of storages, we can draw the conclusion that C_t^H-BES,2>C_t^H-BES 1. This indicates that hydrogen storage will not be activated until the battery is fully discharged or charged. Hence, we finish the proof.


    P_t^B,d=ρ P_t^B,d+(1-ρ) P_t^H,d
       C_t^H-BES,1=c^BP_t^B,dΔ t
       C_t^H-BES,2=(c^Bρ P_t^B,d+c^H(1-ρ)P_t^H,d)Δ t





§ PROOF OF BOUNDED DYNAMIC REGRET

Let {x_i,t} and {x_t} be the sequences generated by Algorithm <ref>. Let {y_t} be a global optimum in the feasible set X. From f_t is convex and (<ref>), we have:

    f_t(x_i,t) - f_t(y_t) ≤⟨∂ f_t(x_i,t), x_i,t - y_t ⟩
       ≤ G ∥ x_i,t - x_i,t+1∥ + ⟨∂ f_t(x_i,t), x_i,t+1 - y_t ⟩
       ≤G^2α_i,t/2 + 1/2 α_i,t∥ x_i,t - x_i,t+1∥^2 + ⟨∂ f_t(x_i,t), x_i,t+1 - y_t ⟩

For the rightmost term of (<ref>), we have:

    ⟨∂ f_t(x_i,t), x_i,t+1-y_t⟩
       =⟨β_i,t+1(∂[g_t(x_i,t+1)]_+)^TQ_i,t, y_t-x_i,t+1⟩
       +⟨∂ f_t(x_i,t)+β_i,t+1(∂[g_t(x_i,t+1)]_+)^TQ_i,t, x_i,t+1-y_t⟩

Since g_t is a convex function, it is trivial to show that [g_t]_+ is also convex; hence the first term of (<ref>) can be relaxed:

    ⟨β_t+1(∂[g_t(x_i,t+1)]_+)^TQ_i,t, y_t-x_i,t+1⟩
       ≤β_t+1⟨ Q_i,t, [g_t(y_t)]_+⟩-β_i,t+1⟨ Q_i(t), [g_t(x_i,t+1)]_+⟩

From Lemma 1 in <cit.>, we have:

    ⟨∂ f_t(x_i,t)+β_i,t+1(∂[g_t(x_i,t+1)]_+)^TQ_i,t, x_i,t+1-y_t⟩
       ≤1/α_i,t(∥ y_t-x_i,t∥^2-∥ y_t-x_i,t+1∥^2-∥ x_i,t+1-x_i,t∥^2)

Combining (<ref>), (<ref>)-(<ref>), we have:

    ℓ_t(x_i,t) - ℓ_t(y_t)   ≤G^2α_i,t/2+1/α(∥ y_t-x_i,t∥^2-∥ y_t-x_i,t+1∥^2)
    
       +β_t+1⟨ Q_i(t), [g_t(y_t)]_+⟩

Since the last term of (<ref>) is non-negative, we have:

    ∑_t=1^T (ℓ_t(x_i,t) - ℓ_t(y_t))≤∑_t=1^TG^2α_i,t/2
       +∑_t=1^T1/α_i,t(∥ y_t-x_i,t∥^2-∥ y_t-x_i,t+1∥^2)

For the first term of (<ref>), we have:

    ∑_t=1^TG^2α_i,t/2≤2^i-1G^2/2∑_t=1^T1/t^c≤2^i-1G^2/2(1-c)T^1-c

By leveraging (<ref>) and update policy (<ref>), we have:

    ∑_t=1^Tt^c/α_0 2^i-1( y_t - x_i,t^2 - y_t - x_i,t+1^2 ) 
       = 1/α_0 2^i-1∑_t=1^T( t^c y_t - x_i,t^2 - (t + 1)^c y_t+1 - x_i,t+1^2 
       +  (t + 1)^c y_t+1 - x_i,t+1^2 - t^c y_t - x_i,t+1^2  
       + t^c y_t - x_i,t+1^2 - t^c y_t - x_i,t^2 )
       ≤1/α_0 2^i-1y_1 - x_i,1^2 + 1/α_0 2^i-1∑_t=1^T( (t + 1)^c - t^c ) (d(𝕏))^2 
       + 2/α_0 2^i-1∑_t=1^T t^c d(X) y_t+1 - y_t
       ≤1/α_0 2^i-1( 1 + (T + 1)^c - 1 ) (d(X))^2 + 2T^c d(X) P_x/α_0 2^i-1
       ≤2/α_0 2^i-1 (d(X))^2 T^c ( 1 + P_x/d(X))

Let i_0=⌊1/2log_2(1+P_x/d(X))⌋+1∈[N], such that we have:

    2^i_0-1≤√(1+P_x/d(X))≤2^i_0.

Combining (<ref>)-(<ref>) yields:

    ∑_t=1^T (ℓ_t(x_i_0, t) - ℓ_t(y_t))   ≤4/α_0 (d(X))^2 T^c ( 1 + P_x/d(X))^1-κ
       + G^2 α_0/2 (1 - c) T^1-c( 1 + P_x/d(X))^κ

Applying Lemma 1 in reference <cit.> to (<ref>) and (<ref>) yields:

    ∑_t=1^Tℓ_t(x_t)-min_i∈[N]{∑_t=1^Tℓ_t(x_i,t)+1/γln1/ρ_i,1}≤γ(Gd(X))^2T/2


    ∑_t=1^T(ℓ_t(x_t)-ℓ_t(x_i_0,t))≤γ_0 (G d(X))^2 T^1 - c/2 + 1/γ_0 T^c ln1/ρ_i_0, 1

From ρ_i,1=(M+1)/[i(i+1)M], we have:

    ln1/ρ_i_0,1≤ln(i_0(i_0+1))≤2ln(i_0+1)≤2ln(⌊κlog_2(1+P_x/d(X))⌋)

From (<ref>) and that f_t is convex, we have

    f_t(x_t)-f_t(y_t)≤ℓ_t(x_t)-ℓ_t(y_t)

Combining (<ref>)-(<ref>) yields:

    Reg≤4/α_0 (d(X))^2 T^c ( 1 + P_x/d(X))^1 - κ+γ_0 (G d(X))^2 T^1 - c/2
       +G^2 α_0/2(1 - c) T^1 - c( 1 + P_x/d(X))^κ+2/γ_0 T^c ln ([κlog_2 ( 1 + P_x/d(X))])

Hence, we finish the proof.



§ RESULTS ON NORTH CHINA DATASET
















Declaration of competing interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.



Data availability

The original data can be downloaded from <cit.> and <cit.>. 



Acknowledgements

This work is partly supported by the National Science Foundation under award ECCS-2239046 and partly supported by the Columbia University SEAS Interdisciplinary Research Seed (SIRS) fund. Ning Qi graciously acknowledges special funding from the China Postdoctoral Science Foundation (No.2023TQ0169).



LDES















 Paper Title: Data-driven Modeling for Grid Edge IBRs: A Digital Twin Perspective of User-Defined Models.







Data-driven Modeling for Grid Edge IBRs: A Digital Twin Perspective of User-Defined Models 





    Kaveri Mahapatra*, Bhaskar Mitra, and Soumya Kundu
Pacific Northwest National Laboratory, United States 



{kaveri.mahapatra, bhaskar.mitra, soumya.kundu}@pnnl.gov































    
=======================================================================================================================================================================================================



§ ABSTRACT 

Recent Odessa disturbance events <cit.> have brought attention to the challenges associated with the interaction between Inverter-Based Resources (IBRs) and the transmission and distribution system. The NERC event diagnosis report has highlighted several issues, emphasizing the need for continuous performance monitoring of these IBRs by system operators. Key areas of concern include the mismatch of control and protection performance of IBRs between the original equipment manufacturer (OEM)-provided models and field measurements. The inability to replicate the realistic response can result in incorrect reliability and resilience studies. In this paper, we developed an approach on how to emulate the behavior of an IBR using measurement data obtained for system operators to utilize in real-time and long-term planning. Two experiments are conducted in the phasor domain and electromagnetic transients (EMT) domain to emulate the behavior for grid forming and grid following inverters under various operating conditions and the effectiveness of the proposed model is demonstrated in terms of accuracy and ease of utilizing user-defined models (UDMs).



Inverter-based resource (IBR) modeling, model calibration, model validation, original equipment manufacturer (OEM), ARMAX, SIPPY, GFM, GFL




§ INTRODUCTION


Precise modeling of inverter-based resources (IBRs)  and distributed generation is necessary for energy systems planning studies and real-time operations. Model accuracy is essential to prevent reliability concerns with interconnections and streamline the interconnection process. Aligning the models of grid-edge assets with high fidelity from measurements ensures accurate control and protection representation and enhances model usability in various planning studies.

 



Absence of the proprietary original equipment manufacture (OEM) information <cit.> may require transmission operators (TOPs) to calibrate the IBR generic models <cit.>. This process requires solving a nonlinear optimization problem with extensive event filtering, event selection, sensitivity analysis, parameter selection, and manual adjustments of parameters. Generic models calibrated based on single events are often inadequate <cit.> to represent a large set of event responses of IBRs. This problem requires continual model validation and calibration methods of the IBR and an understanding of the current behavior before applying control actions to the system. Preparing a 'heavily parametric' IBR model as suggested in recent FERC order 901 <cit.> could be very computationally expensive for generic models with insufficient physics and thus may be only suitable for post-event data performance assessment. It is not "realistic" for fast time scale real-time decision-making with changing operational modes. In this direction, data-driven surrogate/metamodels <cit.> or digital twin models <cit.> could serve as an excellent platform for representing the online behavior of different types of IBRs based on online data ingestion. An additional benefit arises when the surrogate model is assessed at input values where the outcome of the full model is uncertain. It not only provides an estimation of the model's result for those values but also offers an estimation of the associated error and model performance. The error estimate helps pinpoint areas within the input space where obtaining the model result would be most advantageous in minimizing error. This enables iterative refinement of the surrogate model, concentrating efforts on areas where reducing error offers the most substantial benefits.


Therefore, we propose a method for online model estimation of IBR that leverages input-output response patterns obtained from measurements of IBRs under different modes of operation. The proposed approach aims to develop measurement-driven online digital twin/ user-defined models (UDMs) capable of closely capturing a broad spectrum of IBR dynamical information and modeling the uncertainty in the prediction. The calibration problem is framed as a nonlinear system identification problem. This work utilizes phasor data and point-on-wave data accessible from IBR or OEM-supplied model terminals often without access to control or reference parameters. The development of such models relies solely on continuous recordings obtained from IBR sites thus eliminating the need of controlled tests of IBRs. These models are expected for efficient continual model validation, calibration, online decision-making, and virtualization. this in turn enhances situational awareness of IBRs connected to the grid. 

The paper is organized as follows. Section II discusses problem formulation for IBR model calibration. Section III presents the proposed approach along with evaluation criteria. Results with grid-forming inverter (GFM), and grid following inverter (GFL) with both phasor-based and electromagnetic transients (EMT) domain models are presented in Section IV. Section V concludes the paper. 



















§ PROBLEM FORMULATION
 







Modeling an IBR from measurements requires the following steps. (1) Extract the set of measurements necessary for the IBR model calibration and validation such as input and output variable measurement streams; (2) Identify a simulation environment for building the IBR model; (3) Prepare IBR model codes playback in the simulation environment to pass on the input streams and predict the output streams; (4) Calibrate and validate the UDM codes using those streams with a set of prediction error metrics; (5) Evaluate the prediction error from the UDM of IBRs. To prepare a UDM in a simulation environment aimed at emulating the IBR model response, the variables linked to the IBR model from those about the network need to be segregated. One common approach to accomplish this task is by recording the algebraic variables like voltage (V), frequency (f), as well as active and reactive power (P, Q) at the points where the IBR connects to the network.

For positive sequence modeling, in the case of grid following (GFL) IBR configurations, the calibration platform must be structured to enable the playback of recorded variables such as voltage (v) and frequency (f) obtained from the network, alongside simulating the IBR model described by parameters (p). This setup allows for the estimation of active (P) and reactive (Q) powers at the terminals of the IBR. When the estimated parameters closely approximate the true parameters of the IBR model used during data generation, the estimated powers (P̂ and Q̂) should correspond closely to the recorded values of P and Q. In the absence of system connections to the IBR, and model structure, control modes, and references, the calibration problem can be formulated as similar to that of a black-box model identification which does not require the availability of any physics information to estimate dynamic surrogate model.

Similarly, in the case of a grid-forming IBR configuration, the calibration platform must be structured to enable the playback of recorded variables such as active (P) and reactive (Q) powers at the terminals of the IBR, alongside simulating the IBR model described by parameters (p). This setup allows for the estimation of voltage (V) and frequency (f) obtained from the network. 














For phase-based modeling in EMT domain simulation of IBRs, the calibration platform must be structured to enable the playback of recorded variables such as voltage 3 phase recordings (v_abc) and estimate the currents (i_abc) injected by IBRs <cit.>. Figure <ref> shows the input and output attributes for each type of IBR modeling. Therefore, given a set of input and output recorded measurements from an IBR under various operating conditions and disturbances, the IBR model identification problem can be formulated as a system identification-based optimization problem as shown in equation <ref>. The solution gives a set of parameters for UDM which can be used for predicting IBR's transient response to a disturbance event occurring in a grid.



    min_p ∑_m = 1^N_m∑_t = t_1^N_T( ŷ_m,t - y_m,t)




    ŷ_m,t =  f( u,y_t - 1:N_T,p)








The function f is defined as a nonlinear dynamical system of equations, where p represents the coefficients/parameters of the model described by function f. Here, N_m and N_T denote the number of output and input time series signals of the plant, respectively. At a given time t, u_(m,t) indicates the m^th input signal to the plant. The variables y_(m,t) and ŷ_(m,t), respectively, denote the actual response obtained from measurements and the predicted response from the estimated model f.

The parameters p can be obtained generally by solving the nonlinear optimization problem using a stochastic gradient descent algorithm, which aims to minimize the difference between the actual and predicted responses of the UDM, as expressed by a loss function. To address this multistep prediction problem, we propose using an autoregressive (AR) structure for the model, which can serve as a UDM for IBRs.












§ PROPOSED APPROACH
 
In this work, we proposed an Auto-Regressive Moving Average with eXogenous inputs (ARMAX) model <cit.> for the continuous representation of IBR responses. ARMAX method for continual model validation and model updates is proposed in this work which is a type of recursive time series model structure used for forecasting and analysis. It can describe the relationship between a set of time series variables (the response) and one or more input series (exogenous variables), along with its own past response values and past errors as shown in Figure <ref>. An ARMAX model incorporates autoregressive (AR) and moving average (MA) components along with exogenous (X) variables to capture the dynamics of a time series and make predictions based on both the historical and incoming data and external factors. An ARMAX model can be represented as follows <cit.>.


    y_t = ∑_i = 1^N_pα _iy_t - i  + ∑_i = 1^N_qβ _iε _t - i  + ∑_i = 1,j = 1^N_q,N_mγ _ix_t,j  + e_i


where, y_t, e_t, x_t is the dependent variable, error term, and exogenous variable at time t. α _i is the autoregessive parameter,  β _i is the moving average parameter, γ _i is the exogenous weight parameter. N_p is the order of the autoregressive part, N_q is the order of the moving average part. N_m is the number of exogenous variables.



Note that ARMA models have been used in power system modal estimation <cit.> which involves extracting different inter-area mode frequencies from phasor data by estimating an Auto-regressive Moving Average (ARMA) models. Inspired by those concepts, for representing the behavior of IBRs, which also requires phasor data from a network, an ARMAX model structure is proposed in this work. For building a UDM for IBRs with continual validation using ARMAX, a set of steps are proposed as shown in flow chart in Figure <ref> and can be described as follows. Continuous model validation and calibration using IBR terminal measurements involve an integrated workflow starting with the real-time acquisition of data from plant sensors, which is then stored in a time-series database. This data is preprocessed through scaling and normalization to ensure consistency. An initial model is calibrated with historical data to establish a baseline identification model. This model continuously operates in real time, generating predictions which are compared with actual plant measurements to calculate errors. Performance metrics such as Mean Root Mean Squared Error (RMSE) is monitored continuously. When significant deviations are encountered in IBR-predicted responses, an automated recalibration of the ARMAX model is triggered. This feedback loop ensures the model's accuracy and stability, supported by real-time dashboards for monitoring and detailed logs for documentation. Periodic analysis and updates to the model can track the operational changes and can incorporate new changes in the system dynamics. It is expected that continual model validation for IBR models can maintain the identified model's integrity and reliability in real time. The following section discusses the experimental results obtained under continual model validation. 





§ RESULTS
 









This section discusses on the results obtained from testing the proposed approach on different test systems and for different types of IBRs. It is divided thus into two sections, (1) phasor domain modeling; (2) EMT domain modeling. The description of the test system along with the data collection and processing steps used for testing the proposed application is described in the following sections. 





 §.§ Phasor domain modeling of IBR







  §.§.§ Test system description


One of the test systems used to numerically evaluate the performance of the IBR modeling algorithm is a phasor domain (positive sequence) model involving multiple IBRs connected over a power transmission network. Specifically, in this work, a modified IEEE 68-bus test network is used, shown in Fig. <ref>, involving 35 IBRs connected to the network at the load buses <cit.>. Each inverter has the same capacity rating, with their total (cumulative) capacity at 20% of the total load on the network.









Two specific scenarios are synthesized to evaluate the performance IBR modeling algorithm on the phasor-domain positive sequence network: 1) all IBRs modeled as grid-following inverters (GFLs), and 2) all IBRs modeled as grid-forming inverters (GFMs). The dynamic models of the GFL and GFM inverters are adopted from the existing literature <cit.>. In particular, the GFLs are modeled as controllable current sources, with two key component blocks: a phase-locked-loop with a proportional-integral controller and a current control loop. On the other hand, the GFMs are modeled as per the CERTS droop control <cit.>, with an active power vs. frequency droop control loop and a reactive power vs. voltage droop control loop. Note protection techniques are not included as those were not part of <cit.> modeling.

The datasets for learning and validating the ARMAX models of IBRs were generated by performing phasor-domain transient dynamic simulations of the network in response to different disturbance events, such as changes in load, line trips, and generation loss. A set of 25 contingency scenarios in the network are simulated in the phasor domain and responses from all the 35 IBRs are collected in each scenario. The test system was simulated in MATLAB and V, f, P, Q measurements from the model were recorded. Each scenario data was normalized and scaled in the preprocessing steps to prepare those for ARMAX model training and testing. This work utilizes Systems Identification Package for PYthon (SIPPY) <cit.> platform to build ARMAX model for IBRs. This was used to build a simulation engine for IBR for input and output response feedback for recursive prediction of the model response. Both for GFM and GFL type of modeling, 2 input, 2 output, linear ARMAX state space model with output feedback was formulated using the ARMAX modeling structure. For identification of the system for IBRs, a recursive linear least squares solver was used to tune the parameters of the ARMAX model. The following section discusses the results obtained from phasor domain testing of the proposed model. Event scenarios with rich frequency content were selected for preparing the ARMAX model and the rest was used to test the prediction performance of the model.















  §.§.§ Phasor domain modeling results

The results are shown for both GFM and GFL type of IBR model validation results. Fig. <ref>,<ref> shows GFM modeling results with both input active and reactive power time series and output voltage and frequency response obtained under two line trip events in the test system. Both the true/original response from the test system and the estimated response from the ARMAX model for two different IBR locations are closely matching in this case. The RMS error estimated under 20 IBR models under 25 events during the model validation for GFM IBR ARMAX models are shown in Fig. <ref>. This figure shows the box plot of mean RMSE error plots for both V̂ and f̂. The distribution of RMSE errors shows lower average error rate across multiple validation responses and implies better accuracy of the ARMAX models GFM IBR phasor domain response estimation. 


Fig. <ref>,<ref> shows GFM modeling results with both input active and reactive power time series and output voltage and frequency response obtained under two line trip events in the test system. Both the true/original response from the test system and the estimated response from the ARMAX model for two different IBR locations are closely matching in this case. The RMS error estimated under 20 IBRs under 25 events during the model validation for GFM IBR ARMAX models are shown in Fig. <ref>. This figure shows the box plot of mean RMSE error plots for both V̂ and f̂. The distribution of RMSE errors shows lower average error rate across multiple validation responses and implies better accuracy of the ARMAX models GFM IBR phasor domain response estimation.

Figures <ref> and <ref> illustrate the results of GFL IBR ARMAX modeling, displaying both the input voltage and frequency time series, along with the output active and reactive power response during two line trip events in the test system. The actual responses from the test system closely match the estimated responses from the ARMAX model for two different IBR locations. Figure <ref> presents the RMS error estimated under 20 IBRs across 25 events during the validation of the identified GFM IBR models. This figure includes a box plot of the mean RMSE error plots for both P̂ and Q̂. The distribution of RMSE errors indicates a very low average error rate across multiple validation responses, suggesting better accuracy in the proposed ARMAX models' estimation of GFL IBR phasor domain responses.


































































 §.§ EMT domain modeling of IBR




  §.§.§ Test system description

For EMT domain modeling of IBR, a test system was created which consists of a distribution feeder with a set of residential customer loads and corresponding PV generation units. Residential customers are supplied through a single split-phase connection in the test system. Two houses having similar load composition have been connected across a 7.2kV/240V split-phase distribution transformer. The loads' compromise of power electronic load combinations and have been described in Table <ref> <cit.>. Detailed power electronic models for each of the described loads were designed in PSCAD/EMTdc. Voltage and current measurements were recorded at several locations as shown in Fig. <ref>. Variations in voltage to replicate fault conditions upstream from the feeder were created using a custom-designed voltage dip as discussed in <cit.>. The different voltage variations are summarized in Table <ref>. Data on the current and voltage phase point on wave (POW) data at each house, along with the current POW drawn or injected by each load at a sampling frequency of 5kHz was recorded. Additionally, the voltage at both the primary and secondary sides of the split-phase transformer was also recorded. the major load currents drawn from the household loads (HVAC), photovoltaic (PV) IBR units, and electric vehicle (EV) charging were recorded. 

The recordings were obtained under feeder voltage variations and voltage sag in a set of 4 scenarios. Each scenario data is then preprocessed with normalization and scaling to prepare the dataset for ARMAX model training and testing. One scenario data was used for training and rest 3 were used for testing. We conducted two different types of testing in this work for EMT domain calibration and validation. (1) In the presence of local measurements, the measurement data is continuously fed into the estimated model as output feedback. (2) In the absence of local measurements, the estimated output of the model is continuously fed into the UDM as output feedback. 






  §.§.§ EMT domain modeling results

For continual validation case, the past measurements are assumed to be available for doing this experiment and estimating validation error. The ARMAX model response is continuously compared with that of the original EMT simulation response at PV, EV, and HVAC locations. 
The results are presented in Figure. <ref>. The response shows a voltage sag event occurring at the distribution feeder bus and all the currents from the residential unit respond to this event. As can be seen, the estimated response of the ARMAX model accurately tracks the currents drawn by these units similar to that of its EMT simulation. 

Similarly, estimation results in the absence of incoming measurements are also presented in Figure <ref> which shows absence of measurements from time step 300, the estimated model was used to continuously predict output currents based on its past output response and thus is accurate up to  500 time steps after which the estimation error increases. This also suggests the re-estimation of ARMAX model needed when the error increases to maintain accuracy of the model and making online model updates.
The average RMSE across 4 different scenarios were calculated and presented in Table <ref> which shows the average error in time series response estimation both in case of presence and absence of measurements to be within 0.5-1.5% of the original current values. 






























































































































































































§ CONCLUSION

This paper presents a framework for UDM calibration and validation of multiple IBR models both in GFM and GFL configuration and both in phasor and EMT domain. Proposed data driven modeling of the IBR using ARMAX model is shown to accurately match the response of a generic IBR model <cit.> in both phasor domain and EMT domain simulation time steps. Proposed UDMs were validated for a large set of dynamic event simulations and the results in terms of RMSE across multiple reponses were presented. Continual validation result shows good accuracy of the proposed ARMAX models both in the presence and absence of incoming measurements. This implies the proposed model can be used to predict response of not only IBRs but also residential loads/generation currents such as EV, PV, and HVAC loads with great precision.  This type of data-driven modeling can be highly beneficial for optimizing and managing these resources. The proposed model is expected to help in resource optimization, solar forecasting, control and digital twin applications where response of IBRs need to be predicted. Future direction in this research would include testing the proposed modeling framework for field measurements obtained from IBR locations. 



§ ACKNOWLEDGMENT

This work was supported by the Sensors and Data Analytics Program of the U.S. Department of Energy Office of Electricity, under Contract No. DE-AC05-76RL01830. The authors gratefully acknowledge Dr. Rohit Jinsiwale and Dr. Aaqib Peerzada of the Pacific Northwest National Laboratory for their helpful suggestions and feedback.

IEEEtran






 Paper Title: Certifying Robustness of Learning-Based Keypoint Detection and Pose Estimation Methods.















xushengl@andrew.cmu.edu
0000-0003-4342-7234


twei2@andrew.cmu.edu
0000-0003-2505-4585


siminliu@andrew.cmu.edu
0009-0009-1495-6920


ziweiwa2@andrew.cmu.edu
0000-0001-9225-8495


  Carnegie Mellon University
  5000 Forbes Avenue
  Pittsburgh
  Pennsylvania
  USA
  15213



Luis.E.Mattei-Mendez@boeing.com

taylor.s.loper@boeing.com

joshua.neighbor@boeing.com

  The Boeing Company
  PO Box 3707
  Seattle
  Washington
  USA
  98214-2207



fhutchin@nrec.ri.cmu.edu

cliu6@andrew.cmu.edu
0000-0002-3767-5517


  Carnegie Mellon University
  5000 Forbes Avenue
  Pittsburgh
  Pennsylvania
  USA
  15213













§ ABSTRACT 
  This work addresses the certification of the local robustness of vision-based two-stage 6D object pose estimation. The two-stage method for object pose estimation achieves superior accuracy over the single-stage approach by first employing deep neural network-driven keypoint regression and then applying a Perspective-n-Point (PnP) technique.
    Despite advancements, the certification of these methods' robustness, especially in safety-critical scenarios, remains scarce. This research aims to fill this gap with a focus on their local robustness on the system level—the capacity to maintain robust estimations amidst semantic input perturbations. The core idea is to transform the certification of local robustness into a process of neural network verification for classification tasks. The challenge is to develop model, input, and output specifications that align with off-the-shelf  verification tools. To facilitate verification, we modify the keypoint detection model by substituting nonlinear operations with those more amenable to the verification processes. Instead of merely injecting random noise into images, as is common, we employ a convex hull representation of images as input specifications to more accurately depict semantic perturbations. Furthermore, by conducting a sensitivity analysis, we propagate the robustness criteria from pose estimation to keypoint accuracy, and then formulating an optimal error threshold allocation problem that allows for the setting of a maximally permissible keypoint deviation thresholds. Viewing each pixel as an individual class, these thresholds result in linear, classification-akin output specifications. Under certain conditions, we demonstrate that the main components of our certification framework are both sound and complete, and validate its effects through extensive evaluations on realistic perturbations. To our knowledge, this is the first study to certify the robustness of large-scale, keypoint-based pose estimation given images in real-world scenarios. 







<ccs2012>
   <concept>
       <concept_id>10010520.10010553</concept_id>
       <concept_desc>Computer systems organization Embedded and cyber-physical systems</concept_desc>
       <concept_significance>500</concept_significance>
       </concept>
   <concept>
       <concept_id>10010147.10010178.10010224.10010245</concept_id>
       <concept_desc>Computing methodologies Computer vision problems</concept_desc>
       <concept_significance>300</concept_significance>
       </concept>
 </ccs2012>


[500]Computer systems organization Embedded and cyber-physical systems
[300]Computing methodologies Computer vision problems





Certifying Robustness of Learning-Based Keypoint Detection and Pose Estimation Methods
    Changliu Liu
    
======================================================================================





§ INTRODUCTION

In the realm of computer vision, vision-based 6D object pose estimation, i.e., 3D rotation and 3D translation of an object with respect to the camera,  serves as a pivotal method for identifying, monitoring, and interpreting the posture and movements of objects through images <cit.>

This technology is fundamental in granting machines the ability to comprehend the physical environment, finding its utility in diverse domains such as robotics <cit.>, augmented reality <cit.>, and human-computer interaction <cit.>. The evolution of deep learning and the adoption of neural networks, particularly convolutional neural networks (CNNs), have markedly surpassed traditional techniques that depend on manually engineered features. Within the spectrum of learning-based approaches, a distinct classification exists: single-stage methods directly estimate the 6D pose from an image <cit.>. Conversely, a more widespread and accurate category of methods employs a two-stage strategy, initially regressing sparse keypoints <cit.> or dense pixels <cit.> from the image, followed by the utilization of a Perspective-n-Point (PnP)-based strategy for pose estimation through established 3D-2D point correspondences.

Despite the increasing efforts to boost the empirical robustness of these methods against challenges like occlusions, fluctuating lighting, and varied backgrounds, the focus on validating or certifying the reliability of vision-based pose estimation systems remains minimal. The absence of performance assurances for these frameworks raises concerns about their integration into safety-critical applications. In this work, our objective is to certify the robustness of learning-based keypoint detection and pose estimation approaches given input images. We focus on the aspect of local robustness, which refers to the ability to maintain consistent performance or predictions when the input data is perturbed around a given input point. The core question is determining whether pose estimation stays within an acceptable range when the input image undergoes perturbations. To the best of our knowledge, this study is the first one to certify the robustness of large-scale, keypoint-based pose estimation problem encountered in the real world.

Given the crucial role of neural networks in learning-based visual pose estimation, certifying their robustness is inherently linked to neural network verification <cit.>. This area of verification has attracted significant attention in recent years, driven by the paradox of widespread adoption of neural network solutions without adequate assurance of their reliability, primarily due to their opaque nature <cit.>. What distinguishes our problem from existing neural network (NN) verification efforts is our focus on system-level properties rather than on verifying attributes of isolated neural networks, which typically relate directly to the NN outputs <cit.>. For example, in robustness verification of classification models, the objective is to ensure the predicted class remains unchanged despite variations in input. Our research, however, targets the pose estimation framework, wherein the NN constitutes only one component of the entire system. The verification encompasses system-wide requirements, necessitating not just the evaluation of the keypoint detection model's robustness but also that of the PnP-based method.





 §.§ Overview of the Approach

To certify the robustness of the two-stage keypoint-based pose estimation framework, our main idea is to convert the local robustness certification of pose estimation into a standard neural network verification problem for classification networks. The primary challenge involves crafting three verification components—model, input, and output specifications—in a manner that is compatible with existing verification tools. A graphical overview is provided in Fig. <ref>. 


  
Model modification for verification. Considering the keypoint detection model involves complex nonlinear operations such as the softmax function, a variant of the model that is more amenable to verification is created, and an analysis is conducted to understand the properties that are maintained between the original and this modified model. 


  
Input specification through convex hulls. To account for realistic semantic perturbations in input images, we define the input space as a convex hull of possible perturbed images, which captures variations in a mathematically rigorous manner, allowing for a linear representation of input perturbations. Such a specification outperforms existing methods that simply introduce random independent noise into images.


  
Output specification via sensitivity analysis. The core of connecting system requirements with the neural network's output lies in conducting a sensitivity analysis of the downstream PnP method. By understanding how variations in detected keypoints affect the estimated pose, it's possible to translate system-level pose accuracy requirements into error thresholds for keypoint detection. By treating each pixel as a separate class, these thresholds are then used to define classification-like linear output specifications.



 §.§ Contributions

Our contributions can be summarized as follows:

    
  * We propose a local robustness certification framework for the learning-based keypoint detection and pose estimation pipeline;
    
  * We analyse the soundness and completeness properties of this certification framework;
    
  * We demonstrate the method's efficacy through validation on a real-world scale keypoint-based pose estimation problem.























§ RELATED WORK




 §.§ Formal Verification of Neural Networks

The objective of verifying neural networks involves ensuring they meet certain standards of safety, security, accuracy, or robustness. This essentially means determining the truth of a specific claim about the outputs of a network based on its inputs. In recent years, there has been a significant influx of research in this area. For comprehensive insights into neural network verification, one can refer to <cit.>. Verification techniques are generally divided into three main groups: reachability-based approaches, which perform a layer-by-layer analysis to assess network output range <cit.>; optimization methods, which seek to disprove the assertion <cit.>; and search-based strategies which combine with reachability analysis or optimization to identify instances that contradict the assertion <cit.>. In 2020, VNN-COMP <cit.> launched as a competition to evaluate the capabilities of advanced verification tools spanning a variety of tasks, including collision detection, image classification, dataset indexing, and image generation. However, these methods treat deep neural networks in isolation, concentrating on analyzing the input-output relationship.

Concurrently, there is research focused on the system-level safety of cyber-physical systems (CPS) incorporating neural network components, particularly within the system and controls domain. They broadly fall into two categories. The first category <cit.> focuses on ensuring the correctness of neural network-based controllers, taking their input from the structured outcomes of the state estimation module, regardless of the state estimation module is based on perception or not. Neural network controllers of this type generally consist of several fully connected layers, making them relatively straightforward to verify. The second category focuses on validating the closed-loop performance of vision-based dynamic systems that incorporate learning-based components. Among these, studies such as <cit.> examine LiDARs as the perception module, processed by multi-linear perceptrons (MLPs) with a few hidden layers. Other approaches, primarily applied to runway landing and lane tracking, deal with high-dimensional inputs from camera images, employing methods like approximate abstraction of the perception model <cit.>, contract synthesis <cit.>, simplified networks within the perception model <cit.>, or a domain-specific model of the image formation process <cit.>. 
Nevertheless, studies directly dealing with high-dimensional inputs from camera images are still limited due to the images' high dimensionality and unstructured data nature, in contrast to structured robot states such as position and velocity.





 §.§ Certification of Keypoint Detection and Pose Estimation Methods

The investigation of certification methods for pose estimation is relatively limited. <cit.> introduced a certifiable approach to keypoint-based pose estimation from point clouds by correcting keypoints identified by the model, ensuring the correctness guarantee of the pose estimation. <cit.> expanded on this by integrating the correction concept with ensemble self-training. In a similar vein, By propagating the uncertainty in the keypoints to the object pose, <cit.> created a keypoint-based pose estimator for point clouds that is provably correct and is characterized by definitive worst-case error bounds. The above work focuses on point clouds as opposed to images. <cit.> applied an advanced deep learning technique for uncertainty quantification to assess the uncertainty (i.e., the predicted distribution of a pose) in multi-stage 6D object pose estimation methods. In contrast, <cit.> aimed to design a neural network that processes camera images to directly predict the aircraft's position relative to the runway with certifiable error bounds. Of all these studies, <cit.> is the most similar to our work, which focuses on the verification of keypoint detection, excluding the examination of the PnP method for system-wide assurances. Their approach verifies the neural network in isolation and is limited to very slight perturbations, failing to encompass realistic semantic variations.




§ BACKGROUND

In this work, we represent scalars and scalar functions by italicized lowercase letters (x), vectors and vector functions by upright bold lowercase letters (), matrices and matrix functions by upright bold uppercase letters (), and sets and set functions with calligraphic uppercase letters ().



 §.§ Keypoint-based Pose Estimation

The keypoint-based approach consists of two steps to estimate the 6D pose from a 2D model image. First, a neural network is tasked with predicting the 2D locations of keypoints, whose 3D locations are predefined within the object model. A common strategy involves the use of heatmap regression, wherein ground-truth heatmaps are created by placing 2D Gaussian kernels atop each keypoint. The heatmap pixel values are interpreted as the likelihood of each pixel being a keypoint. These heatmaps are then used to guide the training through an ℓ_2 loss. The detection network can be divided into two parts. A backbone network, denoted by b, inputs a 2D image to produce unnormalized heatmaps, which is first followed by a softmax layer that transforms unnormalized heatmaps into normalized ones, and then by argmax operations, or another layer of differentiable spatial to numerical transformation (DSNT) <cit.>  for keypoint extraction. We refer to the part after the softmax (including) as the head network. The entire network is represented by

=  () = h∘b(), 

where ∈^H× W× C represents a 2D RGB image with dimensions being H× W× C, and ∈^K× 2 denotes the 2D coordinates of K keypoints. Here, ∘ denotes function composition. To enhance accuracy and robustness, it's often essential to preprocess the input image  before it is passed to the network, such as resizing and color normalization. Denote this preprocessing step by _0, leading to the equation =  () = h∘b∘_0().  In what follows, we omit the preprocessing step unless it is critical to consider it.



The second step employs the Perspective-n-Point (PnP) algorithm, which executes a nonlinear least squares (NLS) optimization to estimate the 6D pose from established 3D-to-2D correspondences. An illustration of the pose estimation  for an airplane is presented in Fig. <ref>. Let ∈^3×3 denote the camera intrinsic parameter matrix, _k ∈^3 denote the 3D coordinate of keypoint k, where k = 1, …, K, and _k ∈^2 denote the corresponding 2D coordinate. These 3D-2D correspondences are formed through the following perspective projection model:

    λ_k [ _k;  1 ]
       =  (_k + ),

where ∈^3×3 and ∈^3 represent the rotation matrix and translation vector, respectively, that establish the transformation between the object and camera coordinate systems, with λ_k representing the scaling factor. The objective of the PnP method is to approximate this transformation, represented by  and , by minimizing the ℓ_2 norm of the reconstruction error:

    ⟨, ⟩  =   ,       ∑_k=1^K (_k + ) -   λ_k [ _k;  1 ]^2_2.


Let the notation ⟨, ⟩ = (, ) represent the PnP procedure, where ∈^K× 3 denotes the 3D coordinates of keypoints. Note that the keypoint-based pose estimation framework described above is the basic version. Numerous adaptations have been developed to increase accuracy and robustness <cit.>, particularly in challenging conditions such as occlusions, varying viewpoints, and different lighting scenarios. This paper concentrates on validating the robustness of this foundational pipeline, marking a crucial step towards certifying the effectiveness of more intricate keypoint-based pose estimation techniques. Hereafter, we use Ψ to represent this pipeline, which includes keypoint detection followed by the application of a PnP method, that is, ⟨, ⟩ = Ψ() = (, ()).








 §.§ Verification of Neural Networks

Consider a multi-layer neural network representing a function , which takes an input ∈_⊆^d_0 and produces an output ∈_⊆^d_n, where d_0 is the input dimension, and d_n is the output dimension. Any non-vector inputs or outputs are restructured into vector form. The verification process entails assessing the validity of the following input-output relationships defined by the function :
∈⇒ = () ∈,
where sets, ⊆_ and ⊆_, are referred to as input and output constraints, respectively. 

In the context of confirming the robustness of a classification network, the goal is to ascertain that all samples within a proximal vicinity of a specified input x_0 receive an identical classification label. Assuming the target label is i^∗∈{1, …, d_n}, the specification for verification is that y_i^∗ > y_j for every j not equal to i^∗. The constraints on inputs and outputs are established accordingly:
= {| - _0_p ≤ϵ}, 
 = {| y_i^∗ > y_j,  ∀ j ≠ i^∗},
where ϵ represents the maximum permissible deviation in the input space. The metric used to quantify disturbance can be any ℓ_p norm.

Neural network verification algorithms can generally be categorized into three main types: reachability analysis, optimization, and search. NN verification essentially seeks to transform the nonlinear model checking problem into piece-wise linear satisfiability problems, and it can be applied to various nonlinearities, including ReLU and, more recently, softmax <cit.>. Two pivotal attributes—soundness and completeness—are of critical importance. A verification algorithm is sound if it only confirms the validity of a property when the property is indeed valid. It is complete if it consistently recognizes and asserts the existence of a property whenever it is actually present. There is a trade-off between computational complexity and conservativeness (or in-completeness).






§ PROBLEM FORMULATION


The most common type of input specification involves limiting the ℓ_p-norm of the variation to a threshold, that is, - _0 _p≤ϵ. However, ℓ_p perturbation,  often characterized by a small value ϵ,  is not a correct mathematical description of more realistic perturbations, as the independent nature of pixel-wise perturbations falls short in creating perturbations that reflect semantic correlations between pixels, such as large variations in lighting, weather conditions, and the effects of camera motion blur. Another approach for input perturbations is to directly add a generative model that perturbs the input image before the original neural network, such as <cit.>, and then verify them together. However, the verification result can be biased by the generative model used and it is not user friendly due to the difficulty in controlling the changes made to the image. To address these shortcomings, we adopt a strategy based on the convex hull, which involves combining a seed image with a collection of perturbed images. These perturbed images can be created through different methods including simulators and learning-based generative models. Also, the convex hull specification can directly enable users to specify the perturbed images, which makes the perturbation specifications more user friendly.


Given a seed image _0 and a set of n perturbed images {_1, …, _n}, the convex hull constituted by these images is defined by the set of all their possible convex combinations. Mathematically,

    = { |  = ∑_i = 0^n α_i _i,   s.t. α_i ≥ 0, ∑_i=0^n α_i  = 1.}.


Images within the convex hull ∈ result from varying degrees of continuous blending among the provided images.  Through the convex combination of perturbation instances, we can model environmental and sensor-related perturbations, including changes in brightness, contrast, weather conditions, motion blur, and dust on lens. Convex hull perfectly captures the entire range of brightness or contrast shifts, and does a reasonable approximation of color shifts, provided they are small. However, this approach does not capture translational perturbations of the object or perturbations related to camera movements, such as changes in the viewpoint. Preliminary methods exist that characterize pixel-wise perturbations caused by camera motion <cit.>, but these methods cover only a very limited range. Investigating robustness against such perturbations will be addressed in future research.  A collection of perturbed images and sampled images from the convex hull can be found in Fig. <ref> in Section <ref>. 

Given two rotation matrices  and , we define the function _r (, ) = |() - () |, where (·) represents the function that converts rotations to Euler angles, and |·| denotes the element-wise absolute value. Thus, _r measures the rotational difference along each axis. Similarly, define _t (, ) =  | - |, which calculates the translational difference per axis.





 Given a convex hull representation  consisting of a seed image _0 and n perturbed images, and a keypoint-based pose estimation framework Ψ.
 
 
 
 
 The problem is to certify whether the pose estimation framework Ψ is robust to any image within the set . Mathematically,

    [ _r (,  );  _t (, ) ]≤[ ϵ_r; ϵ_t ]  s.t. ⟨, ⟩ = Ψ(),  ∀ ∈,

where ⟨, ⟩ =  (, ) represents the pose estimated given the ground-truth 3D keypoint coordinates  and ground-truth 2D keypoint coordinates   for _0, and ϵ_r ∈^3 and ϵ_t ∈^3 denote the error thresholds specified by the user for rotation and translation. We refer to  and  as the nominal transform, and  and  as the perturbed transform. In simpler terms to describe Eq. (<ref>), the variation between the nominal transform and the perturbed transform is kept within these predefined thresholds.



If a pre-processing step, such as resizing or color normalization, is applied to an image before it is input into the keypoint detection model, then the convex hull should be constructed using images that have also undergone these pre-processing steps. Consequently, any image within the convex hull is represented as = ∑_i=0^n α_i _0(_i), where _0 denotes the pre-processing function. 




 The vector of error thresholds ϵ_r and ϵ_t can specify requirements from system designers in an interpretable manner. For example, such requirement can be setting the error thresholds for translation to be within 1 meter. The difficulty presented by Problem <ref> is that, the error thresholds ϵ are defined with respect to pose estimation rather than the direct outputs of neural networks—keypoints—and there are nonlinear mappings between keypoints and pose estimation. 
 
To address Problem <ref>, the core of our approach is to  transform the local robustness certification problem into a standard neural network verification problem for classification networks. This involves adapting three verification components—model, input, and output specifications—to be compatible with existing verification tools. As detailed in Section <ref>, we have modified the keypoint detection model to handle complex operations like the softmax function, simplifying the verification process. The input space is the a convex hull of possible perturbed images, which captures semantic variations  compared to traditional methods that introduce random noise. Section <ref> outlines how the output specification is established through a sensitivity analysis of the PnP method, detailed further in Section <ref>. This analysis helps translate system-level pose accuracy requirements into error thresholds for keypoint detection. Finally, Section <ref> is dedicated to analyzing the soundness and completeness of our proposed verification framework.





§ FORMULATION OF VERIFICATION OF THE NEURAL NETWORK


To convert Problem <ref> into an equivalent problem that can be verified using existing NN verification tools, we introduce approaches to modify the NN model and specify the output constraints.



 §.§ Verification-Friendly Keypoint Detection Model


As introduced in Section <ref>, a softmax layer follows the backbone network to generate probabilistic heatmaps. This step introduces verification challenges due to the highly nonlinear exponential function in softmax. To address this, we propose to maintain the backbone network but replace the head network with an alternative one. A critical insight is that during the inference phase, the unnormalized heatmaps output by the backbone network b already provide significant clues about the locations of keypoints. The subsequent operations within the head network h, such as softmax and DSNT, essentially serve to aggregate this information across the entire heatmap. This aggregation allows for the generation of predictions in an average manner. In essence, the peaks within the heatmaps play a pivotal role in these predictions, and their locations remain unchanged between unnormalized and normalized heatmaps. Consequently, by focusing on the characteristics of these peaks, we can bypass some of the complexities introduced by the head network's nonlinear operations while still capturing the essential information required for accurate keypoint detection. 


By appending an average pooling layer followed by an argmax layer as the new head network, referred to as h', to the backbone network, we create a proxy model, denoted by proxy = h' ∘b. We refer to the original model as the target model. The introduction of this proxy model simplifies the transformation of the verification problem into one akin to classification, where each pixel is treated as a unique category, as illustrated in Fig. <ref>. The average pooling layer offers two advantages. Firstly, its downsampling effect significantly reduces the number of categories (pixels), thereby simplifying the complexity of verification. Secondly, by aggregating local features within each pooling region, it ensures a more accurate representation than applying argmax directly. 







The selection of pooling parameters should be guided by the error threshold on pose estimation ϵ_r and ϵ_t, as elaborated in Section <ref>. This leads to a scenario where the pooling parameters assigned to each heatmap, corresponding to individual keypoints, vary. This variation arises because keypoints have differing levels of influence on the accuracy of pose estimation. Consequently, keypoints with lesser influence on pose estimation are allocated a larger permissible error range, whereas those that play a critical role in pose estimation are subjected to stricter error ranges. It is worth noting that, with certain assumptions, the target model and proxy model are equivalent in terms of verification. The analysis on how assurances regarding the proxy model's performance can be extended to the target model is conducted in Section <ref>.


































 
 


















 








































 
 















 



 §.§ Output Specification: Polytope Representation

For the coordinate _k of the k-th keypoint, where _k represents its 2D coordinate, let _k denote its averaged ground-truth coordinate following the application of the average pooling and argmax layers. To ensure the classification result is consistent across perturbations, the output specification necessitates that the value at v̅_k exceeds the values at all other entries. This requirement can be encapsulated by a half-space-based polytope, represented as _k _k ≤_k with



    _k = 
    [  1  0  … -1  …  0;  0  1  … -1  …  0;  ⋮  ⋮  ⋱  ⋮  ⋮  ⋮;  0  0  … -1  …  0;  ⋮  ⋮  ⋮  ⋮  ⋱  ⋮;  0  0  … -1  …  1;    ]
    ,   _k = 
    [    y_1;    y_2;      ⋮; y_v̅_k;      ⋮;    y_n ]
    ,   _k = 
    [ 0; 0; ⋮; M; ⋮; 0 ],

where the v̅_k-th column of _k is populated with (-1)'s, and all diagonal elements set to 1 with the exception of the element at (v̅_k, v̅_k), the components of _k are set to zero except for the v̅_k-th element, which is assigned a significantly large integer M. Consequently, the specification for all keypoints is denoted by ≤ with

    𝐀 = 
    [ 𝐀_1   0   0   …   0;   0 𝐀_2   0   …   0;   0   0 𝐀_3   …   0;   ⋮   ⋮   ⋮   ⋱   ⋮;   0   0   0   … 𝐀_K ]
    ,    = 
    [ _1; _2; _3;  ⋮; _K ]
    ,    = 
    [ _1; _2; _3;  ⋮; _K ].

The construction of Eq. (<ref>) presupposes verifying all keypoints. However, should there be a keypoint that does not require verification, potentially due to being an outlier, the matrices associated with it are then adjusted accordingly: _k = 𝐈 and _k = [M, …, M]^T.





























§ ERROR PROPAGATION AND DETERMINATION OF POOLING PARAMETERS

To calculate the parameters for average pooling, the initial step involves mapping the error thresholds ϵ_r and ϵ_t from the pose estimation onto the keypoints' error thresholds that may be correlated to each other, which is accomplished by employing sensitivity analysis methods.

Subsequently, we formulate an optimal error threshold allocation problem, aiming to assign independent error thresholds to each keypoint. This step is crucial for determining the allowable errors for each keypoint individually, ensuring that the overall pose estimation adheres to predefined error limits. The idea is graphically depicted in Fig. <ref>.










 §.§ Error Propagation via Sensitivity Analysis

Sensitivity analysis for nonlinear optimization explores the impact of first-order changes in the optimization parameters on the locally optimal solution <cit.>. It involves analyzing an unconstrained parameterized nonlinear optimization problem expressed as min_ F(; ), with  representing the parameter vector. Here, let z represent the optimal objective value given the parameter vector , i.e.,
z () = min_ F(; ).
Sensitivity analysis <cit.> connects the first-order derivatives ∂ and δ z with ∂:

    [ -F_; -F_ ]∂  
        = [ F_ -1; F_  0 ][   ∂; ∂ z ].

In the context of pose estimation, the keypoint coordinates  are considered as parameters and the pose to be estimated represents the decision variables, the notation for partial derivatives ∂ is substituted with the notation for deivation δ. Consequently, Eq. (<ref>) is transformed into

    [ -G_; -G_ ]δ
        = 
        [ G_ -1; G_  0 ][   δ; δ z ],

where ∈^6 = [, ] represents the 6D pose vector, with the 3D rotation matrix  expressed in the axis-angle or Euler angles representations form , and G corresponds to the NLS optimization objective outlined in Eq. (<ref>). Note that δ∈^2K. The following notation is introduced

    _ :=  [ -G_; -G_ ]∈^7× 2K,  _ :=  [ G_ -1; G_  0 ]∈^7×7.


If _ is invertible, then it follows that
_^-1_δ =  [   δ, δ z]^T.
    
    
    
By extracting the rows in the matrix _^-1_ that correspond to δ and naming the resulting matrix _, we obtain the linear mapping 

_δ = δ.

This leads to the establishment of a linear relationship between  2D keypoints errors and 6D pose errors. Upon specification of the keypoints  and the transform ξ, the matrix _ becomes determined. 

Let Ξ represent the set of tolerable errors, indicating the range within which deviations of the perturbed transform from the nominal transform are acceptable, meaning δ∈Ξ. Let δ denote the derived tolerable keypoint  errors that has the following property:

 δ =  {δ | _δ   =  δ∈Ξ}.

Note that δ is an approximation of actual tolerable keypoint errors that lead to tolerable pose errors δξ. As the pose error bounds decrease, the accuracy of the PnP linearization improves, narrowing this disparity. In what follows, we assume that Ξ is defined as a polytope represented by linear inequalities, specifically Ξ= {δ | _ξδ≤_}. This inequality can be established based on a user-defined thresholds ϵ_r and ϵ_t as outlined in Problem <ref>. By the linear mapping, the set δ can also be characterized as a polytope:
    δ = {δ | _ξ_δ≤_} = {δ | _δ≤_},
where _ =  __ and _ = _.  While larger set  δ is desirable, it is an approximation of actual tolerable errors due to the linearization of sensitivity analysis. To mitigate the risk of over-approximation, we  introduce a scaling factor to reduce the size of  set δ in the next section.



 §.§ Optimal Error Threshold Allocation

The tolerable errors of keypoints are interrelated within the derived polytope set δ. In this section, we allocate the tolerable errors across each keypoint to ensure their tolerances are independent, as current verification tools lack the capability to verify dependencies between  keypoints. The allocated error threshold  aims to be maximized, provided that the pose deviation remains tolerable.

For a given error vector δ, we can uniquely define an axis-aligned, axis-symmetric hyper-rectangle, denoted as (δ) ⊂^2K, centered at the origin with each element δ_i representing the half-length of its sides. Axis symmetry indicates that the error thresholds are identical in opposite directions. We establish the following problem for optimal error threshold allocation:

    max_δ∈_+^2K      w_1 ∏_k=1^2Kδ_k + w_2 Δ
    s.t.     (δ)  ⊆δ, 
         δ_k≥Δ,     k = 1, …, 2K.


Constraint (<ref>) ensures that the hyper-rectangle is encompassed within the set of tolerable errors δ, while constraint (<ref>) sets a minimum threshold for the side lengths. The goal defined in objective (<ref>) is to optimize a weighted sum of two objectives: the first seeks to identify the largest possible axis-aligned, axis-symmetric hyper-rectangle within a convex polytope, and the second strives to extend the minimal side length as much as feasible.

To address the optimal error threshold allocation problem, we translate constraint (<ref>) into a series of linear inequalities, drawing from the method in <cit.> for identifying the largest axis-aligned hyper-rectangle inscribed within a convex polytope. Considering a hyper-rectangle = {∈^d |≤≤} and a convex polytope = {|≤}, <cit.> demonstrates that ⊆ if and only if

    ∑_j=1^d (_ij^+ _j - _ij^- _j) ≤_i,    i = 1, …, n.

where _ij^+ = max{_ij,0} and _ij^- = max{-_ij,0}. Given the axis symmetry in our scenario, it follows that _j = -_j. Substituting _j with δ_j,  with _, and  with _, we can reformulate Ineq. (<ref>) as

    ∑_j=1^2Kδ_j (_ ij^+  + _ ij^-) ≤_ i,    i = 1, …, 6.

Given that |_ ij| = _ ij^+ + _ ij^- where |·| returns element-wise absolute values, it follows that

    ∑_j=1^2Kδ_j |_ ij|≤_ i,    i = 1, …, 6.

In matrix notation, Ineq. (<ref>) is expressed as
|_| δ≤_.
Ultimately, the problem of optimal error threshold allocation, encompassing objectives (<ref>) through constraints (<ref>), is equivalent to

    max_δ∈_+^2K      w_1 ∏_k=1^2Kδ_k + w_2 Δ
    s.t.     κ  |_| δ≤_, 
         δ_k≥Δ,    k = 1, …, 2K,


where _+ represents the set of non-negative integers, and κ∈_+ is the scaling factor that controls the actual size of polytope. The effect of the scaling factor κ on the allocation is discussed in Section <ref>. The solution, denoted by δ^*, returns the maximum tolerable errors for all keypoints while ensuring that the pose errors remains within the error thresholds. This solution will determine the parameters for average pooling in Section <ref>.






































































































































 §.§ Determination of Average Pooling Parameters



A widely accepted practice to train the keypoint detection model is to regress the heatmaps centered on the ground truth keypoint. Building on this approach, we propose the following assumption and support it with evaluation results detailed in Section <ref>.


   The softmax layer in the target model _target produces a distribution with a peak at its center and exhibits symmetry along axes that intersect this peak.


Upon determining the independent error threshold for each keypoint, we proceed to define the parameters for average pooling in the proxy model _proxy. Focusing on a particular keypoint with coordinates = (_h, _v), and considering the error thresholds δ^*_h and δ^*_v allocated for horizontal and vertical directions respectively, we define the set 

    _δ = { (h, v) | | h - _h| ≤δ^*_h  | v - _v| ≤δ^*_v }.

Only pixels within _δ are considered acceptable potential predictions for the keypoint. The average pooling parameters for this keypoint are selected to ensure every pixel in _δ is contained by the same pooling patch. This guarantees that all these pixels are pooled into the same after-pooling pixel as the keypoint, thereby belonging to the same class.

We begin by examining how predictions are generated in the proxy model proxy. Based on Assumption <ref>, the unnormalized heatmap returned by the backbone network _b exhibits symmetry along axes passing through this peak, as softmax operation will not change the overall shape of the unnormalized heatmap. To infer the keypoint prediction, we concentrate on the average pooling patch that predominantly overlaps with the vicinity of the peak. Since the average pooling patches also exhibit axial symmetry, patches closer to the peak yield larger averaged pixel values. Consequently, the pooling patch with its center closest to the peak results in the predicted keypoint. This insight enables us to transform the determination of the location of the maximal pixel value after pooling into identifying the closest pooling patch to the peak of the unnormalized heatmap under Assumption <ref>. Next, we compute the average pooling parameters for each keypoint to ensure that: (i) the ground-truth keypoint aligns with the center of an average pooling patch, and (ii) all pixels in proximity to this pooling patch fall within the allocated error thresholds from the ground truth keypoint, making them suitable to act as the peak of the unnormalized heatmap. Let = (s_h, s_v) denote the stride, = (k_h, k_v) the kernel size, and = (p_h, p_v) the patch size.





  
Determination of stride and kernel parameters. 
The initial step involves determining the stride and kernel parameters, denoted as s_h and k_h, respectively, as in Fig. <ref>. We focus on the horizontal axis (similar logic applies to the independent vertical axis). Let  represent the pooling patch centered at a keypoint, with _-1 and _+1 denoting the adjacent patches to the left and right, respectively. Additionally, let b_-1 and b_+1 denote the perpendicular bisectors between these patches. The distance between these two perpendicular bisectors corresponds to the horizontal stride. To position the ground-truth keypoint at the center, an odd stride is required. As long as the peak of the unnormalized heatmap lies between b_-1 and b_+1, the pooling patch  is the closest to the peak, resulting in the predicted keypoint. Let δ^*_h denote the allocated error threshold. To ensure that peaks falling between b_-1 and b_+1 remain permissible with respect to the error threshold, the threshold should be no less than half the stride distance from the ground-truth keypoint on either side, i.e.,
δ^*_h ≥1/2 (s_h - 1),
which leads to the inequality s_h ≤ 2δ^*_h + 1. Choosing s_h =  2 δ^*_h + 1 minimizes the post-pooling size. It's worth noting that there are no constraints on the kernel parameter. For simplicity, we set the kernel k_h to be equal to the stride s_h.



















  
Determination of the padding parameter.
The next step involves determining the padding parameter to center a pooling patch at the ground-truth keypoint. Padding can be added from the left (horizontal) or from the top (vertical) to achieve this. Considering the horizontal axis where the kernel is equal to the stride, the idea is to identify the closest pooling patch to the keypoint from its right side and then shift it left by the distance between the keypoint and its center, as illustrated in Fig. <ref> and <ref>. Assuming k_h = s_h, let r_h be the number of pixels the keypoint is from the left side of the closest pooling patch, calculated as r_h ≡_h k_h. (i) If r_h ≤k_h + 1/2, the closest patch center to the right of the keypoint, belongs to the pooling patch containing the keypoint. (ii) Otherwise, the closest patch center to the right comes from the pooling patch immediately to the right of the one containing the keypoint. The horizontal padding parameter p_h is determined as k_h+1/2 - r_h if r_h ≤k_h+1/2, otherwise, as 3k_h+1/2 - r_h.









Note that the shift is not unique because multiples of the stride can be added to p_h.

 
 
 
 
 
 
 
 
 
 
 
 




§ THEORETICAL ANALYSIS

With the method we introduced (to reformulate the problem into a verifiable form using off-the-shelf verification tools), it is important to understand whether false analysis results will be introduced. To answer that question, we perform the following analysis.



 §.§ Properties of the Proxy Model

We examine two target models. The models ending with a DSNT layer and an argmax layer are represented by dsnt and argm, respectively. The term target is used to represent either target model. The notation ^k is employed to identify the k-th predicted keypoint. Recall that _k denote the averaged ground-truth coordinate of the k-th keypoint after the application of average pooing and argmax layers. Before providing theoretical results, we establish formal definitions for both the soundness and completeness related to the proxy model, which are visually represented in Fig. <ref>.



    A proxy model is deemed sound if, for cases where the proxy model's predicted keypoint aligns with the averaged ground-truth keypoint _k, the deviation of the target model's prediction from the ground truth _k does not exceed the error threshold δ_k^*. Specifically, 
    
    proxy^k() = _k ⇒|target^k() - _k |≤δ_k^*,   for  k = 1, …, K.

     Here, |·| indicates the computation of element-wise absolute differences.


    A proxy model is deemed complete if the difference between the target model's prediction and the ground truth  _k is confined within the error threshold δ^*_k, then the predicted keypoint by the proxy model matches the averaged ground truth _k. That is, 
    
    |target^k() - _k |≤δ_k^* ⇒proxy^k() = _k,   for  k = 1,…, K.




If Assumption <ref> holds, the proxy model is sound, that is,

    proxy^k() = _k    ⇒|dsnt^k() - |≤δ_k^*,     for   k = 1, …, K. 
    proxy^k() = _k    ⇒|argm^k() - |≤δ_k^*,     for   k = 1, …, K.



To prove Theorem <ref>, we first prove the equivalence of two target models.
    
      If Assumption <ref> holds, the two target models produce the same predictions, i.e.,

    dsnt^k() =  argm^k(),      for    k = 1, …, K.

    
    




 Before delving into the proof, we first introduce how DSNT extracts a keypoint from a normalized heatmap. Let  represent a normalized heatmap of dimensions m × n, which is generated by a softmax layer. Define  as an m × n matrix where each entry _ij is calculated by _ij = 2j-(n+1)/n. Consequently, each column of  contains identical values. These values fall within the range of (-1, 1), with the center value at 0. To determine the column coordinate _h of the keypoint, DSNT computes the sum of the element-wise product between  and , expressed as _h = ∑_i, j_ij_ij. The row coordinate is obtained in a similar way.

Without loss of generality, due to independence of horizontal and vertical axes, we will show for the horizontal direction. Considering a symmetric heatmap with dimensions m × n, we initially position the center of the distribution at the m+1/2-th row and the n+1/2-th column, effectively placing the distribution's center at the heatmap's midpoint. Upon applying the DSNT process, the resulting keypoint coordinate is (0, 0). This outcome is attributed to the distribution's symmetry along both the horizontal and vertical axes that intersect at its center, with the DSNT operation cancelling out symmetric values relative to the origin. Subsequently, we consider a horizontal displacement of the distribution by c columns, where a positive c indicates a shift to the right. The column coordinate _h, as determined by DSNT following the horizontal shift, is then













    ∑_i, j_ij_ij 1= ∑_i, j'_ij'(_ij' +  2c/n) 2= ∑_i, j'_ij'_ij' + ∑_i, j'_ij'2c/n 3=   0 + 2c/n∑_i, j'_ij' 4= 2c/n


In 1, the pixel located at (i, j') is the corresponding pixel at (i, j) prior to the horizontal shift, with both pixels having identical values, denoted as _ij = _ij'. The value 2c/n represents the  difference across every c columns within the  matrix .
For 3, the expression ∑_i, j'_ij'_ij' equals zero, reflecting the distribution's initial centering at the heatmap's midpoint. In 4, the probability over distribution sums up to 1. The calculation of the DSNT column coordinate as 2c/n∈ (-1, 1) aligns it with the heatmap's (n+1/2+c)-th column. On the other hand, the argmax operation in argm^k returns the  distribution's center, located at the (n+1/2+c)-th column following the shift. Consequently, we establish that dsnt^k() = argm^k(), thereby completing the proof.


According to Proposition <ref>, it is sufficient to validate Eq. (<ref>) that connects proxy with argm to prove Theorem <ref>. The distinction between the proxy and argm models lies solely in their method of handling the unnormalized heatmap, specifically whether they use a softmax or an average pooling layer as depicted in Fig. <ref>. With this, we are poised to validate Theorem <ref>, a conclusion that naturally follows from the way we identify average pooling parameters.

Considering the k-th keypoint _k, if proxy^k() = _k, it implies that average pooling parameters ensure that the closest pooling patch __k to the ground-truth keypoint _k is exactly centered at _k.

 The selection of stride parameters ensures that the heatmap's peak is within a δ^*_k distance from the center of __k, which is also the ground-truth coordinate _k. Given that the softmax layer preserves the peak's position, the prediction argm^k precisely returns the peak's location, leading to the conclusion that argm^k() is within a δ^*_k distance from _k, i.e.,
 |argm^k() - |≤δ_k^*.






If Assumption <ref> holds, the proxy model is complete, that is,

    |dsnt^k() - |≤δ_k^*    ⇒proxy^k() = _k,    for   k = 1, …, K. 
    |argm^k() - |≤δ_k^*     ⇒proxy^k() = _k,    for   k = 1, …, K.
 


Based on Proposition <ref>, we focus exclusively on the target model argm. The selection of pooling parameters guarantees that every pixel within the δ^*_k proximity of the ground-truth keypoint—eligible to be recognized as the predicted keypoint—shares the same closest pooling patch. This closest pooling patch is identical to that of the ground-truth keypoint. As a result, these pixels yield an averaged pixel that is the same as that of the ground-truth keypoints.



 


 §.§ Properties of the Optimal Error Threshold Allocation

In the previous section, we examined the properties of the proxy model after determining the error threshold for keypoints. This section focuses on the properties of the optimal error threshold allocation. We introduce the concepts of probabilistic soundness and probabilistic completeness to measure the relationship between tolerable errors in pose and keypoints, as depicted in Fig. <ref>. 

Considering the set of acceptable pose errors Ξ, let δ_Ξ represent the corresponding set of acceptable keypoint errors. Note that δ_Ξ might not be polytopic, due to the non-linear nature of the PnP method, even though Ξ is polytopic. Let μ(·) denote the measure of a set.
    
    Probabilistic soundness is defined by the ratio μ(δ_Ξ∩)/μ(), which is the fraction of keypoint errors in  that result in tolerable pose errors within Ξ.
    
    Likewise, considering the set of tolerable errors on keypoints , let Ξ_ represent the corresponding set of tolerable errors on poses.
     
     Probabilistic completeness is defined as the ratio μ(Ξ_∩Ξ)/μ(Ξ), which represents the fraction of permissible pose errors in Ξ that result in errors on keypoints within .
    
Deriving an exact or lower bound for these two metrics is difficult, due to the challenges in making definitive conclusions using linearization-based sensitivity analysis, as the accuracy of local linearizations varies with instances, such as varying poses. In Section <ref>, we perform statistical tests to approximate these two values, offering insights into the practicality of our framework.
 
    













































































































































 







§ EVALUATION RESULTS





 §.§ Verified Keypoint Detection Model and Perturbations



  §.§.§ Verified keypoint detection model

  The verified keypoint detection model, illustrated in Fig. <ref>, is modified from the proxy model proxy. This verified model includes three components:


  
1. Backbone model  The dataset contains 7,320 images, each with dimensions of 1920 × 1200. 


    
  * CNN. The architecture includes 5 convolutional layers and an equal number of deconvolutional layers, designed to handle inputs of size 64×64×3. The model comprises 39 layers and contains around 6.57× 10^5 trainable parameters.
    
  * ResNet-18. This model includes 8 residual blocks  and  processes inputs of size 256×256×3. It is composed of 84 layers and possesses approximately 1.2× 10^7 trainable parameters, which is approximately an order of magnitude larger than existing works <cit.>.








 
 





  
2. Average pooling Consider that the parameters for average pooling are tailored based on optimally allocated error thresholds that may vary across different keypoints, thereby necessitating distinct pooling parameters for each. To address this issue, the second component includes the division of a multi-channeled unnormalized heatmap layer into individual channels. Each of these channels is then processed through its own average pooling layer. This design allows for simultaneous verification of all keypoints, eliminating the need to verify each keypoint individually.



  
3. Flattening and concatenation. The third component involves the flattening of the previously splitted channels, which are then concatenated into a single-dimensional format for verification.



  §.§.§ Perturbations

 Out of 7000 images, we randomly sample 200 images as seed images, adding local or global perturbations to them.


  
Local object occlusions
We created a set of 40 realistic semantic disturbances featuring personnel and vehicles typically encountered at an airport. These disturbances are depicted in Fig. <ref>. To create perturbed images, we randomly selected 20 objects as patches, each up to 150 pixels in size, positioned randomly on the seed images, with each perturbed image having one patch. The perturbed images are classified into two groups: overlapping and non-overlapping, based on whether the patches overlap with the airplane in the image. There are 4000 perturbed images in total, including 893 overlapping and 3107 non-overlapping images. The convex hull's complexity is adjusted by changing the number of perturbed images m, where m ranges from 2 to 4. These perturbed images are randomly selected, allowing us to systematically assess how the addition of semantic disturbances affects the robustness and performance of the system. Note that the convex hull is comprised solely of either overlapping or non-overlapping images. A collection of perturbed images and sampled images from the convex hull are presented in Fig. <ref>.



  
Local block occlusions Given a seed image, we generate 4 perturbed images. In each perturbed image, 
a 3 × 3 square is placed either away from the airplane or centered over a randomly selected keypoint to create non-overlapping or overlapping images, respectively. All pixel values within this square are randomly selected from the range [0, 255]. A convex hull is formed using these 5 images per seed image, of which all perturbed images are non-overlapping or overlapping ones.



  
Global perturbations
We change each pixel's value through two types of global perturbations: brightness and contrast. For brightness, a variation value b ∈ℤ is applied such that each pixel's value increases by b, that is, I' = (I + b), where I represents the original pixel values, I' the new pixel values, and  ensures the values remain within the range [0, 255]. For contrast perturbation, a variation value c ∈ℝ adjusts each pixel's value by a percentage c, formulated as I' = (I × (1 + c)).  The convex hull is constructed as follows. For a positive value b ∈ℤ_+, two perturbed images are created for b and -b, respectively. These images act as vertices of the convex hull. Along with the seed image, this approach facilitates verification of the model's robustness to any brightness variation within the range [-b, b]. The same methodology applies to contrast variations c ∈ℝ_+. We examine the effects for b values of {1, 2} and c values of 5× 10^-4, 5× 10^-3, 1× 10^-2}.










 

 
 
 
 
 
 




 §.§ Validity of Assumption <ref>


To assess the degree of symmetry and unimodality exhibited by the normalized heatmaps, we use four metrics: skewness, the disparity between mean and median, the disparity between mean and mode (peak), and the Interquartile Range (IQR), which is the difference between the third quartile (Q3) and the first quartile (Q1). A skewness near zero and a minimal difference between the mean and median indicate a symmetrical distribution. Combined with small skewness and close mean and median, a small IQR and close mean and mode suggest unimodality. These metrics are calculated individually for each image dimension. The evaluation uses the ResNet-18-based backbone model  across 3000 seed images and 4000 perturbed images with object occlusions and is presented in Fig. <ref>. The findings reveal skewness values approximately zero and mean and median values that are closely aligned, especially when considering the heatmap dimensions of 256 × 256. The difference between mean and mode and IQR values are also minor relative to the total range of 256. Thus, these statistical outcomes demonstrate that the normalized heatmaps predominantly feature unique peaks and exhibit axis symmetry, supporting Assumption <ref>.







 


 §.§ Probabilistic Soundness and Completeness of Optimal Error Threshold Allocation



  §.§.§ Optimal tolerance allocation

Note that the outcomes of allocating optimal tolerance are influenced by images and defined thresholds for acceptable pose errors. In our experiments, we manipulate these thresholds by incorporating a scaling factor α∈_+, resulting in adjusted error thresholds αϵ_r and αϵ_t.  The pose error thresholds are set as ϵ_r = α· [10^∘, 10^∘, 10^∘] and ϵ_t = α· [4, 4, 20]. The threshold factor α is varied across the values {0.5, 1.0, 1.5, 2.0}. We maintain the scaling factor κ at 1.0 as specified in Eq. (<ref>), and assign the weights w_1 and w_2 values of 1 and 5, respectively. Our analysis includes 3000 images, each contributing 5000 sets of perturbed keypoints, accumulating a total of 1.5 × 10^7 samples. The mean and standard deviation of the tolerance allocated in the horizontal and vertical directions are presented in Fig. <ref>. As the allowable pose errors grows, the tolerance allocated per keypoint increases, predominantly uniform in both the horizontal and vertical axes, with the exception of the 10-th and 11-th keypoints, which are symmetrical with respect to the body axis of the plane (see Fig. <ref>), exhibit larger tolerances horizontally, while the 5-th and 23-th keypoints, aligned along the body axis, have larger tolerances vertically. 

 


  §.§.§ Probabilistic soundness
 In this part, we intend to evaluate the probabilistic soundness of optimal error threshold allocation by verifying if the pose estimation errors resulting from perturbed keypoints fall within the predefined error thresholds. One approach is to perform random sampling within the hyper-rectangle (δ^*). However, given the high dimensionality of , which is 2^2K with K exceeding 20 in our scenario, this method demands an extraordinarily large sample size to achieve sufficient coverage. To address this issue, we only evaluate on vertices. For a seed image we randomly choose vertices from the hyper-rectangle (δ^*), which we then add to the coordinates of the ground-truth keypoints to create perturbed keypoints. Mathematically, in matrix form,
= ⊙δ^* + ,
where the elements of matrix  are randomly set to either -1 or 1, the symbol ⊙ represents element-wise multiplication, and  denotes the perturbed keypoints. This allows the perturbation of keypoints to reach the maximum tolerable errors. 

The simulation setup is largely the same as the previous section. We vary both the scaling factor κ in Eq. (<ref>) and the threshold factor α, and compute the ratio of samples where the pose estimation errors remain within thresholds over the total number of samples.  We report the ratio of samples resulting in tolerable pose errors. The results are summarized in Table <ref>. As we can see, only a small number of samples result in poses that surpass the error threshold. There is a noticeable trend where, vertically, as κ increases, the ratio of acceptable samples rises due to the shrinking of the keypoint error threshold polytope. Horizontally, increasing α results in a lower ratio, as linearization becomes less accurate when the pose moves away from the reference point.
 





















 























  §.§.§ Probabilistic completeness
 
To compute completeness, we randomly sample from the set of tolerable pose errors Ξ, use the perspective projection model (<ref>) to determine keypoints, and then compute the reprojection error relative to the ground truth. We verify whether these reprojection errors remain within the permissible error bounds on keypoints, represented by the hyper-rectangle . This simulation follows the same setup as used for probabilistic soundness. The results are presented in Table <ref>. Vertically, as κ increases, completeness diminishes due to the reduction in the size of , leading to more samples exceeding these bounds. On the other hand, there is no discernible trend when viewed horizontally.

When examining Table <ref>, a trade-off between soundness and completeness is evident. Our framework exhibits better soundness, implying that the set of allocated keypoint errors, , is relatively small compared to the ground-truth set δ_Ξ. This smaller size of  accounts for the reduced completeness observed in the results. To support this, Fig. <ref> illustrates the reprojection errors relative to the optimally allocated bounds (rectangle) for keypoints 1, 3 and 5 from a specific image under the conditions α = 0.5 and κ = 1.0. These reprojection errors are calculated by randomly generating pose errors within tolerable ranges and calculating the difference between ground-truth keypoints and those obtained from perspective projection model (<ref>). The illustration shows that a significant number of samples exceed the computed bounds, contributing to the overall low completeness observed across all keypoints. Therefore, if the verification tool returns a hold, it implies that the pose estimation is robust with high probability. On the other hand, if the tool returns a violation, we cannot draw any definitive conclusions. The conservativeness in error allocation mainly stems from the need to decouple the dependencies among keypoints.





















 §.§ Verification of Local and Global Perturbations

In this section, we assess the robustness of the pose estimation method when subjected to various levels of perturbations. We aim to answer three key questions:

    
    
    
  * How computationally efficient is the resulting neural network verification problem?
    
  * How accurate is the proposed certification method for robust pose estimation?




  §.§.§ Metric


We measure the performance using verification times and verified rates. Verified rate is defined as the proportion of cases where the verification algorithm confirms robustness against those where seed images produce acceptable pose estimation errors.   Another critical measure is verification accuracy which is defined as the proportion of cases where the verification algorithm confirms robustness against those that are indeed robust. However, determining the exact number of truly robust instances is impractical. Consequently, the verified rate serves as a lower bound of verification accuracy as the instances with acceptable pose estimation errors from seed images exceed those that are truly robust.













  §.§.§ Verification results


 We employ the verification toolbox  () <cit.>, which accepts convex hulls as input specifications. is the state-of-the-art verifier that supports a wide range of verification algorithms and is the most user-friendly to extend. It follows a branch-and-bound strategy to divide and conquer the problem efficiently. Two parameters guide this process:  determines the division of an unknown branch into smaller branches for further refinement, and  dictates the approach to navigating through the branch. We set  to use breadth-first search and   to bisect the branch. The computing platform is a Linux server equipped with an Intel CPU with 48 cores running at 2.20GHz and 376GB of total memory, approximately 150GB of which is available owing to multiple users. Additionally, the server includes 4 NVIDIA RTX A4000 GPUs, each with 16GB of memory.

 
 


 
 



  
Local object occlusions for the CNN-based model The results presented in Tab. <ref> indicate that for non-overlapping images, verification time increases as the number m of perturbed images forming the convex hull rises. As the error bounds for pose estimation expand, with threshold factor α increasing from 1.0 to 1.5, the time decreases. A similar effect is observed when the scaling factor κ decreases from 1.5 to 1.0. This reduction occurs because the optimally allocated error thresholds on keypoints expand with increasing α and decreasing κ, as illustrated in Fig. <ref>, which results in fewer nodes. A similar trend is observed for overlapping images. We emphasize that the verification process for overlapping images requires more time than for non-overlapping ones, given the same number of perturbed images and identical pose estimation error bounds, indicating that the keypoint detection model is effective in suppressing disturbances external to the airplane. 

In reference to the verified rates displayed in Tab. <ref>, for non-overlapping images, the rate remains stable regardless of the number of perturbed images, given the same κ and α. Conversely, there is an increase in the verified rate with a decrease in κ and an increase in α, as larger allocated keypoint error thresholds or larger allowable pose error thresholds  result in more images being verified as robust. In the case of overlapping images, a notable trend is the decline in the verified rate when the number of perturbed images increases. This is because a rise in the number of perturbed images means more objects are overlaid on the airplane, which compromises the accuracy of predictions and, in turn, decreases the number of images verified as robust.






  
Global perturbations for the CNN-based model A similar trend to that seen with local perturbations emerges, as indicated in Tables <ref> and <ref>. The verification can handle contrast variations c of 1% and brightness variations b of 2/255. Greater variations in contrast and brightness lead to longer verification times, whereas a larger α reduces verification time and increases the verified rate. 



  
Local block occlusions for the ResNet-18-based model We set κ = 1.0 and α = 1.5. For the convex hull comprised of non-overlapping images, the verification time is 314.4±227.0 in seconds, with a verification rate of 93.5%. Conversely, for the convex hull consisting of overlapping images, the verification time significantly escalates to 1571.7±1213.5 seconds, and the verification rate is 94.0%. 


 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 


 






























 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

 
 
 
 
 
 
 
 
 
 
 
 
 
 
 









 







§ CONCLUSIONS

In this study, we introduce a framework designed to certify the robustness of learning-based keypoint detection and pose estimation methods. Given system-level requirements, our approach transforms the certification of PnP-based pose estimation into the standard verification for classification, allowing us to leverage off-the-shelf tools. The evaluation results demonstrated that our framework can handle realistic semantic perturbations compared to existing methods. We emphasize that our certification framework is general for safety-critical applications that depend on accurate keypoint detection. These include airport runway detection for automatic landing, pedestrain pose estimation for autonomous driving, and anatomical landmark identification for robot-assisted surgery. Future directions of this framework could include: 1)	Expanding the input specifications to represent more perturbations, such as the translational movement of objects. 2) Reducing the conservativeness caused by independent error allocation among keypoints.



§ ACKNOWLEDGEMENT

This material is based upon work supported by The Boeing Company. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of The Boeing Company.


ACM-Reference-Format




