Paper Title: Analysis of Functional Insufficiencies and Triggering Conditions to Improve the SOTIF of an MPC-based Trajectory Planner.
















Analysis of Functional Insufficiencies and Triggering Conditions to Improve the SOTIF of an MPC-based Trajectory Planner
    Mirko Conrad[samoconsult GmbH, Französische Str. 13, 10117 Berlin, Germany], Georg Schildbach[University of Lübeck, Ratzeburger Allee 160, 23562 Lübeck, Germany corresponding author ()]
    July 2024
=============================================================================================================================================================================================





§ ABSTRACT

Automated and autonomous driving has made a significant technological leap over the past decade. In this process, the complexity of algorithms used for vehicle control has grown significantly. Model Predictive Control (MPC) is a prominent example, which has gained enormous popularity and is now widely used for vehicle motion planning and control. 
However, safety concerns constrain its practical application, especially since traditional procedures of functional safety (FS), with its universal standard ISO 26262, reach their limits. Concomitantly, the new aspect of safety-of-the-intended-function (SOTIF) has moved into the center of attention, whose standard, ISO 21448, has only been released in 2022.  Thus, experience with SOTIF is low and few case studies are available in industry and research. 
Hence this paper aims to make two main contributions: (1) an analysis of the SOTIF for a generic MPC-based trajectory planner and (2) an interpretation and concrete application of the generic procedures described in ISO 21448 for determining functional insufficiencies (FIs) and triggering conditions (TCs). Particular novelties of the paper include an approach for the out-of-context development of SOTIF-related elements (SOTIF-EooC), a compilation of important FIs and TCs for a MPC-based trajectory planner, and an optimized safety concept based on the identified FIs and TCs for the MPC-based trajectory planner.



§ KEYWORDS

Automated Driving System (ADS) ,  
Autonomous Vehicles ,  
Trajectory Planning ,  
Model Predictive Control (MPC) ,  
Functional Safety (FS) ,  
Safety of the Intended Functionality (SOTIF) ,  
Functional Insufficiencies (FIs) ,  
Triggering Conditions (TCs) ,  
SOTIF-related Element out of Context (SOTIF-EooC)




§ INTRODUCTION


Since the beginning of the 21st century, automated and autonomous vehicles have become a major subject of research across the globe, in academia and industry <cit.>. The technological progress is fueled by ever more powerful hardware for computation, communication, and sensing. Another relevant factor is the rapid advances in mathematical algorithms, especially in Artificial Intelligence (AI), Machine Learning (ML), and Numerical Optimization (NO). A plethora of new and powerful methods have been developed and successfully demonstrated in actual vehicles on the road, among them Model Predictive Control (MPC). 

MPC is a popular computer-based control method <cit.> that has been frequently tested in automated driving system (ADS) prototypes <cit.>. Thus far, however, it has not been considered a viable option for series production. The main reason is its relatively high complexity, causing relatively high efforts in the development process and increasing the computational hardware requirements. Arguably, an even bigger challenge results from safety concerns related to MPC. These concerns are similar to other state-of-the-art methods of AI, ML, and NO. 

Due to the complexity of these methods, and the complexity of the corresponding automated driving tasks, Functional Safety (FS), which is concerned with safety in the presence of malfunctions, is no longer the single most important aspect of safety. Instead, Safety of the Intended Function (SOTIF), which is concerned with the  safety of the nominal function even if all system elements work as intended, becomes equally important. The intention of this paper is hence to present a contribution towards a viable SOTIF concept for an MPC-based motion planning module in automated driving systems (ADSs).



 §.§ Model Predictive Control


Over the past decades, MPC has developed into a mainstream approach for the control of complex and safety-critical systems, including ADSs. The basic idea is to formulate and solve a finite-horizon optimal control problem (FHOCP) as a numerical optimization program in real time <cit.>. The FHOCP contains a model of the system and naturally integrates constraints on the model states and control inputs. It can be used for motion planning <cit.> and trajectory tracking control <cit.>, where feedback is introduced by a receding-horizon implementation. It can also be used as a hybrid module, covering some or all aspects of planning and tracking control <cit.>. It has even been used to cover some aspects of tactical planning, such as lane change decisions <cit.>.

The current trend is to use MPC as a motion planner, such as in the commercial product ProDriver by Embotech <cit.>. The goal is to integrate as many aspects as possible into the motion planning module, in order to slim down the module interfaces. Interfaces always require additional engineering efforts and they are a common source of functional, and thus also safety problems. The MPC module should thus be capable of making tactical decisions, by introducing integer variables, and ideally should also provide a basic level of vehicle and actuator control.

Recent research in the context of MPC for ADSs has begun to focus on interaction-aware motion planning <cit.>. The basic idea is to consider the predictions of dynamic objects in the environment not as invariably fixed, but as interdependent with the other objects and also dependent on the actions of the ego vehicle. Hence the future trajectories of dynamic objects are included in the model-based predictions in the MPC. The interactions themselves can be modeled by different approaches, such as game theory <cit.> or neural networks <cit.>. 

A common idea is to represent the intention of other drivers as modes <cit.>, corresponding to lane keeping or a lane change, for example. The remaining uncertainty regarding the exact trajectory of the dynamic object can be considered as stochastic noise. Another noteworthy approach is Branch MPC <cit.>, where the modes of each dynamic object and the ego vehicle may change at several specific instances over the prediction horizon.



 §.§ Safety of the Intended Function (SOTIF)


SOTIF is a comparatively new branch of safety that is becoming increasingly important for SAE driving automation levels 3 or higher. The first edition of the applicable SOTIF standard ISO 21448 has only been published in 2022 <cit.>. 
Due to its relatively recent introduction, scientific literature on the application of SOTIF to ADSs is still  sparse. 
Zhu et. al. <cit.> emphasize that the research on the SOTIF of automated and autonomous driving systems is still in its infancy. Their paper focuses on safety analysis methods that could be applied to SOTIF, but it does not cover specifically the topic of identifying and analyzing functional insufficiencies (FIs) or triggering conditions (TCs).


Qidong et. al. <cit.> state that the identification and systematic analysis of FIs and TCs remains a major challenge for the SOTIF. Using an Adaptive Cruise Control (ACC) system as an example, the authors propose a scenario-based analysis method of SOTIF triggering conditions. In this approach, the TCs are analyzed according to elements of SOTIF scenarios that are formed in view of FIs. 
The approach is then applied to the 'Sense' portion of the ACC system. The 'Plan' part is not addressed in this paper.



Junfeng et. al. <cit.> apply the System-Theoretic Process Analysis (SPTA) method to study the SOTIF of a Lane Keep Assistance (LKA) system. The study intends to verify the feasibility of STPA as part of the SOTIF lifecycle.
The paper proposes a set of TCs for the LKA, but not all of them seem to be TCs in the sense of ISO 21448. A subset of these TCs will be used as input for the SOTIF analysis of the MPC-TP. 
In the analysis, there is also no specific focus on the 'Plan' part.



 §.§ Content and Contributions


This paper describes a possible approach for the analysis of FIs and TCs, with the goal of improving the SOTIF of a generic planning module for an ADS. The trajectory planning module is based on the Model Predictive Control (MPC) approach. It is assumed that the MPC-based trajectory planner (MPC-TP) can be integrated into different vehicle-level functions and that it is re-usable for different vehicles within a specified vehicle class. Hence it is treated as a SOTIF-related element out of Context (SOTIF-EooC), which is a particular novelty compared to the existing literature.

The main objective of this paper is to demonstrate how the framework of ISO 21448 <cit.> can be used for improving the SOTIF of a modern, state-of-the-art planning algorithm. The system architecture and functional specifications are outlined in Section <ref> and the details of the MPC-TP are provided in Section <ref>. The relevant SOTIF background is outlined in Section <ref>. The main part of this paper is the SOTIF analysis in Section <ref>, which describes the application of various methods and techniques from ISO 21448 <cit.> towards the identification of FIs and TCs for the MPC-TP. The analysis leads to a refined specification and design of the MPC-TP, as described in Section <ref>. The conclusions of this work are presented in Section <ref>. The main contributions of this paper can be summarized as follows:


  * Application of the abstract / generic framework of the new SOTIF standard ISO 21448 to a realistic case study, with a focus on early parts of the SOTIF lifecycle;

  * Interpretation and application of generic methods in ISO 21448 for the determination of FIs / TCs, in the context of a specific use case;

  * Derivation of a structured list of FIs / TCs as a consolidated result of the SOTIF analysis (in a tabular format);

  * A refinement of the notion of an out-of-context development in the scope of SOTIF.










§ INITIAL CONCEPT FOR VEHICLE LEVEL CONTROL SYSTEM






The starting point for the design of the planning module as a SOTIF-SEooC are assumptions about the encompassing vehicle control system into which the module is to be embedded, as well as the related interfaces. Furthermore, specifications are needed about the overall functionality and the operational design domain (ODD). 



 §.§ Functional Architecture


The functional architecture of a generic ADS is  shown in Figure <ref>. The MPC-TP is shown in the middle (red). The task of the MPC-TP is to compute a reference trajectory, which is then used by the trajectory controller (blue) and lower-level modules to control the vehicle's actuators. The actuators are an integral part of the chassis, including, in particular, the drivetrain and braking systems for longitudinal motion control and the steering system for lateral motion control. 

The MPC-TP receives information from centralized estimation modules (green), which may serve also other functions and/or other modules in the architecture <cit.>. The key point is that they are developed independently of the MPC-TP. These modules provide a local map, including the topology and road geometry and information about the infrastructure, such as traffic signs (Map Fusion). They also include the relative position of the vehicle on the map, the vehicle's dynamic states, a list of relevant vehicle parameters, and diagnostic information about the vehicle's sensors and actuators (Ego Motion Fusion). Furthermore, a list of relevant traffic objects in the vicinity of the vehicle is provided, including their classification, position, speed, and other states, as well as a situation analysis and prediction(s) about (possible) future evolution of the situation (Object Fusion and Prediction). The sensors also deliver important information about the environment, such as ambient temperature or road surface (Environment Fusion). Finally, the MPC-TP may receive high-level commands by the human driver or the ADS, such as the reference speed, which lane or road to use, or where to take a turn (Tactical Planner).









	






 §.§ Encompassing Vehicle Control System


The MPC-TP is treated as a SOTIF-related element out of context (SOTIF-EooC), as detailed in Section <ref>. Hence it shall be possible to integrate the MPC-TP into different the SOTIF-related systems as well as different vehicle types. For this paper, it is assumed that the MPC-TP can be used in the context of the following systems for a given class of passenger cars:

    
  F1) Energy Efficient Lane Centering and Adaptive Cruise Control (E2LCACC): E2LCACC is a SAE level 2 function that combines the classic Lane Centering (LC) and Adaptive Cruise Control (ACC). It performs the lateral and longitudinal control of the vehicle, in order to keep the vehicle on a reference path in its lane and to follow the preceding vehicle. E2LCACC contains AI-based elements for energy efficient driving, in particular to determine the optimal speed profile. 
    
  F2) Active Lane Change Assist (ALCA): ALCA is SAE level 2 or 3 function designed to help drivers changing lanes safely by automatically steering the vehicle into the adjacent lane. ALCA operates in combination with E2LCACC and also provides longitudinal and lateral control of the vehicle during overtaking. The driver can issue lane change requests to the left or the right, e.g., by operating the indicator lever. The ALCA then checks whether a safe lane change is possible within 10 seconds. If it is possible, it performs the lane change; if it is not possible, it keeps the vehicle in the current lane and continues with E2LCACC.
    
  F3) Highway Pilot (HP): The HP is a SAE level 4 function that takes full control of the vehicle during highway driving. Only high level objectives are provided by the user, such as the destination, the desired route, the targeted arrival time, or the driving style. The driving itself, including all decisions, is performed autonomously by the HP.





 §.§ Operational Design Domain (ODD)


The vehicle-level functions F1, F2, F3 are only to be used within the conditions and limits of the ODD () specified in Table <ref>. The ODD documents the specific conditions under which the ADS is designed to function <cit.>. Its description is based on the structure provided in DOT HS 812 623 <cit.>.





 §.§ Assumptions about the Functionality and the Vehicle Class


Since the MPC-TP is considered as a SOTIF-EooC, specific assumptions are required about its integration into the vehicle and the vehicle-level functions F1, F2, F3, as well as its interfaces to other modules. An overview of the upstream and downstream modules can be gathered from Figure <ref>. The perception modules make use of appropriate sensors and communication infrastructure. The control modules are connected with the relevant vehicle actuators. It is beyond the scope of this paper to specify these modules here in detail. A list of the most relevant specifications for the ego vehicle (EgoV) and the module interfaces is provided below.


    
  Spec:ODDj) The EgoV belongs to a specific class of passenger vehicles, consisting of a finite number of vehicle types.
    
  Spec:ODDk) All possible EgoV types are known at the time of the MPC-TP development process, including their physical design and specifications, chassis and suspension systems, and chassis control systems (Anti-lock Braking System, Electronic Stability Control, etc.).
    
  Spec:ODDl) Further parameters of EgoV (such as the vehicle mass, location of the center of gravity, moments of inertia, and tire parameters) are known to be within specified value ranges.
    
  Spec:ODDm) The loads inside the EgoV are within the limits specified by the respective vehicle type.
    
  Spec:ODDn) There are no external loads attached to the EgoV, such as a trailers, roof racks, or any type of load carriers. 




§ INITIAL CONCEPT OF THE MPC-TP


The specifications for the MPC-TP are based on the overall architecture in Figure <ref>. The MPC-TP is (initially) given by the single block displayed in Figure <ref>. The specifications concern the input signals, the output signals, and the contents of the block itself, and they are presented along this structure in the remainder of this section.





 §.§ Input Interface / Perception


For the interface to the map fusion module, the 6 layers road model specified in ISO 34503 <cit.> is considered. See Figure <ref> for an illustration.






  ) Map Fusion: The MPC-TP receives a layered map every 50 ms, including

    
  Spec:MapFusiona) geometry and topology of the road, lane and road boundaries (layer 1),
    
  Spec:MapFusionb) traffic signs, speed limits, traffic guidance, construction barriers (layer 2),
    
  Spec:MapFusionc) temporal modifications, e.g., due to construction sites (layer 3).


For the sake of the SOTIF-EooC, it is assumed that all of this map data is always fully accurate. The MPC-TP must plan its trajectories such that they conform to the applicable traffic rules (e.g., speed limit, no passing zone).



  ) Object Fusion and Prediction: The MPC-TP receives a complete list of dynamic objects in its relevant vicinity every 50 ms, including

    
  Spec:ObjectFusiona) other traffic participants (layer 4),
    
  Spec:ObjectFusionb) animals and other temporary obstacles, like dropped cargo, rocks, etc. (layer 4).


All dynamic objects include additional information about their classification (vehicle, truck, motorcycle, pedestrian, animal type, obstacle, etc.), position, velocity, and heading direction, if applicable. Furthermore, a reliable prediction for all of these objects is provided over the next 6 seconds. The uncertainty in these predictions is accounted for by the use of intervals (e.g., for the velocity) or bounding boxes (e.g., for the position).

No assumptions are made on specific sensors, their field-of-view, or their operational status. All assumptions are based on the interfaces to the sensor fusion modules. If the information flow on these interfaces cannot be guaranteed, the vehicle-level function will hand over to the driver (SAE levels 1,2,3) or transfer the vehicle to a safe state (SAE levels 4,5).



  ) Ego Motion Fusion: The MPC-TP receives relevant information about the states and parameters of the ego vehicle every 50 ms, along with a with maximum estimation error (marked with a “Δ”):

    
  Spec:EgoFusiona) the relative position of the ego vehicle on the map x,y (Δ x, Δ y),
    
  Spec:EgoFusionb) the ego vehicle velocity v_lon,v_lat (Δ v_lon,Δ v_lat),
    
  Spec:EgoFusionc) the ego vehicle's global heading angle ψ (Δψ),
    
  Spec:EgoFusiond) the ego vehicle's acceleration a_lon,a_lat (Δ a_lon,Δ a_lat),
    
  Spec:EgoFusione) the vehicle's mass m (Δ m),
    
  Spec:EgoFusionf) the vehicle's moment of inertia around the vertical axis I_z (Δ I_z),
    
  Spec:EgoFusiong) the position of the center of gravity relative to the midpoint of the rear axle x_cog, y_cog, z_cog (Δ x_cog, Δ y_cog, Δ z_cog),
    
  Spec:EgoFusionh) the cornering stiffness of the front and the the rear tires C_α,f, C_α,r (Δ C_α,f, Δ C_α,r).



  ) Tracking Controller: Further information about the ego vehicle, in particular about the states of the lower-level actuators, are provided to the MPC-TP through the Tracking Controller every 50 ms:

    
  Spec:TrackControla) the front wheels' steering angles δ_f,l, δ_f,r,
    
  Spec:TrackControlb) the current engine torque T_eng,
    
  Spec:TrackControlc) the total gear ratio i_gear,
    
  Spec:TrackControld) the effective brake torque T_brake.
    

Actuator failures, controller failures or performance degradations are not considered in this paper, as the focus is on SOTIF instead of functional safety.


  ) Environment Fusion: The MPC-TP receives accurate information about the environmental conditions every 50 ms, including

    
  Spec:EnvironmentFusiona) conditions of the road surface, in particular the friction coefficient μ (Δμ) (layer 1),
    
  Spec:EnvironmentFusionb) weather conditions and lighting conditions, in particular the side wind speed v_wind (Δ v_wind) (layer 4).


In reality, environment parameters, such as the friction coefficient or the side wind speed, are variable over space and/or time. For simplicity, however, the architectures presumes the communication of constant values, which characterize the entire drivable area of the ego vehicle over the relevant prediction time, i.e., in this case 6 seconds. The tolerance thus includes the measurement uncertainty as well as the true temporal and/or spacial variations, if applicable.



  ) Tactical Planner: The MPC-TP receives tactical choices from the Tactical Planner every 50 ms, containing

  
  Spec:TacticalPlanninga) the desired target lane (e.g., the current lane, the left or right lane), 
  
  Spec:TacticalPlanningb) a desired reference speed (e.g., because the vehicle is about to leave the highway on the next exit),
  
  Spec:TacticalPlanningc) optional information about the desired lateral position on the target lane (e.g., deviation from the lane center line),
  
  Spec:TacticalPlanningd) optional information about desired distances to other vehicles or objects (e.g., the desired gap to the leading or trailing vehicle).



All requests by the Tactical Planner are facultative. For example, if the Tactical Planner requests a lane change, it is the task of the MPC-TP to evaluate if this lane change is feasible and safe, given the dynamics of the vehicle and the traffic situation (dynamic and static obstacles, traffic rules, etc.). If it is not feasible or not safe, the MPC-TP resorts to a trajectory that keeps the EgoV on its current lane. Similarly, the desired distances in (Spec:TacticalPlanningc,d) are for performance purposes only, and not required for safety.

The tactical choices are generated by the active driving function. For example, for the E^2LCACC the unique command is always to keep the current lane, for the ALCA the lane change commands are issued by the driver, and for the HP the commands are issued by a dedicated software module.





 §.§ Output Interface / Reference Trajectory


The main output of the MPC-TP is a reference trajectory, as shown in the architecture in Figure <ref>. The reference trajectory consists of (i) a trajectory header and (ii) a trajectory body. The trajectory header contains a sequential number, the generation time, a validity time interval, and a checksum.



  ) Trajectory Body: The main part of the reference trajectory is the trajectory body. It is represented by N_p-1 linear segments, as illustrated in Figure <ref>. Each segment, numbered with i=1,…,N_p-1, carries the following data:

    
  Spec:TrajectoryBodya) x_i,y_i: the planar coordinates of the starting point of the segment,
    
  Spec:TrajectoryBodyb) ψ_i: the heading angle of the segment,
    
  Spec:TrajectoryBodyc) v_i: the reference speed of the vehicle along the segment,
    
  Spec:TrajectoryBodyd) δ_i: the reference steering angle along the segment,
    
  Spec:TrajectoryBodye) l_i: the length of the segment.
    
  Spec:TrajectoryBodyf) In addition, the final point of the trajectory is given in terms of its planar coordinates x_N_p,y_N_p.


Note that the trajectory body data are partially redundant. This is mainly to avoid unnecessary and mirrored re-calculations within the MPC-TP.





 §.§ Specification and Design of the MPC-TP


This case study considers a trajectory planner based on Model Predictive Control (MPC). A generic MPC formulation with a mainstream application to highway driving is used, in order to make the SOTIF concept in this paper as generally applicable as possible. It should be remarked, however, that numerous variations of MPC-based planning algorithms have been proposed in the recent literature <cit.>. Moreover, commercial modules for trajectory planning with MPC have been made available, such as the software ProDriver by Embotech <cit.>. It assumes a similar functional architecture as the MPC-TP in this paper, with a sampling time of 100 ms. 

The computations of the MPC-TP are based on an internal prediction model. Following <cit.>, a  dynamic bicycle model with a linear tire model (LDBM) is selected and discretizes with an appropriate numerical integration method (e.g., Runge-Kutta):

    𝐱_t+1=f(𝐱_t,𝐮_t) .

The model is augmented with a suitable patch to circumvent the singularity of the tire forces at the origin. Here t=0,1,2,… denote the discrete time steps, corresponding to multiples of the fundamental sampling time t_s=50 ms. The vector 𝐱_t∈ℝ^6 includes the (model) states at step t, namely

    
  * s_t,e_t: coordinates of the vehicle's reference point in road-aligned (Frenét) coordinates, 
    
  * ψ_t,ω_t: yaw angle and yaw rate of the vehicle,
    
  * v_t,u_t: longitudinal and lateral velocity of the vehicle.

The vector 𝐮_t∈ℝ^2 contains the (control) inputs at step t, namely

    
  * δ_t: the steering angle of the front wheel,
    
  * M_t: the torque of the drivetrain at the wheels, used to accelerate or decelerate the vehicle.

Furthermore, a variable for the reference lane n_t is used to indicate the lane decision for the vehicle at time t. It is assumed that the lanes are numbered from right to left, starting with 1 and counting up to the total number of lanes. The set of available lanes at step t is denoted ℕ_t.

Based on the prediction, the concept of the MPC-TP is to set up and solve the following Optimal Control Problem (OCP) in each time step, using an appropriate numerical optimization solver:

    n_t,𝐮_t,𝐱_tminimize      ℓ_s(s_N_p)+∑_t=1^N_pℓ_e(e_t,n_t)+ℓ_o(s_t,e_t)+ℓ_u(δ_t,M_t) ,
    s.t.      𝐱_t+1=f(𝐱_t,𝐮_t)    ∀ t=0,1,…,N_p-1 ,
       𝐱_0=𝐱̅_0 ,
       𝐧_t∈ℕ_t    ∀ t=1,2,…,N_p ,
       𝐮_t∈𝕌    ∀ t=0,1,…,N_p-1 ,
       𝐱_t∈𝕏^(s)_t ,  𝐱_t∈𝕏^(t)_t    ∀ t=1,2,…,N_p-1 ,
       𝐱_N_p∈𝕏_f .

The objective function (<ref>a) comprises the following cost terms:

    
  * ℓ_s(s_N_p): rewards the progress along the road made by the vehicle over the prediction horizon;
    
  * ℓ_e(e_t,n_t): penalizes the lateral deviation of the vehicle from the center line of the corresponding reference lane;
    
  * ℓ_o(s_t,e_t): repels the vehicle from static and dynamic obstacles, e.g., by means of a potential field;
    
  * ℓ_u(δ_t,M_t): favors a low input usage and smooth changes in the inputs, e.g., for driver comfort and energy efficient driving.

These cost terms are multiplied by individual tuning weights, which are selected during the control design process.

The initial state 𝐱̅_0 of the LDBM (<ref>b,c) is obtained from the module for Ego Motion Fusion. The lane variables n_t over the prediction horizon in (<ref>d) are either a decision variable of the OCP (as for the HP), or they can be pre-determined before solving the OCP (as for E2LCACC and ALCA). Equations (<ref>e,f,g) represent the input constraints, state constraints, and terminal constraints, respectively. The state constraints are split into two sets:

    
  * The safety constraints 𝕏^(s)_t avoid collisions of the EgoV with static and dynamic objects, most importantly other vehicles, called target vehicles (TgtVs). Furthermore, the constraints ensure the validity of the tire model, by restricting the tire forces to remain within the admissible bounds of the linear tire regime (i.e., no slip in the tire-road contact patch).
    
  * The traffic rule and comfort constraints 𝕏^(t)_t represent the constraints from traffic rules (e.g., the EgoV must stay within its lane, the EgoV must respect the speed limit) and passenger comfort (e.g., no excessive acceleration or jerk).

The safety constraints are essential and cannot be relaxed, whereas the traffic rule and comfort constraints can be relaxed in emergency situations. Furthermore, in emergency situations, the LDBM in (<ref>) can be replaced with a nonlinear dynamic bicycle model (NDBM), which uses a Pacejka tire model instead of the linear tire model. By these modifications, the MPC-TP may temporarily operate the vehicle close to its dynamic limits and against the traffic rules, e.g., driving in between two lanes, or without considering passenger comfort.



  ) Further specifications related to the MPC formulation include the following:

    
  Spec:MPCa) The fundamental sampling time is t_s=50 ms.
    
  Spec:MPCb) The planning horizon is N_p=120 steps, which corresponds to a prediction time of T_p=6 s.
    
  Spec:MPCc) The input constraint set 𝕌 represents all limits of the steering, engine and braking systems, as well as the relevant limits of their rate-of-change.
    
  Spec:MPCd) The state constraint sets 𝕏_t are time-dependent and prevent collisions with static and dynamic obstacles, with the exception of vehicles driving behind the ego vehicle on the same lane. Moreover, the state constraints keep the vehicle away from its dynamic limits and ensure satisfaction of the traffic rules, e.g., by preventing forbidden lane changes or violation of the speed limit.
    
  Spec:MPCe) The terminal constraint 𝕏_f corresponds to a safe state of the vehicle in which it can remain indefinitely (e.g., at standstill), or it represents a safe exit strategy (e.g., maximum deceleration until standstill). Its purpose is to ensure the recursive feasibility of the OCP.



The numerical computations of the MPC-TP comprise two main steps:

    
  * setup of the OCP problem,
    
  * solution of the OCP problem.

Step 1 is to parse the OCP into the standard form of a nonlinear programming problem, which is realized by handwritten code. Step 2 is to solve the nonlinear program using a generic optimization solver, for which a generous selection of software packages is available <cit.>. 


    
  ) Besides the optimal trajectory, the optimization solver returns an exit flag q_exit in exactly one of four classes: 

    
    
  q_exit=1: A (local) minimum is found with an acceptable tolerance level.
    
  q_exit=2: A feasible point is found, but without an optimality property (e.g., because the gradient / search direction becomes too small or the solver has reached the maximum number of iterations / solution time).
    
  q_exit=3: The solver detects that no feasible point exists (`certificate of infeasibility').
    
  q_exit=4: A feasible point is not found, but without a certificate of infeasibility (e.g., because the solver has reached the maximum number of iterations / solution time), or an identified or unidentified solver error occurred (e.g., due to insufficient memory).



In reality, most solvers provide further diagnostics of the results, i.e., their exit flag is more detailed than the four classes above. The requirement (Spec:ExitFlag) states that it must be possible to map the true exit flag of the solver uniquely to one of the indicated classes. Moreover, it is assumed that the exit flag correctly characterizes the result of the solver. This can be verified by a separate solution checker. Cases where the exit flag is incorrect are not considered further as part of the SOTIF analysis. They represent software fault and hence belong to the realm of FS.



 §.§ Hazards at the Vehicle Level, Safety Goals, and SOTIF Requirements


For the SOTIF analysis in this paper, the following hazards at the vehicle level are considered:


    
    
  ) The ego vehicle leaves the assigned lane.
    
  ) The ego vehicle collides with other vehicles or obstacles.

These hazards lead directly to the specification of the following safety goals:


    
    
  ) The ego vehicle shall not leave the assigned lane(s).
    
  ) The ego vehicle shall not collide with other vehicles or obstacles.


As part of a top-down safety lifecycle, these safety goals would be broken down recursively into more and more detailed safety requirements until one arrives at safety requirements for the component in question. Safety requirements for an ADS comprise both functional safety requirements and SOTIF-related safety requirements (or SOTIF  requirements in short).
Such a refinement of safety requirements does not take place when developing a SOTIF-EooC. Instead, safety requirements must be assumed for the out-of-context element under consideration. The SOTIF-SEooC's fulfillment of these assumed safety requirements represents its contribution to achieving the original safety goals.

For the MPC-TP, we assume the following SOTIF-related safety requirements: 


    
  ) The MPC-TP shall provide a reference trajectory within t_c=30 ms.[The computation time is less than the sampling time t_s=50 ms, in order to account for additional communication delays.]
    
  ) The reference trajectory provided by the MPC-TP has to be consistent regarding the redundant information in the trajectory body. 
    
  ) The reference trajectory provided by the MPC-TP has to be admissible, in the sense that it must satisfy all relevant actuator constraints:

    
  SR:Admissiblea) The minimum / maximum longitudinal acceleration of the vehicle is a_lon,min=-10 m/s^2, a_lon,max=+3 m/s^2.
    
  SR:Admissibleb) The minimum / maximum velocity of the vehicle is v_min=0 km/h, v_max=130 km/h.[The minimum velocity assumption means that the EgoV may not drive in reverse. A maximum velocity is required by the limited forward view of the vehicle's sensor suite.]
    
  SR:Admissiblec) The maximum front wheel steering angle of the vehicle (in each direction) is δ_max=34^∘.
    
  SR:Admissibled) The maximum front wheel steering rate (in each direction) is γ_max=68^∘ 1/s.
    
  SR:Admissiblee) The maximum lateral acceleration of the vehicle (in each direction) is a_lat,max=5 m/s^2.



Similar to the upstream perception modules, no detailed technical specifications have made for the downstream Tracking Controller (TC). As it is developed independently, responsibilities have to be clearly allocated and distributed between the MPC-TP and the TC. Thus requirements have to be specified for the interface, i.e., in this case, the reference trajectory. The approach proposed in this case study is to require “drivability” based on a nonlinear dynamic double-track model (NDDM):

    
  ) The reference trajectory provided by the MPC-TP shall be drivable, i.e., it must be possible to track it with a NDDM using some arbitrary controller, such that all of the following conditions are satisfied:

    
  SR:Drivablea) The initial condition of the NDDM corresponds to the measured initial condition of the vehicle.
    
  SR:Drivableb) The parameters of the NDDM correspond to those of the vehicle (mass m, moment of inertia I_z, position of the center of gravity x_cog and y_cog, front and rear cornering stiffness C_α,f and C_α,r).
    
  SR:Drivablec) The tires of the NDDM do not saturate, i.e., there is no slip in the tire-road contact patch, for the worst-case friction coeffient μ-Δμ. This means that the tire forces remain inside the linear regime at all times.
    
  SR:Drivabled) The bank angle is between β-Δβ and β+Δβ.
    
  SR:Drivablee) The side wind speed is between v_wind-Δ v_wind and v_wind+Δ v_wind.
    
  SR:Drivablef) All existing actuator constraints are respected: communication delays, actuator dynamics, actuator limits, and limits of the rates of change.
    
  SR:Drivableg) The maximum lateral offset from the reference trajectory is e_max=5 cm.
    
  SR:Drivableh) The length of each reference trajectory is such that it takes at least 5 s to reach the end.



The set of all consistent, admissible and drivable trajectories is denoted as 𝒯. Additionally, the trajectory should be such that it does not lead to collisions of the vehicle.


  ) The trajectory shall be collision-free, i.e., it must stay within the boundaries of the road and it must avoid collisions with all dynamic obstacles. 

    
  SR:CollisionFreea) For lateral collsition avoidance, the MPC-TP may rely on a maximum lateral offset e_max^TC produced by the TC:
    
    
  * for v≤ 30 km/h: e_max^TC=10 cm,
    
  * for v≤ 80 km/h: e_max^TC=15 cm,
    
  * for v≤ 130 km/h: e_max^TC=20 cm.
    
    
  SR:CollisionFreeb) The MPC-TP shall plan its trajectories such that it is always safe for the ego vehicle to stay in its current lane, in particular by maintaining an appropriate safety distance to the leading vehicle.

 
To this end, the MPC-TP may assume that all of the input information it receives is up-to-date and fully correct. 

Note that the scope of the paper is limited with regards to the considered vehicle level hazards and SOTIF requirements. In reality, there may be further hazards and derived requirements, which may involve also other aspects of the lateral motion, a such as acceleration and jerk, as well as the longitudinal motion of the vehicle, e.g., for overtaking maneuvers.



§ SOTIF BACKGROUND




 §.§ Aspects of Safety


The safety of a technical system is commonly defined as the absence of unreasonable risk for the health and life of humans <cit.>. A reasonable level of risk has to account for the fact that most technological systems, including ADSs, can never be entirely safe. Safety risks of E/E systems can be categorized according to the source of potential hazards into

    
  1) hazards that occur due to malfunctioning behavior (Functional Safety), addressed by the ISO 26262 <cit.> standard, and
    
  2) hazards that occur due to functional insufficiencies (Safety of the Intended Functionality, SOTIF), addressed by the ISO 21448 <cit.> standard.

Functional safety and SOTIF are distinct and complementary aspects of safety that both need to be accounted for in the development of an ADS <cit.>. The exclusive focus of this paper, however, is SOTIF.
 



 §.§ SOTIF


The international standard ISO 21448 'Road vehicles - Safety of the intended functionality' published in 2022 defines SOTIF as the absence of unreasonable risk due to hazards resulting from functional insufficiencies of the intended functionality, or its implementation <cit.>. As a basic framework, it defines an iterative lifecycle with several activities that are deemed adequate for developing an ADS that satisfies the SOTIF, as shown in Figure <ref>.
















 §.§ SOTIF-related Element out of Context (SOTIF-EooC)
 

As a consequence of the collaborative development typical for the automotive industry, a supplier company often provides a component that will be integrated by the OEM into a SOTIF-related system. Under this scenario, at the time of developing the component, the the supplier may only know certain aspects of the overarching system. One way of facilitating the achievement of SOTIF in such a context is to treat the component as a SOTIF-related Element out of Context (SOTIF-EooC), according to ISO 21448 <cit.>.   

For a component treated as a SOTIF-EooC, assumptions can be made regarding its use within the whole system and its contribution to the intended functionality. Based on this, assumptions about the SOTIF-related functional insufficiencies (FIs) and their allowed target rate of occurrence can be derived. These assumptions need to be documented. They are used as inputs for the subsequent development of the SOTIF-EooC by the supplier. For the SOTIF-EooC, the identified triggering conditions (TCs) of the component and their resulting FIs are documented together with the assumptions of use for the component (a.k.a. integration requirements).
When the OEM integrates the SOTIF-EooC, the validity of the assumptions or integration requirments is established by  activities in the context of whole vehicle-level functionalities, according to ISO 21448, Clause 4.4.3 <cit.>.

In this paper, the MPC-TP is treated as a SOTIF-EooC. 
In fact, the MPC-TP may be used as part of different driving automation systems, namely E^2LCACC, ALCA, and HP. Assumptions regarding the overall systems, the integration of the MPC-TP and the contributions of the MPC-TP to the intended functionality have been discussed in Sections <ref> and <ref>.
The identification of TCs of the MPC and their resulting FIs are discussed in Section <ref>. Naturally, their analysis leads to an improved architecture and refined specifications, which are presented in Section <ref>.





 §.§ Functional Insufficiencies


r0.55
	

	
    < g r a p h i c s >

	

SOTIF scenario categories.
	



FIs lead to hazards only in specific (driving) scenarios, called the hazardous scenarios, as opposed to non-hazardous scenarios. Conceptually, these scenarios are further sub-divided into known and unknown scenarios. This categorization of scenarios is illustrated in Figure <ref>. Hazardous scenarios may be known or unknown at the start of the  analysis. The overall goal of SOTIF is to improve the specification and the design of the system, or the considered element, with respect to reducing the number of (known and unknown) hazardous scenarios. 









FIs may exist at the system level, e.g., the E2LCACC system, or at the element level, e.g., the 'Sense', 'Plan', or 'Act' parts of the system. Since FIs can lead to hazardous behaviour, their identification and the evaluation of their effects are key activities in the development of a SOTIF related item <cit.>. ISO 21448 subdivides FIs into

    
  * specification insufficiencies, which refer to a (possibly incomplete) specification contributing to hazardous behavior, and
    
  * performance insufficiencies, which refer to limitations of the technical capabilities contributing to hazardous behavior <cit.>.
 



Examples of specification insufficiencies include an incomplete specification of the E2LCACC's headway distance resulting in the ego vehicle not keeping a safe distance to the vehicle in front or the inability of the HP to handle uncommon freeway road signs that were not part of its specification and thus cannot be processed appropriately by the system.
Examples of performance insufficiencies include limitations of technical capabilities of the ALCA such as limited calculation performance, limited perception range of its front radar sensor, or limited actuation capabilities of the foundational steering system  <cit.>.

As this paper is concerned with an MPC-TP for different driving functions, the focus is on FIs at element level, in this case, of a “Plan” component. 





 §.§ Triggering Conditions


Hazardous scenarios can be decomposed into the following elements: (1) A traffic scenario with a triggering condition, (2) which leads to a hazardous behavior of the ego vehicle, (3) which may cause a hazard in combination with the surrounding traffic or environment. Hence a TC is defined as a specific condition of a scenario that initiates a subsequent system reaction contributing to hazardous behavior <cit.>(simplified). 


TCs and FIs are interrelated: A TC of a scenario activates a FI, which results in a subsequent system reaction. Figure <ref> illustrates the activation of FI at the element level by a TC and the resulting hazardous behavior. In general, the SOTIF standard also addresses reasonably foreseeable misuse. This aspect however, remains outside the scope of this paper.



For an example, assume an activated E2LCACC system that uses a radar sensor with an insufficient field of view in cased of curved roads. The upper part of Figure <ref> shows a scenario without a TC. A FI is not being activated. The E2LCACC system works as intended, i.e., the scenario is considered not hazardous.  The lower part of Figure <ref> shows a second, hazardous scenario. Here, the TC is a slower moving vehicle ahead in lane on a curved road. In this case, the assumed FI at the sensor is activated, resulting in a late detection of the leading vehicle. As a result, the system brakes too late, resulting in a rear-end collision with the leading vehicle.   








§ SOTIF ANALYSIS FOR THE MPC-TP




 §.§ Identification and Evaluation of Functional Insufficiencies and Triggering Conditions



ISO 21448 suggests a list of specific methods to analyze potential FIs and TCs <cit.>. Table <ref> shows the list of methods that are selected as appropriate to derive FIs and/or TCs of the MPC-TP, and the ones that are considered further in the remainder of this paper.



For the analysis of the “Plan” component of a system, ISO 21448 <cit.> provides an additional, specific list of methods that can be used for finding FIs and TCs. This list is provided in Table <ref>. The categories considered for the SOTIF analysis in this paper are shown in black, the ones not considered further are shown in gray.



This section describes the application of the selected analysis methods, and the FIs and potential TCs that have been identified. Multiple methods may lead to the same results, in which case the corresponding FIs and TCs are identified with the same number.



 §.§ Analysis of ODD, Use Cases, and Scenarios


This analysis method involves a close examination of the ODD specifications in (Spec:ODD), as well as predictable use cases and scenarios that may lead to potential TCs and FIs of the MPC-TP. Using this approach, the following items have been identified for our case study:  


In congestion situations (TC:AOUCS01), the MPC-TP does not consider the formation of a rescue alley for police or ambulances (FI:AOUCS01). Conversely, the formation of a rescue alley by other vehicles or the presence  of police or ambulances in congestion situations may lead the MPC-TP to perform unusual, illegal, counter-intuitive or even dangerous maneuvers (FI:DangerousManeuver).


In obstacle avoidance scenarios (TC:AOUCS02), the MPC-TP does not allow emergency maneuvers without a proper lane change. For example, it is not possible to avoid an obstacle by driving in between two lanes or by using the shoulder (FI:AOUCS02). Conversely, the emergency obstacle avoidance by other vehicles may lead the MPC-TP to perform counter-intuitive or even dangerous maneuvers (FI:DangerousManeuver).


There is no speed limit on certain sections of the Autobahn. Suppose there is blocking obstacle on the road, such as the rear end of a congestion. If the obstacle is at standstill shortly beyond the prediction horizon of the MPC-TP (TC:AOUCS03) and an obstacle avoidance maneuver is not possible, e.g., by performing a lane change. If the EgoV exceeds a certain speed v_max, this can lead to a rear-end collision because the prediction horizon N_p is not long enough for the EgoV to come to a full stop within it (FI:AOUCS03). 






The design of the current MPC-TP is based on clearly defined traffic rules. It assumes, that surrounding vehicles either have the right-of-way, and are thus considered as obstacles, or they have to yield, and can thus be ignored. However, there are situations where the right-of-way is unclear, such as at the end of two merging lanes (TC:AOUCS04a). Without further measures, this may lead to an overly conservative driving behavior (FI:AOUCS04a), and even a complete deadlock for the EgoV (FI:AOUCS04b).


On freeways, cars typically only drive forwards. However, in exceptional situations, e.g., during congestion or due to police or ambulance operations, it may be necessary to perform reverse maneuvers. Moreover, surrounding vehicles may drive backwards (TC:AOUCS05), in which case the MPC-TP may become infeasible (FI:Infeasible) if the EgoV is at rest or slowly moving forwards.


In the case of a wrong way driver (TC:AOUCS06), the MPC-TP may be unable to compute a feasible trajectory (FI:Infeasible). Or it may lead the MPC-TP to perform counter-intuitive or even dangerous maneuvers (FI:DangerousManeuver).


The MPC-TP shall be able handle situations with an unexpected, sudden change of road conditions. Important examples include a dropping friction coefficient, e.g., due to water / hydroplaning (TC:AOUCS07a), snow / ice (TC:AOUCS07b), or oil (TC:AOUCS07c), or an irregular surface, e.g., due to gravel or stones (TC:AOUCS08a), pot holes (TC:AOUCS08b), or road bumps (TC:AOUCS08c). In such situations, the trajectory generated by the MPC-TP may actually be not drivable, in the sense of (SR:Drivable), e.g., because of the limitations of the LDBM using a linear tire model (FI:Drivable).





 §.§ Analysis of System Architecture (Including Redundancies)


This analysis is based on the system architecture, as depicted in Figure <ref>. The goal is to identify potential FIs and TCs based on the overall behavior of the entire system.

The receding-horizon implementation causes the MPC-TP to act also as a feedback controller for the vehicle, which is a well-known feature of MPC <cit.>. The interoperation of the MPC-TP with the tracking controller (TC) may thus lead to undesired bahavior, such as oscillations (FI:ASA01), or even instability of the EgoV motion. This may be triggered, for example, by external disturbances such as a side wind gust, e.g., on a bridge deck (TC:ASA01), or irregular, non-continuous lane markings, e.g., due to suddenly disappearing (TC:ASA02a), dirty (TC:ASA02b), broken (TC:ASA02c), or covered (TC:ASA02d) lane markings.

Difficulties may also appear in situations where the right-of-way is unclear (TC:AOUCS04), or when other traffic participants do not strictly adhere to the traffic rules (TC:ASA05). Their interaction with the MPC-TP may then lead to an overly conservative behavior of the EgoV (FI:AOUCS04), possibly resulting in a poor progress on the road (FI:AOUCS04a), or even a deadlock (FI:AOUCS04b).

The cut-in of a faster TgtV in front of the EgoV (TC:ASA04) may lead to a violation of the front safety distance, and hence the inability of the MPC-TP to generate a feasible solution (FI:Infeasible). This can be resolved by relaxing the minimum safety distance constraint in case of a leading vehicle with a higher velocity. Even if the safety distance is not violated, the cut-in of a faster TgtV (TC:ASA04) may still lead to an overly conservative braking maneuver on behalf of the EgoV (FI:AOUCS04c).


The MPC-TP does not consider a trailing vehicle driving closely behind the EgoV in the same lane (TC:ASA06). According to (Spec:MPCd), even though it is the responsibility of the trailing vehicle to keep a sufficient safety distance, sudden braking maneuvers may lead to a collision with the trailing vehicle and are thus unsafe (FI:ASA03).




 §.§ Analysis of Functional Dependencies


In the context of this case study, functional dependencies are interpreted as other modules that are actually or potentially connected with the MPC-TP, be it upstream or downstream. The MPC-TP needs to cope with challenges arising from the limitations of these modules. With this interpretation of functional dependencies, TCs and FIs identified with this analysis method may overlap with those identified by analyzing the system architecture (in Section <ref>), in the context of the entire system.


Dynamic obstacles suddenly appearing in the field of view of the system (TC:AFD01), may overwhelm the ability of the MPC-TP to solve the OCP in order to compute a new trajectory in due time (FI:InsufficientTime), violating (SR:Timing), or to find a feasible trajectory at all (FI:Infeasible), violating (SR:CollisionFree). Examples of this group of FIs are a previously shadowed traffic participant (TC:AFD01a), an object dropping from a bridge or a truck (TC:AFD01b), or a large animal (e.g., a wild boar) crossing the freeway (TC:AFD01c).  


At points of sudden traffic rule changes (TC:AFD02), such as the beginning of a speed limit area or a no passing zone, the MPC-TP may also unable to compute a feasible trajectory (FI:Infeasible), or may be unable to solve the OCP within the allotted time (FI:InsufficientTime). 





 §.§ Analysis of Algorithms and Their Outputs or Decisions


In the context of the MPC-TP, this analysis pertains to the formulation of the OCP (<ref>) and the numerical optimization algorithm, the decisions taken by the MPC-TP, as well as the reference trajectory and the exit flag as its outputs.

The OCP is based on the LDBM, whose linear tire model may represent a poor approximation of a real vehicle if essential model assumptions are violated (FI:AAOD01). The validity of the linear tire model is already supported by corresponding constraints 𝕏^(s)_t (see Section  <ref>). The other main assumption of the LDBM is a non-negligible roll motion of the vehicle. It can be violated by an extremely asymmetric load (TC:AAOD01a), a highly dynamic cornering maneuver (TC:AAOD01b), or a high position of the CoG (TC:AAOD01c). 


Potential problems arising from the interoperation of the MPC-TP with the TC and from the lack of interaction-awareness have already been discussed (see Section <ref>). In addition, the MPC-TP may cause a lack of string stability in a chain of vehicles (TC:StringStability). If the leading brakes  abruptly, this can lead to the build-up of an increasing wave of braking maneuvers by all following vehicles. This string instability may be fostered by the MPC-TP in the middle of a chain of human-driven or automated vehicle, which may lead to collisions (FI:StringStability), if the MPC-TP is not properly tuned. 


The outputs of the MPC-TP include the reference trajectory and the solver exit flag (see Figure <ref>). 


Without software or hardware faults, the computed reference trajectory will be consistent, admissible, drivable, and collision-free, if the set 𝒯 of such trajectories is non-empty. However, due to the finite prediction time of T_p=6 s, the MPC-TP may lead the vehicle to an (unsafe) situation that cannot be resolved by a future trajectory, i.e., where 𝒯 becomes empty. In MPC language, this is known as the lack of recursive feasibility (FI:RecursiveFeasibility). A possible TC is the fast approach of a slowly driving TgtV in the front of the EgoV, where the TgtV cannot be avoided by a lateral maneuver (TC:SlowLeadingTgtV).

The solver exit flag itself might provide an indication of a triggered FI. In particular, all exit flag values q_exit≠ 1 should be considered carefully:

    
  * q_exit=2 indicates that the MPC may exceed the allotted computation time (FI:InsufficientTime), but produces a feasible trajectory. This could be triggered by rapidly changing traffic situations (TC:AAOD02) or by suddenly appearing obstacles (TC:AFD01). 
    
  * q_exit=3 means that the MPC-TP is unable to generate a feasible (collision-free) trajectory for the given constraints, because 𝒯 is empty (FI:Infeasible). For example, this can be triggered by (TC:AOUCS05), (TC:AOUCS06), (TC:AOUCS07), (TC:AOUCS08), (TC:ASA04), (TC:AFD01),(TC:AFD02).
    
  * q_exit=4 means that the solver is unable to find a solution (FI:NoSolution), even though 𝒯 may be non-empty. As software and hardware faults are out of scope, this must be due to unpredictable numerical problems (e.g., ill-conditioning, floating point arithmetics). To the best of the authors' knowledge, there are no known specific triggering conditions for this case.






 §.§ Analysis of Boundary Values


For the analysis of boundary values, the MPC-TP is examined in the context of scenarios where relevant parameters take on extreme values. Parameters considered in the analysis include

    
  * ODD and road parameters, i.e., the road bank angle, the friction coefficient μ, the road curvature, the change of curvature, and the wind speed;
    
  * maneuver parameters, i.e., the reference velocity, maximum lateral motion (e.g., due to a sudden lane change or obstacle avoidance), maximum longitudinal motion (e.g., due to emergency braking or maximum acceleration);
    
  * vehicle parameters, i.e., the vehicle mass (m), the moment of inertia (I_z), the position of the CoG (x_cog, y_cog, z_cog), the cornering stiffness of the front and the the rear tires (C_α,f, C_α,r), and all chassis and suspension parameters of the specified vehicle class.


The analysis is based on extensive simulations using a high-fidelity vehicle model, e.g, in CarMaker. The simulations scenarios aim to cover possible combinations of extreme values of the above parameters. The results are checked against the requirements (SR:Timing) - (SR:CollisionFree).

To this end, a database of artificial test scenarios covering corner cases, is generated, based on different incarnations of the maneuver parameters. Specifically for the reference velocity, scenarios are separated into low-speed (v≤ v_low=5 km/h), medium-speed (v≤ v_med=50 km/h), and high-speed (v≤ v_high=130 km/h) categories. For road curvature and change of curvature, only scenarios that (at least partially) involve maximum values are considered. The generation process of corner cases can also be automated, as proposed in Lubiniecki et al. <cit.>.

The database of corner cases is then simulated for extreme values of the ODD and road parameters and the vehicle parameters. Due to the combinatorial nature, the number of simulation scenarios thus becomes intractable. To regain tractability, only pairs of extreme parameter values are included in the boundary value analysis, while all other parameters are set to their default or regular values. For some parameters, such as the height of the CoG z_cog, instead of a default or regular values, it is more reasonable to consider only the maximum value in all simulations. Critical cases are expected to result from certain parameter combinations (in some or all corner cases), such as

    
  * the maximum side wind speed v_wind,max = 8 bft and the maximum bank angle β_max = 8^∘ (TC:CommonTrigger1),
    
  * the minimum friction value μ_min=0.15 and the minimum tire cornering stiffness C_α,f,min=60 kN/rad (TC:CommonTrigger2),
    
  * maximum front tire cornering stiffness C_α,f,max=200 kN/rad and the minimum rear tire cornering stiffness C_α,r,min=60 kN/rad, and vice versa (TC:CommonTrigger3).


These TCs may lead the MPC-TP to provide trajectories that are not drivable by a specific vehicle model or configuration (FI:Drivable). However, these are potential TCs and FIs, and their existence and relevance has to be confirmed by actually performing a large-scale, high-fidelity simulation study.



 §.§ Analysis of Common Triggering Conditions


Generally, common TCs are collected from prior experiences with the development of similar systems. They may be available, for example, by internal reports, industry standards, or other publicly documents. The TCs identified in this way may, of course, overlap with TCs identified using other methods.

The authors are currently not aware of a database or any other source for common TCs for a trajectory planner. However, applicable TCs for other ADSs from the relevant literature have been taken into account in this case study. In particular, selected TCs for a LKA system from <cit.> can analyzed with respect to their ability to challenge the MPC-TP and to trigger potential FIs:

    
  * lane keeping / lane changing scenario with suddenly disappearing lane markings (TC:ASA02a);
    
  * lane keeping / lane changing scenario with dirty, broken, or covered lane markings (TC:ASA02b,c,d);
    
  * lane keeping / lane changing scenario with unrecognizable lane markings due to direct sunlight or change of lighting conditions, e.g., when entering a tunnel (TC:ACTC03a,b);
    
  * insufficient visibility of lane markings due to low visibility, e.g., due to snow / ice or fog (TC:ACTC04a,b);
    
  * infrastructure items that could be misinterpreted as lane markings, e.g., curbs or dividers (TC:ACTC05a,b).

Other common TCs for a trajectory planner that can been identified from the existing literature include

    
  * side wind gusts, e.g., on a bridge deck (TC:ASA01a),
    
  * an uneven road surface, due to potholes (TC:AOUCS08b) or road bumps (TC:AOUCS08c),
    
  * a slippery road surface, e.g., due to a wet road with foilage (TC:ACTC09a) or a snowy / icy road (TC:ACTC09b).


For many of the common TCs from literature it remains unclear however, whether they activate FIs in the MCP-TP or elsewhere in the system. This needs to be verified in simulations or real-world experiments.



 §.§ Environment and Location


The systematic analysis of the environment and location aims to identify relevant freeway scenarios and warning signs that may compromise the safety of the MPC-TP. By means of brainstorming, the following scenarios in the ODD can be identified as potential TCs in this context.

An increase or decrease in the number of lanes on the freeway may present a particularly challenging situation for the MPC-TP, including:

    
  * entering the freeway from an on-ramp (TC:EAL01),
    
  * exiting the freeway via an off-ramp (TC:EAL02),
    
  * lane ending / lane merging situations (TC:AOUCS04a),
    
  * lane adding situations (TC:EAL04).


Similarly, situations with special vehicles or vehicles operating beyond traffic rules may pose particular difficulties for the MPC-TP, such as: 

    
  * a wrong way driver (TC:AOUCS06), 
    
  * a reckless driver or person driving under-the-influence (TC:EAL08),
    
  * an approaching emergency vehicle (TC:EAL05),
    
  * a heavy transport or oversized vehicles (TC:EAL06),
    
  * a vehicle parking on the freeway shoulder (TC:EAL07).



Furthermore, rare situations and uncommon road signs on the German Autobahn can be considered in the context of this analysis, leading to the following list of TCs:

    
  * uneven road surface (road sign 112), e.g., due to ruts or blow-ups (TC:AOUCS08d,e),
    
  * an invisible / indetectable substance is substantially lowering the friction coefficient, e.g., due to an oil spillage (road sign 114) (TC:AOUCS07c),
    
  * the freeway shoulder is not available (TC:EAL09) or the shoulder is to be used as an additional lane (road signs 223.1, 223.2, 223.3) (TC:EAL10),
    
  * end of the freeway (road sign 330.1) (TC:EAL11) or a freeway closure (TC:EAL12),
    
  * an unusually steep slope (road signs 108, 110) (TC:EAL13) or an usually high curvature (road signs 103-10, 103-20, 105-10, 105-20) of the road (TC:EAL14),
    
  * dangerous side wind gusts (road sign 117) (TC:ASA01),
    
    
  * potentially unknown traffic scenarios or traffic signs in other European countries (TC:EAL15).


All of the above TCs may affect the ability of the MPC-TP to solve the OCP and compute a trajectory in due time (FI:InsufficientTime), or to find a feasible (collision-free) trajectory at all (FI:Infeasible). Furthermore, they may cause the MPC-TP to provide trajectories which are not drivable and thus lead the EgoV to leave its assigned lane and/or collide with other vehicles or obstacles (FI:Drivable) or which may cause the vehicle to perform unusual, illegal, counter-intuitive or even dangerous maneuvers (FI:DangerousManeuver).



 §.§ Summary of Identified Triggering Conditions and Functional Insuffciencies


Appendix <ref> provides a table with an overview of all triggering conditions and functional insuffciencies that have been identified for the MPC-TP in this case study.



§ REFINED CONCEPT FOR THE VEHICLE LEVEL CONTROL SYSTEM AND THE MPC-TP


As per the SOTIF lifecycle, the SOTIF-EooC analysis in Section <ref> leads to a revision of the overall concept of the vehicle level control system and the MPC-TP, in particular. The results include a refinement of the MPC-TP specifications and design as well as the ODD, as explained in Sections <ref> and <ref>). As a further consequence, the integration requirements to the vehicle level control system are updated, as detailed in Section <ref>. Some situations that could not be (fully) resolved as part of this case study are described in Sections  and .



 §.§ Refined Specification and Design of the MPC-TP


The refined system architecture based on the SOTIF analysis in Section <ref> is depicted in Figure <ref>. The particular refinements of the specification and the design of the MPC-TP are discussed below.





  §.§.§ Design Improvements















The SOTIF-EooC analysis has led to the following concepts to improve the architecture and algorithmic design of the MPC-TP.

An overly conservative behavior of the EgoV (FI:AOUCS04c) may result, e.g., due to the cut-in of a faster TgtV (TC:ASA04). This potential issue can be mitigated by a proper tuning of the MPC-TP.

A trailing TgtV not keeping a proper safety distance to the EgoV (TC:ASA06) may cause a collision in case of a sudden braking maneuvers of the EgoV (FI:ASA03). 

a*) Even though this lies within the responsibility of the TgtV,  the EgoV shall perform sudden braking maneuvers only when strictly necessary and only to an appropriate extent. This can be accomplished by adding a strong penalty on the usage of high braking inputs in the OCP.

Spec:SuddenBrakingb*) Furthermore, the distance to the trailing vehicle shall aslo be included in the tuning of the MPC-TP.

The interaction with other traffic participants if the right-of-way is unclear (TC:AOUCS04) or if they do not strictly adhere to the traffic rules (TC:ASA05) may lead to an overly conservative behavior of the EgoV (FI:AOUCS04). 

*) This should be properly resolved by using an interaction-aware MPC approach, which is the subject of ongoing research <cit.>. For the given architecture, the interaction-aware MPC also has to be designed in coordination with the Tactical Planner.

Finally, a lack of string stability may be fostered by the MPC-TP in a chain of human-driven or automated vehicles (TC:StringStability), which may potentially lead to collisions (FI:StringStability). The problem, however, is not MPC-specific and has been analyzed in the previous literature, including ACC systems <cit.>. It shall be resolved by a proper tuning of the MPC-TP, which should be verified in extensive simulations and/or on-road tests.



  §.§.§ Adaptations to the Reference Trajectory


The interoperation of the MPC-TP with the lower-level TC may lead to oscillations and even unstable behavior (FI:ASA01). 
Spec:TrajectoryBodyg*) As a solution to this, the first N_r=4 steps of the previously computed trajectory remain unchanged, i.e., only the subsequent inputs are newly computed. N_r is called the reaction horizon. The concept is illustrated in Figure <ref>. The initial condition for the OCP in (<ref>c) thus becomes the predicted state x̂_N_r from the previous OCP solution.



As a negative consequence of this modification, the reaction time of the MPC-TP is increased. This is relevant, for instance, if the EgoV has to react to sudden events in its environment, such as in an emergency braking or obstacle avoidance scenario. However, a reaction time of T_d=N_rt_s=200 ms is still superior to that of an average human driver, which is typically between 600 ms and 1,000 ms, according to recent studies <cit.>. Thus a reaction time of 200 ms is deemed to be sufficient for common road traffic.



  §.§.§ Potential Extension of Computation Time


The regular MPC is designed for a maximum computation time of t_c=30 ms. This is less than the given sampling time, t_s=50 ms, in order to account for additional latencies, e.g., due to communication and actuation delays. Obviously, the choice of t_c should be based on generous computational margins and extensive testing. However, it is generally not possible to guarantee that the solver always provides a definite answer within this allotted computation time, i.e., that it finishes with an exit flag q_exit = 1,2,3 in due time (FI:InsufficientTime). Several TCs have been identified as part of the SOTIF analysis in Section <ref> that may represent a potential cause for an excess computation time, such as a suddenly appearing obstacle (TC:AFD01), sudden changes of traffic rules (TC:AFD02), or rapidly changing traffic situations (TC:AAOD02). A strategy to remedy this issue may look as follows:

Due to the concept of the reaction horizon, introduced in Section <ref>, the computation time can be extended by multiple (in this case: N_r=4) time steps. In occasional peaks, it can reach up to a theoretical maximum time of t̅_c=t_c+(N_r-1)t_s=180 ms. After that, a new trajectory must definitely be provided. If multiple time steps are used to solve the OCP, the new decisions (shown in red in Figure <ref>) comprise a corresponding number of time steps, which are now more than a single one.

Furthermore, the solver settings are adjusted such that the iterations can be halted after t_c, if they have not terminated; or again after each t_c+k t_s, for k=1,2,…,N_r-1. After each instance, the status of the current iterate can be checked. If the current exit flag equals to q_exit = 4, the solver may continue its computations from the current iterate, until finally the ultimate maximum solver time t̅_c is reached.





  §.§.§ Backup MPC


The Backup MPC (MPC-TP2) is a redundant implementation of the Primary MPC (MPC-TP1), which has been discussed thus far. The Backup MPC has the same inputs and outputs as the Primary MPC, as shown in Figure <ref>). It is designed according to the same specifications and requirements, as described in Sections <ref> and <ref>, except for the following differences:

    
  * The Backup MPC is utilized for safety and not passenger comfort. It has the ability to perform any maneuver that is physically possible. This means a different tuning of the weight parameters in the cost function and the traffic rule and comfort constraints 𝕏^(t)_t are removed from the OCP in (<ref>f).
    
  * Instead of a LDBM, which has a linear tire model, the Backup MPC shall include a NDBM with a Pacejka tire model. The NDBM provides a higher fidelity for the dynamics of the vehicle at the limits of handling, even though it may have a longer solution time.
    
  * For additional redundancy, the Backup MPC shall apply a different numerical solver (different algorithm and/or software implementation) and it shall be running on an independent computational platform.


The Backup MPC is constantly running in a shadow mode. During normal operation, when the Primary MPC is active, the solutions of the Backup MPC solutions are simply discarded. The trajectories of the Backup MPC are activated, or subsequently deactivated, according to a new module called Infeasibility Analysis. Its logic is based on the exit flag of the Primary MPC q_exit^(P), the exit flag of the Backup MPC q_exit^(B), and various information from the perception modules; see Section <ref> for further details. In particular, the Backup MPC shall be activated if the Primary MPC is unable to find a feasible solution (FI:Infeasible) under the following circumstances:

    
  * The friction coefficient suddenly drops by a value of μ_d>0 within a specified time interval t_d>0, due to water, snow / ice, or oil (TC:AOUCS07), or an irregular surface, e.g., due to gravel or stones, pot holes, or road bumps (TC:AOUCS08).
    
  * Static or dynamic obstacles suddenly appear close to the vehicle, e.g., due to a previously shadowed traffic participant (TC:AFD01a), a lost cargo or other object dropping from a bridge or a truck (TC:AFD01b), or a large animal (e.g., a wild boar) crossing the freeway (TC:AFD01c).
    
  * There is a sudden change of traffic rules (TC:AFD02), such as the beginning of a speed limit zone or a no passing zone.


The Backup MPC remains active until the Primary MPC produces a optimal solution again (case q_exit=1) and it is safe to switch back. For the latter condition, it is important to carefully avoid problems that may arise from a possible periodic switching behavior between the two planners.



  §.§.§ Infeasibility Analysis


The desirable outcome of the Primary MPC is a (local) minimum with an acceptable tolerance (q_exit = 1). The SOTIF analysis in Section <ref>, however, has revealed multiple scenarios where this may not be the case. Then a new module called the Infeasibility Analysis is activated. Its logic is designed as follows:

If the Primary MPC has not converged to such a minimum and returns only a feasible trajectory (q_exit = 2), the sub-optimal solution is acceptable on an exceptional basis. Because the sub-optimal solution satisfies also the terminal constraints, the recursive feasibility of the trajectory planning is not jeopardized. However, situations where the solver detects the definite infeasibility of the OCP (q_exit = 3) or where it is unable to find a feasible solution (q_exit = 4) require further attention. Here a switching logic, as presented in Table <ref>, is proposed as a basis for the Infeasibility Analysis.



Before even activating the switching logic, however, the Infeasibility Analysis filters a few special situations that will obviously or possibly lead to an infeasibility of the Primary MPC (FI:Infeasible). This includes 

    
  * (TC:AOUCS05) a leading TgtV driving in reverse, while the EgoV is at rest or slowly moving forward; 
    
  * (TC:ASA04) at higher speeds, the cut-in of a faster TgtV in front of the EgoV leading to a violation of the safety distance in the present time step.


For (TC:AOUCS05), a special planning mode shall be designed to check if (a) the EgoV is driving with a speed less than or equal to some threshold v_low (e.g., v_low=5 km/h) and (b) the minimum safety distance to the leading TgtV is already violated at the current time step. If this is the case, a stopping trajectory is automatically issued that transitions the EgoV to, or keeps it in, a complete standstill. When the Primary MPC becomes feasible again, the EgoV resumes its normal operation and moves forward. 

For (TC:ASA04), the situation can be resolved by an additional logic that relaxes the minimum safety distance constraint to a leading TgtV in the Primary MPC, if it has a higher velocity than the EgoV. Note that, even if the safety distance is not violated, the cut-in of a faster TgtV (TC:ASA04) may still lead to an overly conservative braking maneuver on behalf of the EgoV (FI:AOUCS04c). This is a separate issue that has to be resolved by a different measure.

In situations with a slippery road surface (TC:ACTC09) or a sudden drop of the friction value μ within a time interval of t_d for more than μ_d (TC:AOUCS07), in a first step the linear tire regime of the LDBM in (<ref>b) shall be exchanged with a (nonlinear) Pacejka tire model. This measure is taken pre-emptively in order to avoid a lack of drivability of the computed trajectory (FI:Drivablea). The Pacejka tire model is not used as the standard tire model of the Primary MPC because of its additional computational complexity.



According to Table <ref>, if the Primary MPC does not produce a feasible solution (q_exit^(P) = 3,4), the Backup MPC is activated already after 2 steps. It should be remarked that this could be extended, if the reaction horizon were increased beyond N_r=4, which is used in this paper. This may allow for further attempts to modify or update the constraints of the OCP in an acceptable way, similar to the steps described above. Or additional attempts may be taken to modify the algorithm or the solver settings in order to achieve an optimal, or at least a feasible solution, of the OCP.







  §.§.§ MPC-TP Driving Modes


To adequately adapt to specific traffic or environmental situations, the MPC-TP needs to introduce a switchable behavior. *) In particular, (at least)the specialized driving modes
listed in Table <ref> shall be introduced.




The proper driving mode needs to be selected by the overarching system and provided to the primary as well as the back-up MPC-TP as additional mode inputs. The MPC-TPs can then adjust their behaviors considering the detected situation. This goes hand in hand with integration requirements to detect the various driving modes.




 §.§ Refined ODD


Spec:ODDi*) To address issues, where the prediction horizon N_p of the EgoV is not large enough for the EgoV to come to a full stop if needed (cf. FI:AOUCS03 in combination with (TC:AOUCS03), we limit the maximum permissible speed in the ODD: 

    v_max=N_p a_lon,min t_s .

As the prediction horizon N_p is static,  v_max must be chosen in accordance with the maximum deceleration a_lon,min.
a_lon,min≤ 0 is the worst-case (highest) deceleration, depending on current road conditions. It depends on the worst-case values of other ODD parameters, in particular the friction value μ.  


Spec:ODDo*) To cope with (FI005) in combination with (TC015c) the ODD will be limited to freeway sections with a game fence. 

(Spec:ODDf*,h*) To mitigate (FI:Infeasible) in combination with (TC:AOUCS07), in particular with (TC:AOUCS07b), we propose to limit the minimum ambient temperature to +4^∘C  (Spec:ODDf*) and the permissible minimum road friction value (Spec:ODDh*). This will be combined with an integration requirement to conservatively estimate the friction value (IR:FrictionEstimation), cf. Section <ref>.  

The refined ODD is provided in in Table <ref>. 





 §.§ Integration Requirements


A key concept to facilitate the out-of-context development of SOTIF-related elements are assumptions on the encompassing vehicle control system. As these need to be ensured during the integration of the SOTIF-EooC into the ADS, they shall be designated as integration requirements. The new integration requirements of the SOTIF analysis are listed in Table <ref>.







Cases where neither the Primary MPC nor the Back-Up MPC is able to generate a permissible trajectory shall be handled by the encompassing system as follows:

(IR:EmergencyOperationRequest) Situations where the MPC-TP requests an emergency operation shall be handled by the encompassing system utilizing a dedicated emergency planner that is independent of the MPC-TP.

Situations involving irregular road surfaces (TC:AOUCS08) where the MPC-TP would be unable to generate a feasible (collision-free) trajectory (FI:Infeasible) shall be identified and handled by the encompassing system. Here, the Tactical Planner needs to decide what to in such cases, e.g., depending on the size of the stones or potholes and provide an appropriate tactical decision to the MPC-TP or handle the situation without the MPC-TP.
This leads to integration requirement (IR:IrregularRoadSurface): Situations involving irregular road surfaces shall be detected and handled by the encompassing system utilizing the Tactical Planner.





 §.§ Situations Requiring Further Analysis


(Further analysis required) The OCP is based on the LDBM, assuming a linear tire model and no roll motion, which may represent a poor approximation of a real vehicle if these assumptions are violated (FI:AAOD01). The non-negligible roll motion may be violated by an extremely asymmetric load (TC:AAOD01a), a highly dynamic cornering maneuver (TC:AAOD01b), or a high position of the CoG (TC:AAOD01c). A detailed simulation analysis is required to examine whether the LDBM is sufficient for the targeted use cases (especially with respect to SR:Drivable), or whether a two-track model is required instead.

Situations encompassing side wind gusts (TC:ASA01) are deemed as adequately covered by the wind limitation in the ODD (Spec:ODDg).

Potentially unknown traffic scenarios or traffic signs, e.g., due to deviating regulations in other European countries, are potentially problematic as well (TC:EAL15). However, the existing ODD restriction limits the permissible roadway types to the German Autobahn (Spec:ODDa) seems sufficient to mitigate this concern.



 §.§ Interaction-aware MPC


Particular combinations of TCs and FIs cannot be adressed without extending the architecture of the encompassing vehicle, e.g. by adding additional senors and/or capabilities. An example is the combination of (TC:AOUCS04a) and (FI:AOUCS04).
As per our analysis the OCP in (<ref>) may lead to an overly conservative behavior of the EgoV (FI:AOUCS04), possibly leading to poor progress on the road (FI:AOUCS04a), or even a deadlock (FI:AOUCS04b). The only safe way to reduce this conservatively is to internalize the relevant TgtVs in the MPC predictions and to reflect the interactions between the TgtVs and the EgoV by appropriate behavioral models. Interaction-aware MPC approaches, however, are the subject of ongoing and future research <cit.> and are beyond the scope of this paper.



§ CONCLUSION


In this paper, multiple aspects of the general SOTIF framework provided in ISO 21448 have been applied to a generic MPC-based trajectory planner, which is part of an ADS. The generic MPC-TP is treated as an SOTIF-EooC, elaborating this concept that is only sketched in the SOTIF standard. In a first step, the initial concept of a generic MPC-TP has been analyzed in order to identify relevant FIs and corresponding TCs. To this end, the analysis techniques that are described in ISO 21448 only in very general terms have to be interpreted and applied to the case study. In a second step, the resulting comprehensive list of relevant FIs and TCs informed an SOTIF-optimized concept of the MPC-TP. The revised safety architecture includes, in particular, an infeasibility analysis as a component and a back-up trajectory planner, a switchable behavior of the MPC-TP to support special driving modes, and adaptations to the reference trajectory.  

An important aspect of the results is the identification of important research topics on MPC for path planning in the context of automated and autonomous driving, from a safety perspective:


  (i) interaction-awareness with other (human) traffic participants;

  (ii) ability to avoid obstacles and drive safely even at low friction values;

  (iii) design of a recursively feasible MPC-based path planner with a reaction horizon;

  (iv) string stability in a chain of vehicles;

  (v) examination of a safe switching logic between a Primary MPC and a Backup MPC. 


In future work, an extensive simulation study would be beneficial for a more detailed understanding of the potential shortcomings of the MPC approach itself, or the respective vehicle model to be used for its predictions in extreme driving situations.



§ ACKNOWLEDGMENT


The research leading to these results was funded by the German Federal Ministry for Economic Affairs and Climate Action (BMWK) through the project EEMotion.










    10cm(4.5cm,2cm)
        

§ APPENDIX A: OVERVIEW OF TCS AND FIS

    
    [c]|p6cm|p6cm|p7cm|
    TC     Triggered FI     Notes



() traffic congestion
    
() no capability to form a rescue alley for police or ambulances() generation of trajectories representing unusual, illegal, counter-intuitive, or dangerous maneuvers
    
(refined specification/design) introduce switchable behavior of the MPC (driving mode: Rescue Alley)
(integration requirement) congestion situations shall be detected and handled by the encompassing system (a,b)



() obstacle avoidance scenario 
    
() no capability to carry out emergency maneuvers without a proper lane change, driving in between two lanes, or by using the shoulder(FI:DangerousManeuver) generation of trajectories representing unusual, illegal, counter-intuitive, or dangerous maneuvers
    
(refined specification/design) introduce switchable behavior of the MPC (driving mode: Obstacle Avoidance)(integration requirement) obstacle avoidance situations shall be detected and handled by the encompassing system (a,b)



() blocking obstacle shortly beyond the prediction horizon N_p of the MPC-TP and no obstacle avoidance is possible 
    
() prediction horizon N_p of the MPC-TP is too short to come to a full stop within it
    
(ODD restriction) limit permissible max. speed to
v_max=|N_p a_lon,min t_s| (Spec:ODDi*)



(a) right-of-way is unclear: at the end of two merging lanes (TC:AOUCS04b) right-of-way is unclear: special situations during general lane changes 
   
(a) overly conservative behavior of the MPC-TP: inability to ensure sufficient progress on the road(FI:AOUCS04b) overly conservative behavior of the MPC-TP: inability to avoid deadlocks in certain situations (FI:DangerousManeuver) generation of trajectories representing unusual, illegal, counter-intuitive, or dangerous maneuvers
    
(Interaction-aware MPC) interaction-aware MPC approach, in coordination with the Tactical Planner (Spec:InteractionAwareMPC*)



() Leading TgtV driving in reverse, in combination with a stopping or slowly moving EgoV 
    
() inability to generate a feasible (collision-free) trajectory in certain situations 
    
(refined specification/design) infeasibility analysis: selection of stopping trajectory in situations with low speed and safety distance violated



() wrong way driver 
    
(FI:Infeasible) inability to generate a feasible (collision-free) trajectory in certain situations (FI:DangerousManeuver) generation of trajectories representing unusual, illegal, counter-intuitive, or dangerous maneuvers
    
(refined specification/design) introduce switchable behavior of the MPC (driving mode: Wrong Way Driver)(integration requirement) if infeasibility still occurs, switch to emergency operation of the encompassing system ()(integration requirement) situations with wrong way drivers shall be detected and handled by dedicated mode of the encompassing system ()



(a) sudden drop of friction coefficient: water (hydroplaning) (TC:AOUCS07b) sudden drop of friction coefficient: snow / ice (TC:AOUCS07c) sudden drop of friction coefficient: oil spillage 
    
(FI:Infeasible) inability to generate a feasible (collision-free) trajectory in certain situations 
   
(ODD restriction) limit permissible minimum temperature to exclude snowy/icy conditions (Spec:ODDf*) 
(ODD restriction) limit permissible minimum friction value (Spec:ODDh*)(integration requirement) conservative estimation of the friction value (a)(refined specification/design) avoid small areas filled with water by evasive maneuvers within the lane boundaries




(TC:AOUCS07a) sudden drop of friction coefficient: water (hydroplaning)(TC:AOUCS07b) sudden drop of friction coefficient: snow, ice (TC:AOUCS07c) sudden drop of friction coefficient: oil spillage 
    
(a) inability to generate a drivable trajectory in certain situations: due to linear tire model in LDBM
    (refined specification/design) infeasibility analysis: temporarily relax the linear tire regime if the friction value drops by more than μ_d within t_d



(a) irregular road surface: gravel or stones(TC:AOUCS08b) irregular road surface: potholes(TC:AOUCS08c) irregular road surface: road bumps
     
 (FI:Drivableb) inability to generate a drivable trajectory in certain situations: due to irregular road surface
    
(integration requirement) situations with irregular road surface shall be detected and handled by the encompassing system in combination with the Tactical Planner ()



(a) side wind gusts: bridge deck(TC:ASA02a) irregular lane markings: suddenly disappearing lane markings(b) irregular lane markings: dirty lane markings(TC:ASA02c) irregular lane markings: broken lane markings(TC:ASA02d) irregular lane markings: covered lane markings 
    
() receding-horizon planning of the MPC-TP in combination with tracking controller could lead to oscillations, or even an unstable behavior of the EgoV 
    
(refined specification/design) MPC-TP to keep the first N_r=4 steps (reaction horizon) of the trajectory unchanged (Spec:TrajectoryBodyg*) 



(TC:AOUCS04) right-of-way is unclear() other traffic participants not strictly adhering to the traffic rules 
    (FI:AOUCS04a) overly conservative behavior of the MPC-TP: inability to ensure sufficient progress on the road(FI:AOUCS04b) overly conservative behavior of the MPC-TP: inability to avoid deadlocks in certain situations(FI:DangerousManeuver) generation of trajectories representing unusual, illegal, counter-intuitive, or dangerous maneuvers
   
(Interaction-aware MPC) interaction-aware MPC approach, in coordination with the Tactical Planner (Spec:InteractionAwareMPC*)



26cm() cut-in of a faster TgtV in front of the EgoV
    
(FI:AOUCS04c) overly conservative behavior of the MPC-TP: overly conservative braking in cut-in scenarios 
    
(refined specification/design) tuning of MPC-TP such that it avoids overly conservative braking to a leading TgtV with a positive relative velocity

2-3
    
(FI:Infeasible) inability to generate a feasible (collision-free) trajectory in certain situations
    
(refined specification/design) infeasibility analysis: if TgtV cuts into the EgoV's lane and is faster than EgoV, then temporarily relax linear minimum distance to this TgtV



() trailing vehicle driving closely behind the EgoV in the same lane, not keeping a proper safety distance 
    () inability of the MPC-TP to consider trailing traffic in the lane of the EgoV, where a sudden braking maneuver of the EgoV may lead to a collision 
   
(refined specification/design) MPC-TP to brake only when strictly necessary and only to an appropriate extent (Spec:SuddenBrakinga*) (refined specification/design) MPC-TP to account for distance to trailing vehicle by means of tuning the weight of ℓ_o(·,·) in the cost function (Spec:SuddenBrakingb*) 



26cm(a) suddenly appearing obstacle: previously shadowed traffic participant(TC:AFD01b) suddenly appearing obstacle: object dropping from a bridge or truck(TC:AFD01c) suddenly appearing obstacle: large animal (e.g., wild boar) crossing the freeway
    
(FI:Infeasible) inability to compute a feasible (collision-free) trajectory in certain situations () insufficient time to solve the OCP, in order to compute a new trajectory
   
(refined specification/design) infeasibility analysis(refined specification/design) potential extension of computation time to multiple time steps based on the concept of a reaction horizon

2-3
   
(FI:DangerousManeuver) generation of trajectories representing unusual, illegal, counter-intuitive, or dangerous maneuvers
    
(ODD restriction) limit ODD to freeway sections with a game fence (Spec:ODDo*) (refined specification/design) introduce switchable behavior of the MPC (driving mode: Defensive Driving)



(a) sudden traffic rule change: start of a speed limit area(TC:AFD02b) sudden traffic rule change: start of a no passing zone 
    
(FI:Infeasible) inability to generate a feasible (collision-free) trajectory in certain situations(FI:InsufficientTime) insufficient time to solve the OCP, in order to compute a new trajectory 
   
(ODD restriction) limit permissible max. speed to
v_max=|N_p a_lon,min t_s| (Spec:ODDi*)(refined specification/design) infeasibility analysis(refined specification/design) potential extension of computation time to multiple time steps based on the concept of a reaction horizon




(a) LDBM assumption violation: asymmetric load(TC:AAOD01b) LDBM assumption violation: dynamic cornering(TC:AAOD01c) LDBM assumption violation: high position of the CoG 
    
()  poor approximation of real vehicle behavior by the LDBM in the OCP
    
(further analysis required) experimental analysis: evaluate if MPC-TP behaves safely in these situations; otherwise consider usage of a more complex model (e.g., a two track model) in the OCP



() chain of vehicles, EgoV in the middle, first vehicle brakes suddenly 
    
() lack of string stability of MPC-TP in a chain of vehicles, potentially leading to collisions
    
(refined specification/design) proper tuning of MPC-TP for string stability, to be verified in simulations and/or on-road tests



() fast approach of a slowly driving TgtV in the front of the EgoV, where the TgtV cannot be avoided by a lateral maneuver

   
(FI:Infeasible) inability to generate a feasible (collision-free) trajectory in certain situations() lack of recursive feasibility 
    
(refined specification/design) infeasibility analysis(refined specification/design) if collision is unavoidable, activation of a collision mitigation mode in the encompassing system



() rapidly changing traffic situations 

   
(FI:InsufficientTime) insufficient time to solve the OCP, in order to compute a new trajectory 
   
(refined specification/design) potential extension of computation time to multiple time steps based on the concept of a reaction horizon



no specific TCs known
   
() inability to generate a solution in certain situations at all 
   
(refined specification/design) infeasibility analysis



Pairs of extreme parameter values in some or all corner cases:() max. side wind speed v_wind,max = 8 bft and max. bank angle β_max = 8^∘ () min. friction value μ_min=0.15 and min. tire cornering stiffness C_α,f,min=60 kN/rad(a) max. front tire cornering stiffness C_α,f,max=200 kN/rad × and min. rear tire cornering stiffness C_α,r,min=60 kN/rad(TC:CommonTrigger3b) min. front tire cornering stiffness C_α,f,min=60 kN/rad × and max. rear tire cornering stiffness C_α,r,max=200 kN/rad...
   
(FI:Drivablea) inability to generate a drivable trajectory in certain situations: due to linear tire model in LDBM(FI:AAOD01) poor approximation of real vehicle behavior by the LDBM in the OCP
    
(further analysis needed) experimental analysis for pairs of boundary values, conduct simulations to confirm/discard relevance of LDBM for MPC-TP; otherwise consider usage of a more complex model (e.g., a two track model) in the OCP



(TC:ASA02a) irregular lane markings: suddenly disappearing lane markings (TC:ASA02b) irregular lane markings: dirty lane markings(TC:ASA02c) irregular lane markings: broken lane markings(TC:ASA02d) irregular lane markings: covered lane markings 
    
    
(further analysis needed) common TCs, conduct simulations to confirm/discard relevance for MPC-TP 



(a) unrecognizable lane markings: direct sunlight (TC:ACTC03b) unrecognizable lane markings: change of lighting conditions, e.g., when entering a tunnel 
    
    
(further analysis needed) common TCs, conduct simulations to confirm/discard relevance for MPC-TP 



(a) insufficient visibility of lane markings: snow / ice  (TC:ACTC04b) insufficient visibility of lane markings: fog 
    
    
(further analysis needed) common TCs, conduct simulations to confirm/discard relevance for MPC-TP 



(a) infrastructure items that could be misinterpreted as lane markings: curbs (TC:ACTC05b) infrastructure items that could be misinterpreted as lane markings: dividers 
    
    
(further analysis needed) common TCs, conduct simulations to confirm/discard relevance for MPC-TP 



(TC:AOUCS08b) irregular road surface: potholes(TC:AOUCS08c) irregular road surface: road bumps 
    
    
(further analysis needed) common TCs, conduct simulations to confirm/discard relevance for MPC-TP 



(a) slippery road surface: wet road with foilage(TC:ACTC09b) slippery road surface: snowy / icy road
    
(FI:Drivablea) inability to generate a drivable trajectory in certain situations: due to linear tire model in LDBM
   
(ODD restriction) limit permissible minimum temperature to exclude snowy/icy conditions (Spec:ODDf*)(refined specification/design) infeasibility analysis: temporarily relax the linear tire regime if the friction value drops by more than μ_d within t_d(further analysis needed) common TCs, conduct simulations to confirm/discard relevance for MPC-TP 












26cm() entering the freeway from an on-ramp () exiting the freeway via an off-ramp () lane adding situations
    
(FI:InsufficientTime) insufficient time to solve the OCP, in order to compute a new trajectory(FI:Infeasible) inability to generate a feasible (collision-free) trajectory in certain situations
    
(refined specification/design) potential extension of computation time to multiple time steps based on the concept of a reaction horizon(refined specification/design) infeasibility analysis

2-3
    
(FI:DangerousManeuver) generation of trajectories representing unusual, illegal, counter-intuitive, or dangerous maneuvers
    
(further analysis needed) conduct simulations and/or real experiments



26cmsituations with special vehicles or vehicles operating beyond traffic rules:(TC:AOUCS06) wrong way driver () approaching emergency vehicle () heavy transport or oversized vehicles () vehicle parking on the freeway shoulder() reckless driver / person driving under-the-influence
    
(FI:DangerousManeuver) generation of trajectories representing unusual, illegal, counter-intuitive, or dangerous maneuvers
    
(refined specification/design) introduce specialized driving modes for particular situations (Spec:SpecialDrivingModes*)(further analysis needed) conduct simulations and/or real experiments

2-3
    
(FI:InsufficientTime) insufficient time to solve the OCP, in order to compute a new trajectory(FI:Infeasible) inability to generate a feasible (collision-free) trajectory in certain situations    
(refined specification/design) potential extension of computation time to multiple time steps based on the concept of a reaction horizon(refined specification/design) infeasibility analysis



26cm(TC:AOUCS08d) irregular road surface: ruts(TC:AOUCS08e) irregular road surface: blow-ups
   
(FI:InsufficientTime) insufficient time to solve the OCP, in order to compute a new trajectory(FI:Infeasible) inability to generate a feasible (collision-free) trajectory in certain situations
    
(refined specification/design) potential extension of computation time to multiple time steps based on the concept of a reaction horizon(refined specification/design) infeasibility analysis


2-3
   
(FI:DangerousManeuver) generation of trajectories representing unusual, illegal, counter-intuitive, or dangerous maneuvers
    
(further analysis) extension of perception required




() freeway shoulder not available () freeway shoulder to be used as additional lane 
   
(FI:DangerousManeuver) generation of trajectories representing unusual, illegal, counter-intuitive, or dangerous maneuvers
   
(refined specification/design) introduce specialized driving modes for particular situations (Spec:SpecialDrivingModes*)(further analysis needed) conduct simulations and/or real experiments




() end of freeway () freeway closure
    
 (FI:DangerousManeuver) generation of trajectories representing unusual, illegal, counter-intuitive, or dangerous maneuvers
   
(further analysis needed) conduct simulations and/or real experiments

() steep road slope () high road curvature 
   
(FI:Drivablea) inability to generate a drivable trajectory in certain situations: due to linear tire model in LDBM(FI:AAOD01) poor approximation of real vehicle behavior by the LDBM in the OCP
   
(further analysis required) experimental analysis: evaluate if MPC-TP behaves safely in these situations; otherwise consider usage of a more complex model (e.g., a two track model) in the OCP



(TC:ASA01) side wind gusts
   
(FI:DangerousManeuver) generation of trajectories representing unusual, illegal, counter-intuitive, or dangerous maneuvers(FI:Infeasible) inability to generate a feasible (collision-free) trajectory in certain situations(FI:InsufficientTime) insufficient time to solve the OCP, in order to compute a new trajectory
    
(no change needed) covered by wind limitation in the ODD (Spec:ODDg)



() potentially unknown traffic scenarios / traffic signs in other European countries
   
(FI:DangerousManeuver) generation of trajectories representing unusual, illegal, counter-intuitive, or dangerous maneuvers(FI:Infeasible) inability to generate a feasible (collision-free) trajectory in certain situations(FI:InsufficientTime) insufficient time to solve the OCP, in order to compute a new trajectory
    
(no change needed) covered by permissible road type restriction in the ODD (Spec:ODDa) 



    
    









 Paper Title: Leveraging PRS and PDSCH for Integrated Sensing and Communication Systems.







Leveraging PRS and PDSCH for Integrated Sensing and Communication Systems
 

This work was funded by the European Commission through the H2020 MSCA 5GSmartFact project under grant agreement number 956670. Also, this work was supported by the project 6G-DISAC under the European Union's Horizon research and innovation program with grant agreement number 101139130.

    Keivan Khosroshahi12, Philippe Sehier2, and Sami Mekki2,

1
Université Paris-Saclay, CNRS, CentraleSupélec, Laboratoire des Signaux et Systèmes, Gif-sur-Yvette, France

2 Nokia Standards, Massy, France

 keivan.khosroshahi@centralesupelec.fr, {philippe.sehier, sami.mekki}@nokia.com

    August 5, 2024
=============================================================================================================================================================================================================================================================================================================================================================================

§ ABSTRACT 
 From the industrial standpoint on integrated sensing and communication (ISAC), the preference lies in augmenting existing infrastructure with sensing services while minimizing network changes and leveraging available resources. This paper investigates the potential of utilizing the existing infrastructure of fifth-generation (5G) new radio (NR) signals as defined by the 3rd generation partnership project (3GPP), particularly focusing on pilot signals for sensing within the ISAC framework. We propose to take advantage of the existing positioning reference signal (PRS) for sensing and the physical downlink shared channel (PDSCH) for communication, both readily available in 5G NR. However, the use of PRS for sensing poses challenges, leading to the appearance of ghost targets.
 To overcome this obstacle, we propose two innovative approaches for different PRS comb sizes within the ISAC framework, leveraging the demodulation reference signal (DMRS) within PDSCH to eliminate ghost targets. Subsequently, we formulate a resource allocation problem between PRS and PDSCH and determine the Pareto optimal point between communication and sensing without ghost targets. Through comprehensive simulation and analysis, we demonstrate that the joint exploitation of DMRS and PRS offers a promising solution for ghost target removal, while effective time and frequency resource allocation enables the achievement of Pareto optimality in ISAC.



PRS – PDSCH – DMRS – ISAC – Ghost target 






§ INTRODUCTION

Sixth-generation (6G) wireless networks aim to revolutionize communication by integrating sensing capabilities. This paradigm shift, known as integrated sensing and communication (ISAC), promises to unlock numerous applications, from enhanced environmental monitoring to real-time object tracking and smart cities, all within a single network <cit.>. The reference signals in fifth-generation (5G) New Radio (NR) have emerged as promising candidates for sensing tasks, owing to their ideal auto-correlation and cross-correlation properties, as well as their robust anti-noise capabilities <cit.>. Furthermore, the utilization of reference signals in ISAC systems offers advantages such as reduced hardware expenses, simplified implementation, and decreased computational complexity <cit.>. Some recent studies have explored the potential of utilizing various reference signals for sensing applications within 5G networks. Authors in <cit.> investigated the use of positioning reference signal (PRS) for range and Doppler estimation, comparing its performance to other 5G pilots such as demodulation reference signal (DMRS), channel state information reference signal (CSI-RS) and synchronization signal (SS). In <cit.>, the authors proposed a two-stage joint optimization scheme for channel estimation, target detection, and pilot selection within an ISAC framework. In <cit.>, a multi-range dual radar and communication system based on a pilot-based orthogonal frequency division multiplexing (OFDM) waveform is presented. CSI-RS and DMRS are used and compared in <cit.> for range and velocity estimation.

Among the available pilot signals, PRS, introduced in 3rd generation partnership project (3GPP) Release 16, holds significant promise for sensing applications due to its rich time-frequency resources and flexible configuration. PRS has high resource element (RE) density and superior correlation properties compared to existing reference signals because of the staggered or diagonal RE pattern. However, PRS is not particularly designed for sensing, and its exploitation in sensing passive targets poses some challenges that cannot be ignored. More precisely, empty REs in the structure of PRS would lead to ambiguity in range estimation. Such ambiguity makes the distinction between real and ghost targets quite challenging. While the proposed solution in <cit.> has focused on multiple reference signals to address range ambiguity, it is not directly applicable in ISAC scenarios due to interference between physical downlink shared channel (PDSCH) for data transmission and other reference signals such as PRS and CSI-RS. Additionally, with the proposed configuration, there will be no flexibility in choosing the comb size. Furthermore, the approach outlined in <cit.> results in a notable decrease in the maximum detection range, especially by using high PRS comb sizes. Thus, to the best of our knowledge, a 3GPP-compliant solution to alleviate range ambiguity in the ISAC framework has not been proposed.



In this paper, we introduce two novel methods exploiting available reference signals in the 5G NR to provide sensing services for passive targets with enhanced maximum detection range without ambiguity. To enable sensing services in addition to communication, we propose an ISAC OFDM resource grid incorporating PRS for sensing and PDSCH for data transmission. To eliminate the range ambiguity, we propose two methods for different comb sizes to repurpose DMRS in PDSCH for sensing besides communication. Moreover, since sensing and communication symbols cannot share the same REs in the resource grid, we propose time and frequency resource allocation between PDSCH and PRS. Elaborating on this, we introduce and formulate a resource allocation problem between PDSCH and PRS to achieve Pareto optimality between sensing and communication without range ambiguity. Finally, through comprehensive simulations, we demonstrate the effectiveness of our methods in mitigating range ambiguity and identify the Pareto optimal point between communication and sensing.







§ 5G REFERENCE SIGNALS

Based on TS 38.214 <cit.>, the 5G-NR waveform is built on a time-frequency resource grid with the smallest time-frequency resource that can be allocated, known as physical resource block (PRB). Each PRB comprises 12 contiguous sub-carriers and 14 consecutive symbols. In the following two sections, we describe two of the 5G reference signals used in this work namely, PRS and DMRS.





 §.§ PRS

The PRS is generated as explained in TS 38.211 <cit.> as:
 


    r(m) = 1/√(2)(1 - 2c(2m)) + j1/√(2)(1 - 2c(2m + 1)),

where c(i) is a Gold sequence of length-31. Generating PRS from the Gold sequence leads to suitable auto-correlation and anti-noise characteristics <cit.>. The initial value of c(i) for PRS can be found in TS 38.211 <cit.>.
Depending on the subcarrier spacing, the PRS in 5G NR can occupy a minimum of 24 PRBs and a maximum of 272 PRBs. PRS offers a versatile configuration of time-frequency resources to meet sensing accuracy requirements across various application scenarios. In accordance with the PRS resource mapping outlined in TS 38.211 <cit.>, PRS supports four comb patterns—Comb 2/4/6/12—in the frequency domain and five symbol number configurations—Symbol 1/2/4/6/12—in the time domain. 

Figure  <ref> shows two examples of comb size patterns within a slot with PRS structure of 12 symbols. 





 §.§ DMRS

The DMRS is also generated using (<ref>), and the initial value of c(i) for DMRS is available in <cit.>.
DMRS is generated within the designated PDSCH allocation, as outlined in TS 38.211 <cit.>. DMRS is used for channel estimation and is confined to the resource blocks (RBs) assigned for PDSCH. The structure of DMRS is designed to accommodate various deployment scenarios and use cases. The position of the DMRS symbols in the time and frequency resource grid depends on the mapping type, which can be either slot-wise (Type A) or non-slot-wise (Type B). The positions of any additional DMRS symbols are determined by a set of tables, as detailed in TS 38.211 <cit.>. Additionally, 1 to 4 OFDM symbols can be occupied by front-load DMRS in the time domain. DMRS exists within physical channels allocated for communication receivers and is transmitted using the same beam specifically chosen for communication receivers. However, we aim to reuse it for sensing purposes beside communication.





§ ISAC SYSTEM MODEL

The current 5G NR signals were not originally designed to support radar sensing applications. In this section, we present a methodology for effectively utilizing both communication and sensing signals within the ISAC framework. We envision a downlink ISAC system wherein a gNB transmits signals toward a sensing area containing K point-like targets. The echoes from these targets are then captured by a receiver for range and velocity estimation. We assume the targets are in the line-of-sight (LoS) of the transmitter and the receiver. 
An example of the envisioned scenario is depicted in Fig. <ref>. 




The continuous time domain expression of the transmitted signal can be expressed in the following form: <cit.>
 


    s(t) = ∑_n = 0^N-1rect(t - nT_0/T_0)∑_m=0^M-1 v(m,n) e^j2π mΔ f(t - nT_0),

where we consider an OFDM resource grid containing N symbols in the time domain and M sub-carriers, and rect(t/T_0) represents the rectangular pulse, Δ f denotes the subcarrier spacing, and T_0 = T_CP + T_s designates the total duration of the OFDM symbol, where T_s = 1/Δ f denotes the symbol duration, and T_CP is the cyclic prefix (CP) length. v(m,n) is the complex symbol transmitted at n-th OFDM symbol and m-th sub-carrier, where n=0,...,N-1 and m=0,...,M-1 within an M × N OFDM resource grid. Some of these symbols can be set to zero or not allocated.

The reflected signals received by the receiver can be expressed as <cit.>
 


    r(t)=∑_k=1^Kβ_k s(t-τ_k)e^j2π f_d,kt + u(t),

where β_k represents the attenuation factor of the k-th target, f_d,k is the Doppler frequency shift originated from the k-th target, τ_k is the target k delay and u(t) ∈ℂ is the complex additive white Gaussian noise (AWGN) with zero mean and variance of 2σ^2.


Conventional OFDM receivers sample the signal at each symbol time and perform an fast fourier transform (FFT) to extract the modulated symbols. The delay and Doppler are compensated in normal conditions, and there is no inter-symbol interference. The cyclic prefix is usually exploited to remove receive timing errors and the multi-path spread. In our case, it cannot be guaranteed that the path delay and Doppler are in the usual range due to the potentially large uncertainties on the position and speed of the target. Therefore, there can be inter-symbol interference, both in the time and frequency domains. Therefore, the received samples extracted after FFT can be written as 




    y(m,n) =   ∑_k=1^Kβ_k e^j2π n T_0 f_d,k e^-j2π m Δ f τ_k v(m,n) + ISI
       + q(m,n),

where ISI indicates the inter-symbol interference term caused by Doppler and delay, q(m,n) ∈ℂ is the AWGN noise with zero mean and variance of 2σ^2 on the m-th subcarrier and the n-th OFDM symbol obtained from sampling and FFT over r(t).




§ ESTIMATION OF THE RANGE AND VELOCITY 

It is a well-known result that received samples undergo rotations on the time axis due to Doppler and on the frequency axis due to time shift. Estimating range and Doppler requires a two-dimensional search, for which different strategies can be considered. Several methods are described in <cit.>. We limit ourselves here to the method that requires the least computing power, which operates in 2 stages: first, the range is estimated, then the Doppler, after compensating for the delay. The time-of-flight (ToF) from the transmitter to the target and then to the receiver can be extracted from range evaluation using the periodogram <cit.>. To that end, we remove the transmitted sensing symbols from the received echos by point-wise division as follows



    g(m,n) =   y(m,n)/v(m,n),     If  v(m,n) ≠ 0,
    
        g(m,n) =    0,     If  v(m,n) = 0.


Then the M-point inverse fast fourier transform (IFFT) of the PRS on the n-th column of the g(m,n) is calculated:




    R_n(l) =    |IFFT(g(m,n))| = |∑_k=1^K (β_k e^j2π n T_0 f_d,k∑_m=0^M-1
        e^-j2π m Δ f  R^tot_k/c e^j2πml/M+ q'(m,n)/v(m,n)e^j2πml/M)|,

where l= 0,...,M-1, |.| is the absolute value and q'(m,n) = q(m,n) + ISI. The reason for taking the absolute value is to reduce Doppler sensitivity. We replaced τ_k = R^tot_k/c while R^tot_k is the bistatic range, i.e., distance from gNB to the target and then to the receiver, c is the light speed. When the arguments of e^-j2π m Δ f  R^tot_k/c e^j2πml/M in (<ref>) cancel each other, the maximum value occurs. The next step consists in summing the R_n(l) over N symbols to increase the signal-to-noise ratio (SNR):



    R(l) = 1/N∑_n=0^N-1 R_n(l).


The maximum value named l̂_k in (<ref>) gives the bistatic distance of each target <cit.>: 



    R̂^tot_k = l̂_kc/Δ f M.


The range resolution can be calculated as follows



    Δ R = c/Δ f M,

and the maximum detection range can be written as follows



    R_max = c M/Δ f M = c/Δ f.


Based on <cit.>, we can improve the accuracy performance by increasing the number of IFFT points m_a as follows



    R_n(l) =    |IFFT(g(m,n))| = |∑_k=1^K (β_k e^j2π n T_0 f_d,k∑_m=0^m_aM-1
        e^-j2π m Δ f  R^tot_k/c e^j2πml/m_aM + q'(m,n)/v(m,n)e^j2πml/m_aM)|.

In this case

    R̂^tot_k = l̂_kc/m_aΔ f M.


As pointed out in <cit.>, by increasing m_a, we can improve the range accuracy at the cost of increased computational complexity. 

After compensating the delay, Doppler can be estimated as follows. We perform N-points FFT on the m-th row of the g(m,n)



    v_m(d) =    |FFT(g(m,n))| = |∑_k=1^K (β_k ∑_n=0^N-1 e^j2π n T_0 f_d,k e^-j2πnd/N
        + q'(m,n)/v(m,n)e^-j2πnd/N)|,

where d= 0,...,N-1. Then, we perform FFT over all rows of the g(m,n) and averaging over them as follows



    v(d) = 1/M∑_m=0^M-1 v_m(d).


Next, we find the index of the maximum value, i.e., d̂_k for target k in (<ref>), and the doppler can be estimated as 



    f̂_d,k = d̂_k/T_s N.


The velocity can be obtained from Doppler as follows <cit.> 


    v = cf_d,k/2 f_c,

 where f_c is the carrier frequency. Hence, velocity can be estimated as
 
    v̂ = d̂_kc/2T_s f_c N.


The velocity resolution can be written as 

    Δv̂ = c/2T_s f_c N,

and the maximum detectable velocity estimation is 

    v̂_max = c/2T_s f_c.


We can improve the accuracy of velocity by increasing the FFT point similar to range estimation. 



 §.§ Ghost Targets in Range Estimation

PRS was originally designed for positioning. In this operating mode, time-frequency uncertainties are reduced thanks to the time-frequency servo loop. This is not the case for sensing, as the uncertainty ranges are potentially larger, and larger auto-correlation ranges of PRS need to be considered. 
PRS exhibits a recurring pattern according to the selected Comb value, 

as illustrated in Fig. <ref> in which a comb size of 12 is configured. 



Ghost effects make it challenging to distinguish between real and false targets, as depicted in Fig. <ref> and Fig. <ref> in our simulation results. Table <ref> presents the periodic distances between ghost targets, derived from the two-way propagation delay based on Eq. (<ref>), where SCS denotes the subcarrier spacing. Notably, for one-way propagation delay, ambiguity intervals are twice the values indicated in Table <ref>.



    Δ R_ambiguity = c/2 K_comb^PRSΔ f,

where K_comb is the PRS comb size. Therefore, to estimate the range of the targets without encountering ambiguity, a practical approach that aligns with 3GPP standards is imperative.














§ USING DMRS TO REMOVE GHOST TARGETS

The main idea involves leveraging DMRS as a reference signal available in PDSCH for ISAC use cases, where an OFDM frame contains PRS for sensing and PDSCH for communication. DMRS serves to mitigate ambiguity in range estimation and sensing. To achieve this, we generate an OFDM frame and allocate different time/frequency resources based on the application requirements for communication and sensing. Fig. <ref> illustrates an example of such resource allocation in the time domain between PRS and PDSCH. DMRS not only serves communication purposes such as channel estimation but can also contribute to sensing by eliminating ghost targets without compromising accuracy and without necessitating changes to gNB configuration, as it aligns with 3GPP standards. To tackle this issue, we propose two novel algorithms for different comb sizes by allocating similar bandwidth to PDSCH and PRS. 









 §.§ Algorithm 1: When Comb size is equal to 2 or 4

Initially, we extract DMRS from the received slots containing PDSCH. Then, we use the range estimation algorithm explained in <ref> for DMRS and PRS independently. If we obtain r from PRS and DMRS using (<ref>) with the same IFFT size and name them r_PRS and r_DMRS, respectively, instead of finding the maximum value for each of them separately, we find the index of the maximum value from r_PRS,DMRS =  r_PRS∘ r_DMRS, where ∘ is the Hadamard product. This method maintains the range resolution and maximum unambiguous range as derived in (<ref>) and (<ref>), respectively. 
In practice, ghost targets resulting from range estimation using DMRS may appear in positions where PRS with comb sizes of 2 and 4 do not show any ghost targets, and vice versa. By performing element-wise multiplication of r_PRS and r_DMRS, we effectively remove the ghost targets, enabling clear distinction between real targets and ghost ones. If PRS and PDSCH occupy different bandwidths, we can adjust the lengths of the vectors by changing the size of the IFFT points, resulting in similar vector sizes for r_PRS and r_DMRS.



 §.§ Algorithm 2: When Comb size is equal to 6 or 12

In the case that the comb size is equal to 6 or 12, some ghost targets of PRS and DMRS overlap by using algorithm 1. In this case, from the extracted PRS and DMRS of the received PDSCH slots, we can obtain g_DMRS(m,n) and g_PRS(m,n), then add all of their columns as follows



    g_PRS^tot(m)    = ∑_n =1^N-1 g_PRS(m,n),
    
        g_DMRS^tot(m)    = ∑_n =1^N'-1 g_DMRS(m,n),

where N' is the number of DMRS symbols in the time domain. Then, we normalize them to balance their effect in IFFT.



    g_PRS^tot(m)    = g_PRS^tot(m)/max_m{g_PRS^tot(m)},
    g_DMRS^tot(m)    = g_DMRS^tot(m)/max_m{g_DMRS^tot(m)}.


Afterwards, we perform M-point IFFT on the summation of g_PRS^tot(m) and g_DMRS^tot(m) as follows



    r(l) = |IFFT(g_PRS^tot(m) + g_DMRS^tot(m))|.


Eventually, we can find the index of the maximum value of r(l) to estimate the bistatic range of targets. The rest of the parameters, i.e., bistatic distance, range resolution, and the maximum detection range can be calculated using (<ref>), (<ref>), and (<ref>), respectively.
In this way, we are actually disturbing the periodicity of the pattern of PRS and DMRS, which leads to suppression of the magnitude of the fake peaks. Therefore, we can easily differentiate between real and fake targets. 
If PRS and PDSCH have different bandwidths, we pad zeros to the reference signal with the smaller bandwidth so eventually, g_PRS^tot(m) and g_DMRS^tot(m) can have similar vector sizes.



§ RESOURCE ALLOCATION TO REACH PARETO OPTIMALITY IN ISAC

In this section, we define an optimization problem for resource allocation between PDSCH and PRS to find the Pareto optimal point between communication and sensing without ghost targets through the two proposed algorithms. To that end, we define the maximum throughput of the received PDSCH per one slot and one PRB as R_0, c/Δ f as the range resolution metric, and c/2T_s f_c as the velocity resolution metric. We introduce the following resource allocation problem to find the Pareto optimal point. 





    m_0,n_0,...,m_K,n_K max   α_0 m_0 n_0/R_max R_0 -  ∑_k=1 ^K( γ_k,1/d_max m_kc/Δ f
       + γ_k,2/v_max n_kc/2T_s f_c ),  
    s.t.    m_i ∈{1,...,M_max-1},  i = 0,...,K, 
         n_i ∈{1,...,N_max-1} , i= 0,...,K,  
       ∑_i=0^K m_i = M_max, 
       ∑_i=0^K n_i = N_max,


where α_0, γ_k,1, and γ_k,2 are the communication weight, the range and velocity resolution weights of the target k, respectively, which can be chosen based on the priorities of the application and α_0 + ∑_k=1^K (γ_k,1 +γ_k,2) = 1. R_max, d_max, and v_max are the maximum throughput, range, and velocity resolution that can be achieved by allocating all the available resources in the time, i.e., N_max, and frequency, i.e., M_max, domain for communication data transmission, range and velocity estimation, respectively. m_0 and n_0 are the number of PRBs and slots, respectively, allocated for communication, and m_k and n_k are the number of PRBs and slots, respectively, allocated for the range and velocity estimation of the target k. By solving the introduced optimization problem, we can find the Pareto optimal point of resource allocation between communication and sensing.

















§ SIMULATION RESULTS

To prove our method, we utilize Matlab 5G toolboxes, making this work compatible with 3GPP standards. We simulated a scenario in which two targets with the bistatic distance of 1057m and 1242m and the velocity of 5m/s for both are considered. SNR should be enough to be able to detect PRS and DMRS, f_c=28GHz and SCS equals to 120kHz. PRS number of symbols is 12, DMRS configuration type is 2, there are 3 DMRS symbols placed at {3,8,12} symbols of each slot, PDSCH mapping type is "A", the code rate used to calculate transport block sizes is 490/1024 based on <cit.>. Virtual resource block (VRB) bundle size is 4, and VRB to PRB interleaving is disabled. β_1 =β_2=1 and we consider the 16 quadrature amplitude modulation (16-QAM) modulation for PDSCH. 
In Fig. <ref>, we can see the presence of multiple ghost targets for range estimation of passive targets when using only PRS with the comb size equal to 4. However, in Fig. <ref>, the combination of PRS and DMRS can eliminate the range ambiguity, as explained in algorithm 1. It is worth mentioning that Fig. <ref> and <ref> are normalized. Figures <ref> and <ref> show the ability of algorithm 2 to remove the ghost targets when the PRS comb size is equal to 12.
















Once the ghost targets are removed, we can obtain the Pareto optimal point from the introduced resource allocation optimization, as illustrated in Fig. <ref>, where F is the objective function of the optimization in (<ref>). In this simulation, we considered the maximum available PRBs and slots in 5G NR for PRS, which are 272 and 80, respectively. We also set K=1 and α_0 = 2, γ_1,1 = γ_1,2 = 1. As depicted in Fig. <ref>, with such configurations and weights, the Pareto optimal point is achieved by allocating 6 slots and 12 PRBs for PRS and the rest to PDSCH.








§ CONCLUSION

In this work, we tackled the ghost targets issue of PRS in the ISAC framework by leveraging DMRS available in PDSCH for passive target localization. We proposed two methods for different comb sizes, fully compliant with 3GPP standards and readily implementable in current 5G networks, distinguishing our approach from prior works. Moreover, our solution facilitates sensing without introducing additional overhead, necessitating changes in gNB configurations, or deteriorating positioning accuracy while the maximum detectable range is enhanced. Finally, we introduced a resource allocation problem between PDSCH and PRS to determine the Pareto optimal point between communication and sensing while effectively eliminating ghost targets using the proposed methods.






IEEEtran




 Paper Title: Low-Coherence Sequence Design Under PAPR Constraints.






Low-Coherence Sequence Design 
 Under 
PAPR Constraints

    Gangle Sun, Student Member, IEEE, Wenjin Wang, Member, IEEE, 
 Wei Xu, Senior Member, IEEE, 
and Christoph Studer, Senior Member, IEEE
Gangle Sun, Wenjin Wang, and Wei Xu are with the National Mobile Communications Research Laboratory, Southeast University, Nanjing 210096, China, and also with the Purple Mountain Laboratories,
Nanjing 211100, China (e-mail: sungangle@seu.edu.cn; wangwj@seu.edu.cn; wxu@seu.edu.cn). 
Christoph Studer is with the Department of Information Technology and Electrical Engineering, ETH Zurich,  Switzerland (e-mail: studer@ethz.ch). 


    August 5, 2024
==========================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================

§ ABSTRACT 
Low-coherence sequences with low peak-to-average power ratio (PAPR) are crucial for multi-carrier wireless communication systems and are used for pilots, spreading sequences, and so on.

This letter proposes an efficient low-coherence sequence design algorithm (LOCEDA) that can generate any number of sequences of any length that satisfy user-defined PAPR constraints while supporting flexible subcarrier assignments in orthogonal frequency-division multiple access (OFDMA) systems.

We first visualize the low-coherence sequence design problem under PAPR constraints as resolving collisions between hyperspheres. 

By iteratively adjusting the radii and positions of these hyperspheres, we effectively generate low-coherence sequences that strictly satisfy the imposed PAPR constraints.

Simulation results (i) confirm that LOCEDA outperforms existing methods, (ii) demonstrate its flexibility, and (iii) highlight its potential for various application scenarios.



low-coherence sequences, OFDMA, PAPR.






§ INTRODUCTION

Low-coherence sequences are extensively used in wireless communications and signal processing, such as non-orthogonal multiple access (NOMA), multiple-input multiple-output (MIMO) systems, and massive machine-type communication (mMTC) <cit.>. 

Moreover, with the widespread adoption of multi-carrier systems, such as orthogonal frequency division multiplexing (OFDM)-based systems, it is crucial to ensure that the time-domain signals adhere to a given peak-to-average power ratio (PAPR) constraint in order to (i) mitigate nonlinear distortion and (ii) improve the power efficiency <cit.>. 

The design of low-coherence sequences with low PAPR is, therefore, vital for improving the performance of wireless communication systems.

Recent research in <cit.> has focused on designing non-orthogonal sequences. 

Reference <cit.> utilized unimodular masking sequences on the columns of the discrete Fourier transform (DFT) matrix to generate low-coherence sequences. 

Reference <cit.> developed an iterative collision-based packing algorithm for generating low-coherence sequences.

Reference <cit.> used sequential iterative decorrelation to construct low-coherence sequences approaching the Welch lower bound <cit.>. 

Reference <cit.> utilizes Zadoff-Chu (ZC) and power-residue sequences to construct low-coherence sequences with fixed length; reference <cit.> explored the use of m-sequences and Sidel'nikov sequences for more flexible sequence lengths. 

To improve channel estimation performance, reference <cit.> constructs non-orthogonal sequences, aiming at minimizing their average coherence.

References <cit.> and <cit.> design non-orthogonal sequences to enable fast, non-iterative processing and facilitate collision detection for massive random access, respectively. 

There are several efforts focusing on designing low-coherence sequences with low PAPR <cit.>.

Based on <cit.>, reference <cit.> constructs low-coherence sequences under frequency-domain PAPR constraints.

Reference <cit.> utilizes Golay complementary sequences to create 3× 2^m low-coherence sequences of the length of 2^m, ensuring a maximum PAPR of no more than 4. 

Building on this, reference <cit.> develops 4× 2^m binary Golay spreading sequences of the same length and a PAPR not exceeding 2. 

Additionally, reference <cit.> constructs L× 2^m binary Golay complementary sequences of length 2^m, where the maximum PAPR is 3 decibels (dB).

Reference <cit.> innovates with a two-stage genetic algorithm to design non-orthogonal sequences that achieve both low coherence and low PAPR. 

Using extended Boolean functions, reference <cit.> designs p^m+1 or 2× p^m+1 (p≥ 3) sequences of length p^m, achieving a maximum coherence of p^-m/2 or p^-m-1/2, respectively, and a PAPR of no more than p.

Reference <cit.> designed low-coherence sequences with low PAPR to enable fast collision detection for massive random access.

Most existing work on low-coherence and low-PAPR sequence design is limited by fixed lengths, number of sequences, and specific PAPR constraints. 

Moreover, these sequences are typically tailored to OFDM systems and may exhibit higher PAPR when applied to only a subset of subcarriers.

In contrast to the above work, we propose an efficient algorithm that generates any number of low-coherence sequences with adjustable length. 

These designed sequences support flexible subcarrier assignments in orthogonal frequency-division multiple access (OFDMA) systems and can meet user-defined PAPR constraints.



§ SIGNAL MODEL AND PROBLEM FORMULATION

We now introduce the signal model and formulate the sequence-design problem.





 §.§ Signal Model


We first define the sequence set 𝒫≜{𝐩_n}_∀ n∈𝒩 with 𝐩_n∈ℂ^L and 𝐩_n=1,  ∀ n∈𝒩.

Here, 𝒩≜{1,…, N} is the sequence index set, L denotes the sequence length, and N (with N > L) signifies the cardinality of the set 𝒫, also referred as to the number of sequences. 

In OFDMA systems, L subcarriers out of a total N_C are assigned for sequences 𝒫 and the index vector of these L subcarriers is given by 𝐜∈ℕ_+^L. 

Assuming that the lth entry of sequences correspond to the 𝐜(l)th subcarrier, the time-domain signal corresponding to the nth sequence 𝐩_n can be written as

    x_n(t) = 1/√(N_C)∑_l = 1^L 𝐩_n(l) e^2π j 𝐜(l) t,   t∈ [0,1).

Then, the PAPR corresponding to the nth sequence 𝐩_n, denoted as PAPR(𝐩_n), is defined as

    PAPR(𝐩_n) ≜max_t∈ [0,1)|x_n(t)|^2/1/N_C≈ L max_s∈𝒮 |𝐰_s^H𝐩_n|^2,

where 𝒮≜{1,…, N_S} is the index set of time-domain discrete sampled signals and N_S (with N_S≥ N_C) denotes the number of time-domain discrete sampled signals[The larger N_S is, the more accurate the approximation in (<ref>) will be.]  during the time interval [0,1). In addition, 𝐰_s(l)≜1/√(L)exp(-2π j 𝐜(l)(s-1)/N_S) with 𝐰_s=1, ∀ s∈𝒮.

In addition, the mutual coherence (also called coherence) of sequences in the set 𝒫 is defined as <cit.> 

    μ(𝒫)
        
        =max_m≠ n, ∀ m,n∈𝒩|𝐩_m^H𝐩_n|.




 §.§ Problem Formulation


Our goal is to design the sequence set 𝒫 that minimizes the mutual coherence μ(𝒫) <cit.>. 

Simultaneously, to mitigate nonlinear distortion and improve power efficiency, we would like to manage the time-domain signal within the linear operating range of the amplifier, which can be achieved by ensuring that their PAPR does not exceed a specific threshold[The value of the PAPR threshold depends on the power amplifier.], denoted as Γ_PAPR.

Given these requirements, the sequence-design problem can be formulated as follows: 

    
    𝒬_1:     𝒫^⋆ = 𝐩_n∈ℂ^L,∀ n∈𝒩min  μ(𝒫)
    s.t.     PAPR(𝐩_n)≤Γ_PAPR,  ∀ n∈𝒩, 
       𝐩_n=1, ∀ n∈𝒩.

    


Remark 1: In multi-user shared access (MUSA), the designed sequences serve as spreading sequences for different users <cit.>; 

In downlink MIMO systems, these sequences are utilized as frequency-domain broadcast pilots for downlink channel estimation <cit.>; 

In mMTC, these sequences act as frequency-domain pilots of sporadically active users <cit.>.



§ SEQUENCE DESIGN METHOD


Problem 𝒬_1 is framed as a minimax optimization problem with a quartic objective function and numerous quadratic constraints. 

We now present a transformation strategy for problem 𝒬_1 and introduce an efficient and intuitive optimization method to solve problem 𝒬_1 approximately.




 §.§ Problem Transformation

To tackle the minimax problem 𝒬_1, we first plug (<ref>) and (<ref>) into 𝒬_1 and equivalently transform problem 𝒬_1 as follows:

    
    𝒬_2:     {𝒫^⋆,Γ_mut^⋆} = 𝐩_n∈ℂ^L,∀ n∈𝒩, Γ_mut>0minΓ_mut
    s.t.     |𝐩_m^H𝐩_n|≤Γ_mut,   m n,∀ m, n∈𝒩, 
       |𝐰_s^H𝐩_n|≤√(Γ_PAPR/L), ∀ n∈𝒩, ∀ s∈𝒮,
       𝐩_n=1, ∀ n∈𝒩.


Here, Γ_mut is an auxiliary variable that minimizes the mutual coherence. 

To solve problem 𝒬_2 effectively, inspired by <cit.>, we fix Γ_mut to optimize the sequences 𝒫, and then iteratively adjust Γ_mut to ultimately find the sequences 𝒫 that achieve the lowest Γ_mut.

To find a feasible solution 𝒫 for a given Γ_mut in problem 𝒬_2, we optimize these sequences individually, keeping the other sequences fixed during each sequence's optimization process.

Taking the optimization of the nth sequence 𝐩_n as an example, with the auxiliary variable Γ_mut and the other sequences {𝐩_m}_m  n, ∀ m ∈𝒩 fixed, we can reformulate problem 𝒬_2 as follows:

    
    𝒬_3:     𝐩_n^⋆ = 𝐩_n∈ℂ^Lmin  Γ_mut
    s.t.     |𝐩_m^H𝐩_n|≤Γ_mut,  m n,∀ m∈𝒩, 
        |𝐰_s^H𝐩_n|≤√(Γ_PAPR/L), ∀ s∈𝒮,
       𝐩_n=1.


In the following, we will introduce a novel approach to solve 𝒬_3 approximately and the details of adjusting Γ_mut.



 §.§ LOCEDA: LOw-Coherence sEquence Design Algorithm

Given the unit-norm vectors {𝐰_s }_∀ s and sequence vectors {𝐩_n}_∀ n, we can visualize them as points situated on the surface of a unit-radius hypersphere  <cit.>, as illustrated in Fig. <ref>. 

To showcase the constraints (<ref>) and (<ref>), we construct hyperspheres centered on {𝐩_n}_∀ n with a radius R_seq and on {𝐰_s}_∀ s with a radius R_PAPR. 

We refer to the hypersphere centered on 𝐩_n as the nth sequence hypersphere and the one centered on 𝐰_s as the sth PAPR hypersphere. 

Collisions between sequence hyperspheres indicate violations of constraints (<ref>), while collisions between sequence and PAPR hyperspheres signify violations of constraints (<ref>).

As such, we redefine the goal of finding a feasible solution to problem 𝒬_3 as resolving collisions between these hyperspheres.

We initialize Γ_mut with the Welch lower bound of sequences without PAPR constraints <cit.>, Γ_bound=√(N-L/L(N-1)), and start with random sequences 𝒫 with 𝐩_n=1,∀ n∈𝒩. 

The specific steps to resolve collisions are as follows:






  §.§.§ Sequence-Sequence Hypersphere Collision Resolution

We define the sequence-sequence position vector 𝐝_n,m, representing the displacement from 𝐩_m to 𝐩_n, as follows <cit.>:

    𝐝_n,m≜𝐩_n - ⟨𝐩_n,𝐩_m⟩/|⟨𝐩_n,𝐩_m⟩|𝐩_m,

where ⟨𝐩_n,𝐩_m⟩/|⟨𝐩_n,𝐩_m⟩|=min_ϕ∈ℂ,|ϕ|=1𝐩_n-ϕ𝐩_m.

A collision between the nth and mth sequence hyperspheres occurs if 

    𝐝_n,m < 2R_seq,

where the sequence hypersphere radius R_seq is defined as

    R_seq≜√(0.5(1-Γ_mut)).

Such a sequence-sequence hypersphere collision means that the constraint |𝐩_m^H𝐩_n|≤Γ_mut in 𝒬_3 is not satisfied. 

When sequence-sequence hypersphere collisions happen, the displacement vector 𝐮_n of the nth sequence vector 𝐩_n is given by <cit.>

    𝐮_n =    ∑_m∈𝒞_n(2R_seq-𝐝_n,m)𝐝_n,m/𝐝_n,m,

where 𝒞_n is the index set of sequence hyperspheres that collide with the nth sequence hypersphere. 

We then update 𝐩_n through

    𝐩_n←𝐩_n + τ_seq𝐮_n/𝐩_n + τ_seq𝐮_n

with τ_seq being the step size to resolve the sequence-sequence hypersphere collisions. 

In this stage, we displace all sequence vectors {𝐩_n}_∀ n in parallel for K rounds.




  §.§.§ Sequence-PAPR Hypersphere Collision Resolution
 
Similarly, we define the sequence-PAPR position vector 𝐝̅_n,s from 𝐰_s to 𝐩_n and the radius of PAPR hyperspheres as follows 

    𝐝̅_n,s≜𝐩_n - ⟨𝐩_n,𝐰_s⟩/|⟨𝐩_n,𝐰_s⟩|𝐰_s,
    
        R_PAPR≜√(2(1-√(Γ_PAPR/L)))-√(0.5(1-Γ_mut)).

A collision between the nth sequence hypersphere and the sth PAPR hypersphere occurs if 

    𝐝̅_n,s < R_seq + R_PAPR,

The collision between the nth sequence hypersphere and the sth PAPR hypersphere means the constraint |𝐰_s^H𝐩_n|≤√(Γ_PAPR/L) in 𝒬_3 is not satisfied. 

To resolve these sequence-PAPR hypersphere collisions, the displacement vector 𝐮̅_n of the nth sequence 𝐩_n is expressed as

    𝐮̅_n = ∑_s∈𝒞̅_n(R_seq + R_PAPR-𝐝̅_n,s)𝐝̅_n,s/𝐝̅_n,s,

where 𝒞̅_n is the index set of PAPR hyperspheres that collide with the nth sequence hypersphere. 

Then, the sequence vector 𝐩_n will be updated as follows 

    𝐩_n←𝐩_n + τ_PAPR𝐮̅_n/𝐮̅_n/𝐩_n + τ_PAPR𝐮̅_n/𝐮̅_n,

where τ_PAPR is the step size for resolving the sequence-PAPR hypersphere collisions. 

Unlike the update process in (<ref>), we introduce the normalization factor 𝐮̅_n in (<ref>) to control the magnitude of 𝐮̅_n, preventing instability issue caused by an excessively large norm of 𝐮̅_n, especially when N_S is large. 

In this stage, we displace sequences 𝒫 in parallel through (<ref>) until all sequences satisfy the PAPR constraints (<ref>).




  §.§.§ Parameter Updates

After resolving collisions through steps 1) and 2), we obtain the sequences 𝒫 with its mutual coherence μ(𝒫). 

Since the radii R_seq and R_PAPR and the auxiliary variable Γ_mut are interdependent due to (<ref>) and (<ref>), a change in one will correspondingly affect the others. 

Therefore, we only provide the update strategy for R_seq as follows:

    
  (i) If μ(𝒫)< μ_min, where μ_min is the lowest mutual coherence of sequences from all previous iterations, this indicates that the current sequence set 𝒫 is the best solution found so far.  Greedily, we increase R_seq slightly to accelerate the exploration.
    
  (ii) If μ_min≤μ(𝒫) ≤Γ_mut, the current sequences meet constraints in problem 𝒬_3 but do not improve upon the best solution found. To avoid local optima, we significantly increase R_seq to move beyond the current region and explore better solutions.
    
  (iii) If μ(𝒫)>Γ_mut, no feasible solution has been found under the current R_seq, implying R_seq may be too high. We reduce R_seq to find potential superior solutions.

Specifically, we update the radius R_seq as follows: 

    R_seq←{[ min{R_seq+γ,  R_bound},   if μ(𝒫)< μ_min;           R_bound,  if μ_min≤μ(𝒫) ≤Γ_mut;                 R_seq-γ,  if μ(𝒫)>Γ_mut, ].

where R_bound=√(0.5(1-Γ_bound)) and γ is the step size to update radius R_seq.

To improve the optimization procedure, we dynamically adjust the step sizes τ_seq and τ_PAPR every K_1 iterations, based on the trends in lowest mutual coherence μ_min. 

Specifically, we increase the step sizes to improve empirical convergence when the lowest mutual coherence decreases compared with the one before previous K_1 iterations. 

Conversely, we decrease the step sizes to allow for finer optimization. The step sizes are updated as follows:



    τ_seq   ←(1-ρ+2ρ 𝕀{μ_min<μ̅_min})τ_seq,
    τ_PAPR   ←(1-ρ+2ρ 𝕀{μ_min<μ̅_min})τ_PAPR,


where ρ>0 controls the update speed, μ̅_min is the mutual coherence of sequences 𝒫_best before previous K_1 iterations, and 𝕀{·}=1 if the condition is true and 0 otherwise.

The proposed LOw-Coherence sEquence Design Algorithm (LOCEDA) is summarized in Algorithm 1. 

Since non-orthogonal sequences are generally designed offline, the computational complexity of LOCEDA is not a primary concern and is, therefore, not studied further.







§ SIMULATION RESULTS



We now provide simulation results to verify the efficacy and flexibility of LOCEDA.






 §.§ Simulation Setup



The simulation results presented below are based on the following setup unless specified otherwise. 

We set the number of time-domain samples, N_S, equal to the number of subcarriers, N_C, with both N_S and N_C being 1024.

The index vector of assigned subcarriers is 𝐜=[1, 2, …, L]^T. 


For LOCEDA, we initialize the step sizes τ_seq=τ_PAPR=0.05 and generate initial sequences 𝒫 by normalizing the vectors with entries following a standard complex Gaussian distribution. 

In addition, we set K = 5, K_1 = 20, ρ=0.05, and γ=10^-4 for parameter updates.

We stop LOCEDA either after 10^4 iterations or if μ_min has not been updated for more than 500 consecutive iterations.





 §.§ Baseline Methods



To verify the effectiveness of LOCEDA, we compare our method to existing sequence-design methods from references <cit.>, <cit.>, <cit.>, <cit.>, and <cit.>, denoted as NOGBSS, NCBGSS, BGSSRMC, ETF, and ZC[NOGBSS, NCBGSS, BGSSRMC, and ZC, which only generate sequences of specific lengths and numbers, are used to generate sequences with the maximum supported length of no more than L. These are then zero-padded to L and normalized to ensure 𝐩_n = 1. 
If the number of generated sequences is at least N, we select N sequences with the lowest mutual coherence; otherwise, we exclude the corresponding methods from the comparison.
], respectively. 

For reference, we also provide the Welch lower bound of sequences without PAPR constraints Γ_bound, abbreviated as “Welch bound."





 §.§ Results and Analysis



In Fig. <ref>, we illustrate the trade-offs between mutual coherence μ(𝒫) and maximum PAPR max_n ∈𝒩PAPR(𝐩_n) for sequences generated by all methods under sequence length L=36 and the number of sequences N=100. 

Note that NOGBSS fails to produce 100 such sequences of length 36.

Evidently, LOCEDA achieves the best trade-offs between different mutual coherence values and PAPR constraints, whereas all baselines are limited to fixed mutual coherence values. 

This result confirms LOCEDA's ability to generate sequences under flexible user-defined PAPR constraints.

Additionally, LOCEDA’s sequences consistently exhibit lower mutual coherence than those from baseline methods for any given maximum PAPR, further demonstrating the superiority of our approach.





Fig. <ref> compares the mutual coherence of sequences generated by various methods across different sequence lengths, where we consider different PAPR thresholds Γ_PAPR = 1.5, 2, 4, and +∞.

Notably, the NOGBSS and NOBGSS methods are unable to produce sequences of the specified length and number of sequences, even in the absence of PAPR constraints (i.e., Γ_PAPR=+∞). 

Additionally, sequences generated by other baselines only meet some PAPR constraints, indicating limited adaptability. 

In stark contrast, LOCEDA consistently generates sequences with the required lengths while meeting each PAPR constraint, highlighting its exceptional flexibility and effectiveness. 

Moreover, LOCEDA achieves the lowest mutual coherence among the compared methods, significantly outperforming the competition. 

Furthermore, as the sequence length L nears the number of sequences N, the mutual coherence corresponding to LOCEDA approaches the theoretical lower bound Γ_bound, demonstrating its efficacy in optimizing sequences to near-optimal levels.



Fig. <ref> compares the mutual coherence of sequences generated by various methods under different numbers of sequences, with a sequence length of L=72 and an index vector 𝐜=[2,4,…,2L]^T. Here, we also consider different PAPR thresholds Γ_PAPR = 1.5, 2, 4, and +∞.

In these scenarios, none of the baselines can guarantee a PAPR of 2 or less for their sequences. 

Compared with them, LOCEDA can generate sequences that satisfy the required length and number of sequences, satisfying any specified PAPR constraints under different subcarrier assignments. 

This demonstrates LOCEDA's superior flexibility and robustness.

When Γ_PAPR=4 or +∞, the mutual coherence corresponding to LOCEDA consistently and markedly outperforms the baselines, verifying its superiority.





§ CONCLUSION

This letter has proposed an efficient low-coherence sequence design algorithm, LOCEDA, which can generate any number of low-coherence sequences of any length. 

The designed sequences are compatible with flexible subcarrier assignments in OFDMA systems and satisfy user-defined PAPR constraints. 

Simulation results have confirmed LOCEDA's efficiency and flexibility under various scenarios.

LOCEDA would provide valuable sequences for multi-carrier wireless communication systems.

Finally, a convergence analysis of the proposed method is challenging and left for future work.













IEEEtran





 Paper Title: Pedestrian Inertial Navigation: An Overview of Model and Data-Driven Approaches.















Pedestrian Inertial Navigation: An Overview of Model and Data-Driven Approaches
    Itzik Klein 0000-0001-7846-0654
I. Klein heads the Autonomous Navigation and Sensor Fusion Lab, Hatter Department of Marine Technologies, Charney School of Marine Science, University of Haifa, Haifa 3498838,
Israel (e-mail: kitzik@univ@haifa.ac.il).

    
==============================================================================================================================================================================================================================================================

§ ABSTRACT 
The task of indoor positioning is fundamental to several applications, including navigation, healthcare, location-based services, and security. An emerging field is inertial navigation for pedestrians, which relies only on inertial sensors for positioning. In this paper, we present inertial pedestrian navigation models and learning approaches. Among these, are methods and algorithms for shoe-mounted inertial sensors and pedestrian dead reckoning (PDR) with unconstrained inertial sensors. We also address three categories of data-driven PDR strategies: activity-assisted, hybrid approaches, and learning-based frameworks.















§ INTRODUCTION

Indoor positioning is a fundamental task for various applications, including navigation, health-monitoring, location-based services, and security. Because global navigation satellite system (GNSS) signals are not available indoors,  positioning relies on other approaches such as WiFi  <cit.> or other radio frequency signals <cit.>, visual positioning <cit.>, floor plan matching <cit.>, and inertial sensing  solutions <cit.>.
Of the latter, several approaches exists, including inertial navigation system (INS), shoe-mounted INS (SM-INS), and  pedestrian dead reckoning (PDR).

For indoor navigation, low-performance inertial sensors are  commonly used, for example, those installed in smartphones and wearable devices. Therefore, without external updates, the classical INS algorithm, which requires three integrations on the measured inertial data, results in large positioning errors.  To overcome this limitation, inertial sensors can be mounted on a shoe. To limit the inertial drift, zero velocity updates and other methods of information aiding are used. But for unconstrained inertial sensors, like in smartphones, information aiding is not applicable, therefore approaches like PDR are used, which require less integrations on the inertial readings. 

The present paper is focused on SM-INS and PDR approaches, as shown in Figure <ref>. Section <ref> presents the model-based PDR framework and explains each part and its role in a common PDR algorithm. Section <ref> describes the SM-INS approach, with all relevant algorithms, including INS, nonlinear filtering, zero velocity detector, and information aiding.  Section <ref> lists three strategies for data-driven PDR, including activity-assisted, hybrid, and learning-based. Finally, Section <ref>  summarizes the paper.





§ MODEL-BASED PEDESTRIAN DEAD RECKONING

Inertial sensors, such as accelerometers and gyroscopes, are required for using model-based PDR algorithms. Smartphones or wearable devices with inertial sensors are commonly used for this purpose. An exception is shoe-mounted inertial sensors (discussed below in Section <ref>).


Model-based PDR consists of four stages (Figure <ref>):

    
  * Step detection: The pedestrian steps are detected generally based on accelerometer readings.
    
  * Step-length estimation: Several approaches may be used to estimate pedestrian step-length, including regression-based, biomechanical models, and empirical relationships.
    
  * Heading determination: The heading of the pedestrian is estimated from the gyroscope and/or magnetometer readings.
    
  * Position update: The current pedestrian position is determined based on initial conditions,  heading angle (stage 3), and step-length size (stage 2).

As with any dead-reckoning method, the user's initial position must be known before undertaking the PDR cycle. Additionally, most step-length algorithms require a calibration phase to determine their parameters. Finally, we note that most model-based PDR approaches assume that the user is walking on the same plane (floor), allowing only the estimation of the pedestrian's 2D position, excluding  altitude. 




In the following subsections, we  elaborate on each stage in the model-based PDR approach   and provide frequently used implementation methods. 



 §.§ Step detection approaches

Steps can be detected in several ways. Although some approaches have been derived for specific inertial sensor locations, with appropriate parameter tuning they can be applied also for other inertial locations.

As noted in <cit.>, step detection methods can be of four types: peak detection, zero crossing, autocorrelation, and spectral analysis.
The peak detection method detects the maximum peaks of the  accelerometer signal to determine the user step instances <cit.>. In zero crossing detection, the zero crossing of the accelerometer signal is monitored for step detection <cit.>. The cyclic nature of walking leads to strong periodicity in inertial sensor readings. It is possible to extract a cycle from a sequence of sensor data by looking for maxima in the mean-adjusted autocorrelation of the data over a period of time. Autocorrelation-based step detection can be applied to accelerometer <cit.> or gyroscopes readings <cit.>. Steps can also be detected in the frequency domain, for example, by using Fourier <cit.> or wavelet transforms <cit.>. 


Of the methods mentioned above, peak detection is the most popular. These approaches are typically based on the magnitude threshold of the specific force value and the minimum step period.  In this case, a step is defined as the interval between two successive peaks. Moreover, because PDR is a dead-reckoning approach, without external aiding, only short time scenarios can be considered.  Thus, the inertial frame (i-frame) is defined at the user’s starting point. For simplicity, we assume that the body frame (b-frame) coincides with the sensitive axes of the inertial sensors. 

To formulate the peak detection method, we denote the specific force vector expressed in the body frame, 𝐟^b_ib,  as

    𝐟^b_ib = [f_x  f_y  f_z]^T

The magnitude of the specific force vector (<ref>) at time k is

    f_mag,k = √(f_x,k^2+f_y,k^2+f_z,k^2)

The mean of the specific force magnitude throughout the trajectory is defined by

    f̅_mag = 1/n∑^n_k=1f_mag,k

where n is the number of samples. Next, The mean of the specific force magnitude (<ref>) is subtracted from the magnitude of the specific force vector (<ref>)

    f_m,k = f_mag,k - f̅_mag

Finally, the standard deviation (STD) of the specific force magnitude, after reducing its mean, is calculated by

    σ_f=  [
        1/n∑^n_k=1(f_m,k - f̅_mag)^2
     ]^1/2

 Two parameters are needed to apply a basic peak detection approach:

    
  * The minimum time interval between steps, defined empirically.
    
  * The minimum peak height, i.e., the minimum value allowed for a valid peak. It is common practice to use 1.5σ_f as the minimum value. 

We consider the following scenario to illustrate the peak detection algorithm: The user walks holding a smartphone in texting mode, that is,approximately waist-high. The user makes 26 steps to cover a distance of  25.4 meters. The specific force was measured by the smartphone accelerometers. Its magnitude, calculated by (<ref>), and magnitude mean value (<ref>) are shown in Figure <ref>. Next, the  mean is subtracted as in (<ref>) and the STD is calculated by (<ref>). Finally, the STD is multiplied by a factor of 1.5 and the resulting value is adopted as the minimum peak height. The minimum time interval between steps was set to 0.3 seconds. Using these two parameters, a simple maxima search is performed to find the user steps. 



Figure <ref> shows the specific force magnitude after subtracting its mean. The red circles show maxima values identified as steps. The peak-detection algorithm manged to detect all 26 steps. Note, however, that such approaches are sensitive to user dynamics (walking speed) and sensor location. 





 §.§ Step-length estimation

Pedestrian step-length can be estimated using several approaches, including regression-based, biomechanical models, and empirical relationships. The underlying idea behind these approaches is that a step can be detected based on  accelerometer readings. Therefore, many approaches in the literature attempt to establish a relationship between the specific force exhibited during a step and the length of the step. In a recent study, the use of smartphone inertial sensors for estimating step-length was analyzed and compared with 13 representative model-based step-length estimation models <cit.>. Lately,  <cit.> provided a survey of step-length from various perspectives, including the research method used, the length of the test path, various walking speeds, the location of the sensor device on the user's body, and the accuracy achieved in estimating step-length.


We consider three step-length approaches:

    
  * SL1: A constant step-length approach that assumes that all steps during a walk have an equal length and are a function of the user's gender and height <cit.>
    
    s_c = k_cḣ

    where s_c is the constant step-length, h is the user height, and k_c is a gain equal to 0.415 for men and 0.413 for women. As it is based solely on user height, this is one of the simplest approaches to estimating step-length. Yet, as a constant value approach, it fails to cope with varying step-lengths.
    
  * SL2: The Weinberg <cit.> is a biomechanical approach, based on the inverted pendulum model. Originally, it relied on the difference between the maximum and minimum vertical acceleration values during a step. Later, it was shown to operate successfully using  the specific force magnitude instead <cit.>. Based on the specific force magnitude, the Weinberg approach uses the formula:
      
    s_w = k_w  (f_mag,max - f_mag,min )^1/4

    where k_w is the Weinberg gain, s_w is the Weinberg-based step-length, f_mag,max is the maximum value of the specific force during the step interval, and f_mag,min is the minimum value of the specific force during the step interval. Before using the approach, a calibration procedure, with the user walking a certain distance, should be applied to determine the gain, s_w.  Taking the sum operator on both sides of (<ref>) along the trajectory, allows  determining the gain because the sum of the step-lengths is the known travelled distance. Thus, Weinberg's approach relies on a single empirical gain, which must be calibrated before its use.
    
  * SL3: Adaptive step-length estimation. The step-length of a pedestrian is not constant and varies with walking speed, step frequency, acceleration variance, and other parameters. To increase accuracy in estimating walking distances,  adaptive step-length estimation algorithms have been proposed.  Of these, in <cit.>, the step-length is a function of step frequency and the variance of the specific force during the step, as given by:
    
    s_a = k_a,1· SF + k_a,2·σ_f + k_a,3

    where SF is the step frequency defined as the inverse of the time duration of the step, σ_f is the specific force variance (<ref>), and k_a,1-k_a,3 are predefined empirical gains.  As in the Weinberg approach, the determination of the empirical gains requires a calibration procedure.  

Returning to our numeric example from Section <ref>, we tested the three step-length approaches: SL1-SL3. For SL1, the walking user was a 190 cm-tall male. Calibration procedures with the same user were carried out to obtain the gains of approaches SL2 and SL3.  The estimated distances of the three approaches are shown in Figure <ref>. Although the user walked at a constant pace to make sure that his steps were equal, the constant-length approach, SL1, performed worse than the adaptive approaches.  SL2 and SL3 obtained a distance error of less than 2% of the travelled distance. 




Figure <ref> shows the the lengths of each of the 26 steps made during the trajectory. In SL1, a constant step-length of 79cm was obtained. In both SL2 and SL3, the average step-length was 99cm, reflecting the actual step-length.
 



Step-length approaches are sensitive to user characteristics (like height and weight), inertial sensors location, and walking speeds.



 §.§ Heading and walking direction

To estimate the pedestrian's trajectory, it is necessary to estimate the  walking direction and heading. In some cases, the direction of walking does not coincide with the direction of the inertial sensor's sensitive axis.  For example, when using the smartphone, the user may point it with an offset to the  walking direction, as illustrated in Figure <ref>. 



The figure shows the heading geometry defined relative to the starting point of the trajectory. From the geometry we define:

    ψ_u = ψ_p - ψ_s

where ψ_u is the user's heading (walking direction), ψ_p is the heading angle of the smartphone (or any other inertial sensor), and ψ_s, is the sliding (offset) angle between the smartphone and the user. 
Equation (<ref>) represents a general walking scenario. To better illustrate the difference, consider the scenario shown in Figure <ref>, where a user holding a smartphone starts walking while the smartphone is aligned to the walking direction (segment A). During the walking, the user maintains the same direction but changes the smartphone direction by 45^∘, as shown in segment B. Next, simultaneously, the user changes  walking direction by 45^∘ and  smartphone direction by 90^∘,  aligning the two, as shown in segment C. Finally, in segment D, the user rotates the phone by 45^∘ relative to the  walking direction. The user trajectory and both smartphone and user heading angles are illustrated in Figure <ref>.  
  



Two spacial cases may occur: (a) the inertial sensors are held in the same direction as the walking direction, satisfying ψ_s(t)=0, ∀ t, for example, a  user walking and holding the smartphone in texting mode; (b) the inertial sensors are rigidly mounted to the user, resulting in a constant offset angle ψ_s(t)=ψ_s,c, ∀ t. 

One of the most commonly used methods for estimating the heading of smartphones, ψ_p, is based on fusing data from accelerometers, magnetometers, and gyroscopes in an attitude and heading reference system (AHRS) framework. There are many types of AHRS approaches, including the Kalman filter <cit.>, nonlinear estimation <cit.>, complementary filtering <cit.>, and hybrid-learning approaches <cit.>. Regardless of the AHRS algorithm choice, they all rely on the  kinematic equations of the gyroscope. Following  <cit.>, we provide a simplified description of these equations, assuming initial zero roll and pitch angles, and that the x-axis of the smartphone is aligned with the walking direction, resulting in an initial zero heading. This corresponds to an initial quaternion of q_0∈ℝ^4:

    q_0 = 1 + 0i + 0j + 0k

where 1, i,j,k are the quaternion basis elements.

Given the gyroscope's measurement, [ω_x ω_y ω_z]^T, the kinematic equation for the quaternion (its rate of change in time) is

    q̇ = 1/2 [ [ -q_2 -q_3 -q_4;  q_1  q_4 -q_3; -q_4  q_1  q_2;  q_3 -q_2 -q_1 ] ]  [ [ ω_x; ω_y; ω_z ] ]

Given q̇ and the previous quaternion, the current quaternion can be calculated to construct the transformation matrix from the reference  frame to the body frame:
 
    T^b_r =  [ [ q^2_1+q^2_2-q^2_3-q^2_4        2(q_2q_3-q_1q_4)        2(q_1q_3+q_2q_4);        2(q_2q_3+q_1q_4) q^2_1-q^2_2+q^2_3-q^2_4        2(q_3q_4-q_1q_2);        2(q_2q_4-q_1q_3)        2(q_1q_2+q_3q_4) q^2_1-q^2_2-q^2_3+q^2_4 ]]


Using (<ref>), the smartphone heading is calculated by

    ψ_p = atan2  ( 2(q_2q_3-q_1q_4), 1-2(q^2_3+q^2_4)   )

As noted, using (<ref>) for the estimation of the user's walking direction is valid only when the smartphone is aligned with the user's walking path so that ψ_s=0 and thus ψ_u=ψ_p. If this is not the case, other approaches to estimate  the user's walking direction are needed. 
The walking direction may be extracted from acceleration measurements using a kinematic “rolling-foot” model <cit.>, by principle component analysis of the accelerometer readings <cit.>, or using gravity-based  <cit.> and deep learning approaches <cit.>.

Here, we describe a simple yet efficient gravity-based approach for estimating the user's walking direction. First, the gravity direction vector is obtained and the angular velocity is projected to obtain its vertical component, which is integrated to find the walking direction.
As stated in <cit.>, to reduce or eliminate the large temporal variations typically associated with the acceleration of a pedestrian in motion, it is necessary to apply low-pass filtering to each axis of the force vector. Then, the gravity vector direction is given by

    γ_g = -f_LPF/||f_LPF||

where γ_g is the gravity direction vector and f_LPF is the filtered specific force vector in the sensor frame.

Denoting the vertical projection of the gyroscope reading by ω_v and the measured angular velocity by  ω, it can be calculated at any instance of time by

    ω_v = γ_g^Tω

As ω_v measures the pedestrian’s turning rate in the horizontal plane, an approximation for the change in the walking direction during the time interval Δ t is 

    Δψ_u = ω_vΔ t

The (<ref>)-(<ref>) approach for finding the change in the user's walking direction is valid assuming that the angular velocity is constant during the time interval Δ t, which holds for most gyroscopes sampling at a high-rate. Notice further that to filter out the user acceleration using a low-pass filter, the user should be walking. In transient walking motion, as at the beginning or end of walking, we expect a degradation in filtering quality, leading to large heading errors.



 §.§ Two-dimensional positioning

Given the estimated step size, s, and the user's walking direction angle, ψ_u, the two-dimensional PDR user position vector is:

    [[ x_k; y_k ] ] =
      [[ x_k-1 + s_kcos(ψ_u,k); y_k-1 + s_ksin(ψ_u,k) ] ]

where x and y are the two-dimensional position components and k is the current epoch. 

Algorithm <ref> summarizes the two-dimensional model-based PDR method.





 §.§ Three-dimensional PDR

Most PDR approaches assume that the user walks in the same plane, without vertical movement. Yet, in typical walking environments vertical movement is required, for example, when using staircases, elevators, or
escalators. A typical solution for solving the altitude problem is to use a barometer to determine the change in altitude, as applied in <cit.>. Given a barometer, the 3D PDR algorithm, for each sensor role, is presented in Figure <ref>.



Yet, as pointed out in <cit.>, not all smartphones or wearable devices have a barometer. In these cases, the altitude can be estimated using accelerometer measurements. In <cit.>, a modified Weinberg approach was suggested to estimate the user's steps during movement on a staircase and a calibration phase is required to estimate the Weinberg gain.
A later study, <cit.>, proposed estimating the change in height by identifying peaks in the motion of the user during movement on a staircase in the course of which the user is changing position. This approach does not require prior calibration.



§ SHOE-MOUNTED INS

In SM-INS (or foot-mounted) devices, the inertial sensors are rigidly mounted on a shoe<cit.>, so that zero velocity events are identified and information aiding can be applied to mitigate the inertial drift <cit.>. Therefore, instead of experiencing a velocity drift caused by a strapdown INS mechanism, a shoe-mounted INS produces a saw-tooth like velocity error behaviour. Figure <ref> illustrates this assuming ideal initial conditions and only accelerometer bias. 



In this example, every one-second stationary condition is identified and information aiding is applied to nullify velocity drift.  In practice, the same velocity error behavior is observed even when taking into account all accelerometer and gyroscope errors. 


The shoe-mounted INS framework, using an extended Kalman filter (EKF), is illustrated in Figure <ref>. Note that any other nonlinear filter can be applied instead.



The INS navigation solution is produced in accordance with the inertial conditions and inertial readings. The inertial measurements, together with the INS solution, are used in the EKF prediction phase. Next, an algorithm to detect zero velocity instances is applied. If such instances are detected, information aiding is used in the EKF update phase to produce a corrected navigation solution. Otherwise, the INS solution is not corrected by the filter.
We examine each component of the shoe-mounted INS in detail below. 



 §.§ Inertial Navigation

The INS equations of motion are a set of first order differential equations. Given initial conditions and inertial measurements, they can be solved to give the navigation solution, namely the position, velocity, and orientation. These equations are valid for any platform,  regardless of its operating environment <cit.>. In shoe-mounted INS, however, some simplification is applied, reducing the complexity of the INS equations of motion.  Generally, low-performance inertial sensors are used in shoe-mounted INS, therefore, angular velocity vector of the earth can be neglected. The transport rate is also neglected because the user velocity is relatively low. Last, it is assumed that the body frame coincides with the inertial sensor frame located on the shoe, as illustrated in Figure <ref>. 



Taking those assumptions into account, the continuous form of the INS equation of motion is:

    v̇^r = T^b_rf^b_ib + g^r

where v̇^r is the velocity vector expressed in the reference frame,  f^b_ib is the specific force vector expressed in the body frame, T^b_r is the reference to body transformation matrix defined in (<ref>), and g^r is the local gravity vector defined by:

    g^r =  [ 0, 0, g  ]^T

where g is the local gravity, assumed constant during the motion.

The position rate of change, ṗ^r,  is the integration on the velocity vector:

    ṗ^r =  v^r

Last, the transformation matrix rate of change is

    Ṫ^b_r = T^b_rω^b_ib

where ω^b_ib is the measured gyroscope angular velocity vector.

Given inertial conditions, the INS equations of motion, (<ref>), (<ref>), and (<ref>) are integrated to obtain the navigation solution. The choice of the numeric integration algorithms is a trade-off between precision and processing capability. For a detailed discussion of this topic, refer to <cit.>. In SM-INS, the Pade approximation is regularly used to propagate the transformation matrix <cit.>:

    T^b_r,k = T^b_r,k-1 ( 2I_3+Ω_kΔ t
         )
          ( 2I_3-Ω_kΔ t
         )^-1

where Δ t is the time-step, k is the current epoch, and Ω_k is the skew symmetric of the measured angular velocity

    Ω_k = 
      [ [      0 -ω_z,k  ω_y,k;  ω_z,k      0 -ω_x,k; -ω_y,k  ω_x,k      0 ] ]

The velocity is updated using the Euler method:

    v^r_k = v^r_k-1+Δ tv̇^r

where v̇^r is defined in (<ref>). Similarly, the position update is given by:

    p^r_k = p^r_k-1+Δ tṗ^r

where ṗ^r is defined in (<ref>).



 §.§ Navigation Filter

The nonlinear nature of the INS equations of motion necessitates a nonlinear filter when INS is fused with external sensors or information, as in SM-INS. Before designing the filter, we define the inertial sensor measurement error model as:

    f_ib^b    = f_ib^b + b_a + w_a 
    ω_ib^b    = ω_ib^b + b_g + w_g

where w_a and w_g are the zero mean white Gaussian noise of the accelerometer and gyroscope measurement, respectively, and the sensor biases are modeled as random walk processes

    ḃ_a      =  w_ab
    ḃ_g      =  w_gb

where b_a is the accelerometer bias,  b_g is the gyroscope bias, and w_ab and w_gb are the zero mean white Gaussian noise of the accelerometer and gyroscope bias modeling, respectively.

An error-state EKF implementation is used with an  error-state vector

    δ𝐱 = [ [ δ𝐩^r δ𝐯^r  ϵ^r  𝐛_a  𝐛_g ]]^T∈ℝ^15

where δ𝐩^r is the position error-state vector, δ𝐯^r is the velocity error-state vector, and ϵ^r is misalignment error-state. 

The linearized error-state model is <cit.>

    δ𝐱̇ = 𝐅δ𝐱  + 𝐆𝐰

where 𝐅 is the system matrix, 𝐆 is the shaping matrix, and δ𝐰 is the noise vector.
The residuals of the accelerometers and gyros  are modeled as random walk processes although any other suitable models may be used instead, such as the first-order Gauss-Markov process. The system matrix is given by

    𝐅 = [ [   0_3×3     𝐈_3 0_3 × 3  0_3× 3  0_3× 3;    𝐅_vp   0_3×3    𝐅_vϵ   𝐓_b^n  0_3× 3;   0_3×3   0_3×3   0_3×3  0_3× 3   𝐓_b^n;   0_3×3  0_3× 3  0_3× 3  0_3× 3  0_3× 3;   0_3×3  0_3× 3  0_3× 3  0_3× 3  0_3× 3 ]].

where F_ij are 3× 3  submatrices obtained from the linearization of the nonlinear equation of motion (more details on the internalization process can be found in navigation textbooks such as <cit.>). The shaping matrix is given by

    𝐆 = [ [  0_3×3 0_3× 3 0_3× 3 0_3× 3;  𝐓_b^n 0_3× 3 0_3× 3 0_3× 3;  0_3×3  𝐓_b^n 0_3× 3 0_3× 3;  0_3×3 0_3× 3    𝐈_3 0_3× 3;  0_3×3 0_3× 3 0_3× 3    𝐈_3 ]]

and the noise vector is

    𝐰 = [ [  𝐰_a  𝐰_g 𝐰_ab 𝐰_ab ]]^T


The implementation algorithm for the prediction phase of the EKF error-state closed loop is

    δ𝐱̂^-_k    =     0 
    𝐏^-_k    =    Φ_k-1𝐏^+_k-1Φ^T_k-1+𝐐_k-1

where  δ𝐱^-_k is the a priori estimate of the error-state, 𝐏^-_k is the covariance of the a priori estimation error, Φ_k is the state transition matrix, and 𝐐_k is the process noise covariance assumed to be constant throughout the trajectory.

The EKF update phase is:

    δ𝐱̂^+_k    =    𝐊_kδ𝐳_k
    𝐏^+_k    =     [𝐈-𝐊_k𝐇_k]𝐏^-_k
    𝐊_k    =    𝐏^-_k𝐇^T_k[𝐇_k𝐏^-_k𝐇^T_k+𝐑_k]^-1

where δ𝐱^+_k is the a posteriori estimate of the error-state,   𝐏^+_k is the covariance of the a posteriori estimation error, δ𝐳_k is the measurement residual vector,  𝐊_k is the Kalman gain, 𝐑_k is the measurement noise covariance assumed to be constant for all samples, and  𝐇_k is the measurement matrix.



 §.§ Zero Velocity Detectors

The function of zero velocity detectors (ZVD) in SM-INS is to determine whether information aiding can be applied at the filter update stage (<ref>)-(<ref>). In a recent review of ZVD, Wahlström and Skog <cit.> recount the history of foot-mounted inertial navigation, characterize the main sources of error, and analyze current approaches to robust ZVD. These methods include heuristic approaches, adaptive thresholding, gait cycle segmentation, other model-based approaches, as well as data-driven methods. 

There exist several heuristic ZVD that rely on different inertial features, such as the acceleration variance or magnitude, the angular rate energy detector, and the stance hypothesis optimal detection (SHOE) detector. As pointed out in <cit.>, ZVD can be formulated as a likelihood ratio check. To this end, let ℋ_0 denote a hypothesis that the inertial sensor unit is moving and ℋ_1 a hypotheses that the inertial sensor unit is stationary. Note that stationary conditions occur during walking in the stance phase or while standing still. The likelihood-ratio test based on inertial measurements, z_j, decides on hypothesis ℋ_1 if and only if

    L(z_j) = p ( z_j| ℋ_1)/p ( z_j| ℋ_0)>γ

where γ is some user-defined threshold and  z_j) = m_n^j+w_f_j-w_b with w_f and w_b as the forward and backward window length applied to the required set of measurements. 

A multi-condition bringing together both accelerometer and gyroscope measurements was proposed in <cit.>. It includes three conditions for declaring a foot to be stationary:

    
  * Acceleration magnitude. Although referred to as acceleration, this is actually a condition concerning the specific force magnitude defined by:
    
    C1 = {[                                    1 γ_fmag,min   <  f_mag,k < γ_fmag,max;                                    0                            otherwise ] .

    where f_mag,k is defined in (<ref>), and γ_fmag,min and γ_fmag,max are the minimum and maximum threshold values, respectively.
    
  * Local acceleration variance. Defines the foot activity as
    
    σ^2_f = 1/2w+1∑^j=k+w_j=k-w (f_mag,k - f̅_mag)^2

    where w is the window size and f̅_mag is defined in (<ref>). 
    The second condition is satisfied when
        
    C2 = {[           1 σ^2_f>γ_σ_f;           0   otherwise ] .

    where γ_σ_f is the local acceleration variance value.
    
  * Angular velocity magnitude. This condition requires that the angular velocity magnitude
    
    ω_mag,k = √(ω^2_x,k+ω^2_y,k+ω^2_z,k)

    must be below a given threshold γ_σ_ω
     
    C3 = {[           1 σ^2_f>γ_σ_ω;           0   otherwise ] .


Detection of a stationary foot requires that all three logical conditions be satisfied simultaneously, so a logical "AND" is applied, and the result is filtered out with a neighboring window median filter. 

Rather than use heuristic or other model-based approaches for ZVD,  several papers have recently explored the possibility of using machine learning approaches to identify zero velocity instances. For example, <cit.> used support vector machines and  <cit.> a long
short-term memory (LSTM) neural network.



 §.§ Information Aiding

Once a zero velocity instance is detected, information aiding can be applied in the EKF update phase (<ref>)-(<ref>). Mostly, two types of information aiding are used in SM-INS: (1) zero velocity update (ZVU) and (2) zero angular rate (ZAR) <cit.>.

    
  * Zero velocity update 

    The principle of this aiding states that while in stationary conditions, the velocity vector in the reference frame is zero. The ZUV measurement residual is given by:
    
    δ𝐳__ZVU   = v__INS^r - 0_3 × 1 = H__ZVUδ𝐱 + ν__ZVU

    where v__INS^r is the calculated INS velocity vector (<ref>) and ν__ZVU is zero mean white Gaussian measurement noise. The corresponding measurement matrix is defined by:
    
    H__ZVU = [ 0_3 × 3     I_3 0_3 × 3 0_3 × 3 0_3 × 3 ].

    
  * Zero angular rate 

    Similarly to ZVU,  while in stationary conditions, the angular velocity vector in the reference frame is zero. The ZAR measurement residual is given by:
    
    δ𝐳__ZAR = ω^b_ib,_INS - 0_3 × 1 = H__ZARδ𝐱 + ν__ZAR = b_g+ ν__ZAR

    where ω^b_ib,_INS is the measured INS angular rate vector, ν__ZAR is zero mean white Gaussian measurement noise,  and b_g is the gyroscope bias vector expressed in the body frame. The corresponding measurement matrix is defined by:
    
    H__ZAR = [ 0_3 × 3 0_3 × 3 0_3 × 3 0_3 × 3     I_3 ].
 



 §.§ Summary and Analytical Assessment

Algorithm <ref> summarizes the SM-INS approach. The output of the algorithm is the updated error-state vector used to correct the INS solution and inertial sensor errors. 



Next, we provide a simplified analytical assessment comparing the distance errors of the INS solution, model-based PDR, and SM-INS. For this, we assume that only an accelerometer bias of b_a=5mg is present in the system. For short time periods, the INS position error is approximated by <cit.>:

    δ p_INS = 1/2b_a t^2

where t is the time.

To assess the distance error of model-based PDR, we follow <cit.>, which proposes the MoRPI framework, a horizontal PDR-like approach for mobile robots. It was shown that the position error using the Weinberg step-length approach (<ref>) is the sum of all peak-to-peak distance errors  

    δ p_PDR = ∑^N_j=1Δ k_w Δ f_j

where N is the number of steps, Δ k_w is the error in the Weinberg gain, and Δ f_j is

    Δ f_j =  (f_mag,max - f_mag,min )^1/4

For this example, we set Δ k_w=5%.

To produce the SM-INS distance error, the error was propagated using (<ref>) during the filter prediction phase, and when zero velocity was detected, it was assumed that the filer manages to correct 90% of the existing drift. In the examined scenario, stationary conditions were detected every one second. 

Figure <ref> shows the distance error of the approaches for a duration of 30 seconds. Because of the inertial measurement errors and the dead-reckoning nature, the distance solution of all three approaches drift, although at different rates. The INS reaches a distance error of 22.5m; the model-based PDR obtains a distance error of 3m; and the SM-INS performs best, achieving a distance error of 0.75m. This error represents an improvement of 75% over the model-based PDR approach. In real world scenarios, this improvement may be higher, depending on several factors, including the walking duration.   





§ DATA-DRIVEN PEDESTRIAN DEAD RECKONING

Both machine learning and deep learning approaches learn from data, therefore they are referred to also as data-driven approaches. 
These approaches are used in a variety of navigation tasks <cit.>, including PDR <cit.>, as described below.
Data-driven PDR can be categorized into three types:

    
  * Activity-assisted PDR: Data-driven approaches are used to classify the user motion and inertial sensor location. This information is used within the model-based PDR algorithm.
    
  * Hybrid PDR approaches: The heading angle or step-length are regressed by a data-driven approach while the other by using a model-based method.
     
  * Learning-based PDR: Rather than being based on models, PDR positioning relies only on learning methods to perform the positioning task.

The above-mentioned categories as well as the model-based PDR are illustrated in Figure <ref>. In the next section, a brief introduction to neural networks (NN) provides basic concepts for those who are unfamiliar with them.





 §.§ A Brief Introduction to Neural Networks

A typical NN consist of three types of layers: input layer, holding the initial data for the entire network, output layer, producing the result of the network, and hidden layer, referring to all intermediate layers between the input and output layers. The output of the network is refereed to as the predicted value, which is compared to the true value. To this end, a loss function (cost function) is chosen to measure the difference between predicted and true values. Thus, the loss function is used as a criterion in the optimization process for obtaining undated network parameters. The optimization process is applied until the loss value is lower than a predefined threshold.  A cycle of this process is illustrated in Figure <ref>.



From a mathematical point of view, the j-th hidden layer h_j is defined as

    h_j = a ( ∑^m_i=0x_i× w_i  )

where a is a nonlinear activation function, x_i is the i, i=0… m, input to the layer, and w_i are the corresponding weights. The input layer l_in∈ℝ^n× k contains the input data of dimension n to the network, where k stands for the sequence length. For example, using the specific force vector for a sequence of 30 samples gives n=3 and k=30. The output layer, l_out∈ℝ^p, gives the output with dimension p. The complete network, n, is represented using a composition of functions, where each function is a layer in the network:    

    n = l_in∘h_1⋯∘h_f∘l_out

where h_f is the last hidden layer in the network. Note that the network performed the following mapping:

    n : ℝ^n× k→ℝ^p

A detailed explanation of NN can be found in <cit.> and <cit.>.



 §.§ Activity-Assisted PDR

One of the critical phases in model-based PDR is the step-length estimation. As addressed in Section <ref>, commonly, an empirical gain is required to estimate the step-length. This gain was found to be sensitive to the user activity mode (walking normally/slow/fast, standing, running) <cit.> and to the smartphone activity, which implies  the location of the inertial sensors <cit.>. For example, when using the smartphone sensors, the location could be texting, when writing a message, or pocket, when the smartphone is placed in the user's pocket. Each of such mode results in a different gain value for the step-length estimation phase. One solution to this problem is to take an average gain value of all expected walking mode and smartphone locations during the pedestrian trajectory. Yet, as shown in <cit.>, even an average gain value results in a 10% position error because of gain inaccuracy alone. To mitigate the influence of the human activity and sensor location, a dedicated machine learning algorithm is added to the model-based PDR stages, as illustrated in Figure <ref>, to create an activity-assisted PDR framework. 
Notice that, nevertheless, the burden of prior gain calibration is needed for all modes and sensor locations expected in the pedestrian trajectory. As soon as the classification learning algorithm identifies the current mode, the appropriate gain is selected for the PDR algorithm.



To train the learning algorithm, a data acquisition phase is conducted to collect inertial sensor data. Next, a preproccssing phase is applied, including handling missing data, normalization, noise reduction, and outlier rejection. If a machine-learning algorithm is used for classification, certain features are required. These features are extracted based on the specific force and angular velocity vector components. Feature types include statistical, time domain, cross-sensor, and frequency domain features. As the feature set is created and the required activity or location classes are defined, the training process is performed with a goal of finding the most efficient classifier for the given task. 

The same data acquisition and prepossessing phases are used with deep learning approaches but the feature extraction part is not required because the network creates its own features from the raw inertial measurements. For classification problems, a commonly used loss function is the cross entropy loss, which evaluates the difference between the predicted and true class probability distributions.  

After the classifier has been designed, during the inference phase, raw inertial data (or features) are input to the classifier to output the human activity or the inertial sensor location. An example of this cycle is illustrated in Figure <ref>, where raw inertial sensor data are plugged into a smartphone recognition network to classify the smartphone location texting, pocket, and talking. 

See  <cit.> to get a better understanding of recent advances in pedestrian-related activity recognition. Note that, as stated in <cit.>, supervised networks are trained on a set of defined user modes (smartphone locations or user activities), available during the training process. As a result, when the classifier encounters an unknown mode, it must identify it as one of the original modes that it was trained on. It is likely that such classification errors will reduce the accuracy of the PDR solution, therefore, appropriate algorithms should be applied to identify unknown modes.





 §.§ Hybrid PDR Approaches

Data-driven approaches to PDR were initially hybrid approaches. This means that they provide only one of the two quantities required for PDR. Those are the step-length and user heading. The second one is provided using a model-based PDR approach. Although recent approaches focus on a complete learning-based framework, as described in the next subsection, hybrid PDR still has utility because it requires a lighter computational load. PDR step-length estimation is of interest in healthcare and biomechanical applications. 


    
  * Step-length learning. Generally, data-driven approaches are aimed to regress the step-length estimation. They replace both the step detection and step-length estimation in a model-based procedure (Figure <ref>), so that both the heading and the user position estimation are use model-based approaches, as illustrated in Figure <ref>. In <cit.>, three StepNet architectures for the regression task are proposed. Two of them require the step detection phase and regress the step-length within the pedestrian step cycle (varying time intervals); the third, omits the step detection phase and regresses the change in distance at a predefined (constant) time interval. The last one demonstrated the best performance. Subsequently, the accuracy of step regression approaches was further improved by using magnetometer readings as well <cit.>. 
    
    
    
    A variety of neural network architectures can be used for the regression task but these are beyond the scope of this paper.  Regardless of the architecture chosen, to solve the regression problem, the mean square error (MSE) loss is commonly applied:
    
    ℒ_MSE = 1/N∑^N_i=1 (x̂_i - x_i)^2

    where N is the number of samples, x̂_i is the estimated value of sample i, and x_i is the corresponding true value. In the training process, the network is optimized by minimizing the loss function. 
    
  * Heading learning.
    Similarly, only the user heading can be regressed using data-driven approaches within the model-based PDR framework, as shown in Figure <ref>. As any other regression problem, the MSE loss (<ref>) can be used, but to better reflect the circular nature of heading, other loss functions may be considered. For example, in <cit.>, the following heading loss was suggested:
    
    ℒ_Heading = 1/N∑^N_i=1 (sin(ψ̂_i) - sin(ψ_i))^2+(cos(ψ̂_i) - cos(ψ_i))^2
 
    where ψ is the heading angle.
    
    
    




 §.§ Learning-Based PDR Frameworks

In learning-based PDR approaches, the entire model-based PDR algorithm is replaced by neural network architectures to estimate the pedestrian's position. In general, such approaches fall into two categories: 

    
  * Pedestrian position regression. Following the same line of thought as in model-based PDR, such approaches regress the change in heading and distance to estimate the user position. IONet <cit.> examined LSTM and a bidirectional LSTM architecture, whereas <cit.> used residual networks (ResNets) for the regression only after the smartphone location was classified using a different network. 

    As both the change in distance and heading are regressed simultaneously, the loss function accounts for both of them, as for example:
    
    ℒ_PDR = 1/N∑^N_i=1 (d̂_i - d_i)^2 +λ1/N∑^N_i=1 (δψ̂_i - δψ_i)^2

    where d is the distance, δψ is the change in heading, and λ is a factor to balance the two losses. 
    
  * Pedestrian velocity regression. This type of approach regresses the velocity of the user and integrates the data to determine the position of the user based on that velocity. The first work in this field, named RIDI <cit.>, regressed the user velocity and used it to correct the user acceleration. Next, double integration was applied to the corrected acceleration to estimate the pedestrian position.  By building upon RIDI, RoNIN <cit.> offers a heading-agnostic coordinate frame representing the input and output of the network. They examined three different network architectures based on ResNet, LSTM, and temporal convolutional layers. In contrast to other approaches, RoNIN uses the device orientation together with the inertial readings to provide input to the network, as opposed to using only the inertial readings.     
 



§ SUMMARY

Inertial navigation for pedestrians is an emerging discipline that has wide applications in many fields, including healthcare, security, and indoor location-based services. 
In this work, we presented model and learning approaches to inertial pedestrian navigation. We provided detailed algorithms for shoe-mounted inertial sensors and classical PDR with unconstrained inertial sensors. These algorithms include the INS equations of motion and the EKF framework aided by information in stationary conditions. For PDR, we described methods for solving each phase of the algorithm for step detection, step-length estimation, and heading determination. Following these model-based approaches, we addressed three categories of data-driven PDR: activity-assisted and hybrid approaches, and learning-based frameworks.

IEEEtran





 Paper Title: Who should I trust? A Visual Analytics Approach for Comparing Net Load Forecasting Models.
































Who should I trust?A Visual Analytics Approach for Comparing Net Load Forecasting Models



Parts of this work were supported by the U.S. Department of Energy Solar Energy Technologies Office and the Office of Electricity Sensors Program, and performed jointly at the Pacific Northwest National Laboratory under Contract DE-AC05-76RL01830 and at the Lawrence Livermore National Laboratory under Contract DE-AC52-07NA27344).

    Kaustav Bhattacharjee1, Soumya Kundu2, Indrasis Chakraborty3, and Aritra Dasgupta1

1Department of Data Science, New Jersey Institute of Technology, USA. Email: {kb526, aritra.dasgupta}@njit.edu 
2Optimization and Control Group, Pacific Northwest National Laboratory, USA. Email: soumya.kundu@pnnl.gov
3Center for Applied Scientific Computing, Lawrence Livermore National Laboratory, USA. Email: chakraborty3@llnl.gov 

    August 5, 2024
========================================================================================================================================================================================================================================================================================================================================================================================================================================



§ ABSTRACT 
Net load forecasting is crucial for energy planning and facilitating informed decision-making regarding trade and load distributions. However, evaluating forecasting models' performance against benchmark models remains challenging, thereby impeding experts' trust in the model's performance. 

In this context, there is a demand for technological interventions that allow scientists to compare models across various timeframes and solar penetration levels. 

This paper introduces a visual analytics-based application designed to compare the performance of deep-learning-based net load forecasting models with other models for probabilistic net load forecasting. 

This application employs carefully selected visual analytic interventions, enabling users to discern differences in model performance across different solar penetration levels, dataset resolutions, and hours of the day over multiple months. We also present observations made using our application through a case study, demonstrating the effectiveness of visualizations in aiding scientists in making informed decisions and enhancing trust in net load forecasting models.



visual analytics, AI/ML, net load forecasting




§ INTRODUCTION



Comparing multiple computational models for their performance is crucial for enhancing trust in model outcomes because it provides a basis for evaluating the reliability and consistency of each model <cit.>. By assessing how different models perform under various conditions and scenarios, stakeholders can better understand their strengths, weaknesses, and overall effectiveness. Such comparative analysis helps to identify the most suitable model for specific tasks or applications, thereby instilling confidence in the reliability of the chosen model. This becomes more important while predicting the net load of an electric grid, which is defined as the difference between total electricity demand and generation from behind-the-meter resources like solar and distributed generators, and is influenced by various factors such as weather conditions and time of day <cit.>. 

Accurate net load forecasting enables grid operators, policymakers, and energy providers to make informed decisions regarding energy trade, load distribution, and resource allocation. However, the rise of solar energy generation sources in residential settings has significantly impacted the performance of traditional net load forecasting models, highlighting the need for robust time-series forecasting techniques <cit.>. Collaborating with scientists, we developed a deep-learning model that integrates variables such as temperature, humidity, apparent power, and solar irradiance to achieve strong predictive performance and resilience in the face of missing data <cit.>. However, the model's sensitivity to seasonal variations and noisy inputs underscores the need for further exploration with domain experts. Hence, we developed an interactive tool that empowers users to investigate the model's performance across diverse time periods and input scenarios <cit.>.





However, while we initially recognized the importance of comparing the model's performance with traditional models, feedback from domain experts further emphasized its significance in building trust in model outcomes. Visual analytics can play a pivotal role here, as evidenced by prior research demonstrating its importance in enhancing trust in machine learning models <cit.>.

This is exemplified by the interactive tool we developed, which enabled domain experts to extract valuable insights concerning the model's sensitivity toward temperature and humidity. Moreover, recent discourse, as highlighted in <cit.>, emphasizes the critical role of visual analytics in fostering trust-augmented applications of artificial intelligence and machine learning (AI/ML) within the energy sector. Building upon this, we designed  , incorporating carefully selected visual analytic interventions. These interventions facilitate the comparison of multiple models across various parameters, including solar penetration levels, dataset resolutions, and different hours of the day, enhancing stakeholders' confidence in model performance.



The aim of our application is to build trust through model comparison, primarily comparing our net load forecasting model with a reference model. We enhanced our model to process inputs at varying resolutions and then devised the reference model, which generates predictions by averaging net load ground truths for the last 30 days at the same time point. Despite lacking predictive ML components, this reference model serves as a benchmark in various net load forecasting competitions, including the recent Net Load Forecasting Prize by the National Renewable Energy Laboratory (NREL) and the U.S. Department of Energy Solar Technologies Office (SETO) <cit.>. Through the web interface of , we were able to uncover patterns in the performance that help improve trust in the model outcomes. In this work, we identify the visual analytic tasks that are required to compare these models. These tasks can be extended to compare other similar net load forecasting models, enabling users to make well-informed decisions based on the outcomes of these models.

While previous research has predominantly focused on developing interfaces during the model development phase, tailored to assist model developers, our emphasis lies in the post-hoc evaluation of model performance, specifically addressing the needs of energy scientists and grid operators <cit.>. Other studies have explored the performance of probabilistic net load forecasting models through different visualization charts but have often lacked an integrated interactive interface <cit.>. In contrast, our interactive visual analytic tool provides carefully designed visual cues for comparing model performance across different factors like dataset resolutions and solar penetration levels, thus offering a novel approach in this domain.

In this paper, we first introduce the different models used in this work and the rationale behind choosing them. Subsequently, we outline the identified visual analytic tasks and detail the design decisions guiding the development of our interactive interface. This is followed by some of the observations made by our power scientist collaborator through that demonstrate its efficacy in comparing model performance across multiple facets. Finally, we conclude by sharing insights gained from this development and discussing some of the future research opportunities in this domain. Additionally, a short demonstration video is available https://bit.ly/trustvishere.



§ MODEL DESCRIPTION




We start with a deep learning-based probabilistic model tailored for net load forecasting in high behind-the-meter solar scenarios <cit.>. This model has three key components: a kernelized probabilistic forecasting (kPF) module, an autoencoder (AE), and a long short-term memory (LSTM) network. This model effectively captured complex temporal dependencies and uncertainties inherent in net load data, which is crucial for reliable forecasting in environments with high solar penetration. By incorporating kernel methods into probabilistic forecasting, the model handled non-linear relationships and captured subtle variations in net load influenced by solar energy fluctuations. The autoencoder component enhanced feature extraction and dimensionality reduction, facilitating the LSTM network's ability to capture long-term dependencies and predict future net load values accurately. Experimental results demonstrated the superior performance of this model compared to traditional forecasting models, showcasing its efficacy in addressing the challenges posed by high solar scenarios and advancing the state-of-the-art in net-load forecasting methodologies. 

This model was used in the Net Load Forecasting Prize competition hosted by NREL and SETO <cit.>. However, during this competition, we observed its underperformance on the data provided by the organizers, which had lower resolutions. We discovered that while the autoencoder component excelled with high-resolution datasets (such as 15-minute intervals), it struggled to effectively capture temporal dependencies in lower-resolution datasets (e.g., 1-hour intervals provided by the organizers). Consequently, this limitation led to subpar outcomes generated by the LSTM component. In light of this, we opted to remove the kPF and autoencoder components and instead developed a version of the model solely utilizing the LSTM component. Additionally, fine-tuning the number of layers in the LSTM component yielded significantly improved results in the competition.

The competition employed a reference model to assess model performance across various probability levels. As previously mentioned, this reference model simply utilizes historical input data from the past 30 days to generate probabilistic forecasts for a specific time point. This model can serve as the initial benchmark for assessing the effectiveness of other models. Therefore, in this work, we developed a reference model following the same principles. Since these forecasts are probabilistic in nature, we calculated the Continuous Ranked Probability Score (CRPS) for both the reference model and our model. Subsequently, we computed the Continuous Ranked Probability Skill Score (CRPSS) based on these CRPS scores, evaluating whether our forecast presents an improvement or deterioration compared to the reference forecast <cit.>. A positive CRPSS indicates that the forecast outperforms the reference forecast, whereas a negative value suggests inferior performance. Utilizing these CRPSS values, we compare the performance of our model against the reference model across multiple dates throughout the year, and present these values in . 



§ VISUAL ANALYTICS-BASED DESIGN


Our application employs a visual analytics-based design featuring coordinated views enhanced with visual cues to help users compare model performance. It combines interactive visualization with comparison metrics like CRPSS to improve trust in the model outcomes and also allows the users to probe the model and understand its performance across different solar penetration levels and months. In this section, we highlight the two tasks performed by and how its visual analytics-based design aids in executing these tasks:
T1: Compare model performance across different solar penetration levels and data resolutions: Model performance may vary across different solar penetration levels due to the increased variability in net load data caused by intermittent solar generation. LSTM-based models might struggle to capture and predict these dynamic behaviors accurately. Furthermore, datasets with higher resolutions, such as sub-hourly intervals, enable models to more effectively capture short-term fluctuations and dependencies. Hence, this task essentially involves comparing model performance at different solar penetration levels and data resolutions.
T2: Identify patterns across different timeframes: By analyzing performance across multiple timeframes, power scientists can assess the net load forecasting models' robustness and consistency in capturing both short-term fluctuations and long-term trends. This evaluation helps to identify whether a model's performance is consistent across various temporal scales or exhibits variability or biases at specific time periods. This task relates to identifying patterns in the model's performance across different months, various hours of the day, and different time periods.

Our application implements multiple coordinated views and components in order to fulfill these tasks. Next, we discuss the design of these views and components along with the rationale behind them:
Comparison View: As comparing model performance is the main objective of , we begin with the Comparison View, which utilizes modified box plots to compare models. Figure <ref>a depicts CRPSS values on the y-axis and different data resolutions on the x-axis. The box plots illustrate the distribution of CRPSS values, with the median typically exceeding zero, indicating superior performance of our model over the reference in most cases. Users can utilize the solar penetration level filter to compare performance across various levels (20%, 30%, 50%) (Figure <ref>c) (T1). However, based on initial feedback from domain experts, we recognized the importance of displaying the distribution of CRPSS values. Therefore, we integrated dots representing the CRPSS values, jittered along the x-axis to illustrate their distribution over the entire year. These dots align with insights from the box plot and additionally reveal instances where our model performs worse than the reference on certain dates. This led to the development of the Patterns View, where we can identify the timeframes where the model underperforms compared to the reference.

Patterns View: In this view, utilizes a heatmap to depict performance patterns for each month across different hours of the day (T2). The x-axis represents the 24 hours of the day (0-23), while the y-axis displays the months of the year (Feb - Dec) (Figure <ref>b). Each box in the heatmap denotes the average CRPSS value for each month at each hour, indicated by the color. Darker blues signify more positive CRPSS values, indicating the superior performance of our model compared to the reference at that time. Conversely, darker reds indicate more negative CRPSS values, signifying poorer performance of our model compared to the reference at that time. This diverging color scale aids in easily identifying performance patterns across different time points and facilitates the identification of instances where our model does not outperform the reference. Users can filter specific months to focus on particular time periods and analyze performance accordingly (Figure <ref>e). Additionally, based on feedback from domain experts, we implemented the Sidebar component to allow users to select specific date ranges and analyze performance patterns within those ranges.
Sidebar: The Sidebar facilitates user selection of start and end dates to filter results across all views, enabling focus on specific date ranges (T2) for comparing model performance within those periods (Figure <ref>d). Additionally, based on feedback from domain experts expressing interest in comparing model performance across all solar penetration levels simultaneously, offers a comparison mode toggle in the Sidebar. Enabling this mode updates both views to display box plots and heatmaps for all solar penetration levels side by side, aiding in point-to-point comparison and identification of performance patterns across different solar penetration levels.



§ RESULTS FROM A CASE STUDY

The efficacy of an application can be validated if the application is able to perform the intended tasks effectively. In this section, we show the results through a case study that demonstrates how can be used to compare and select net load forecasting models effectively.

This case study involved a power scientist with over 10 years of experience in power and grid systems. With expertise in nonlinear dynamics, large-scale networks, and distributed control, he played a crucial role in developing the model. His main objective was to assess the model's performance relative to the reference model across various time points and solar penetration levels, which are critical factors to consider before deploying it for a project. We informed him that we had integrated CRPSS values for both the model and the reference model for all dates throughout the year. He then accessed through a browser and examined the distribution of CRPSS values across different data frequencies at a 20% solar penetration level (Figure <ref>a). Notably, he discovered that the model performed better with the lower resolution dataset (1-hour), contrary to expectations for an LSTM-based model (T1). Upon further examination, he noted a marginal difference in the median CRPSS between high and low-resolution datasets (0.85 and 0.89, respectively). Therefore, he enabled the comparison mode through the Sidebar, enabling him to assess the model's performance as solar penetration increased. He noted that the model consistently performed well with the higher resolution dataset (15-min) across all other solar penetration levels, confirming the notion that our model excels with high-resolution datasets in high solar penetration scenarios (Figures <ref>b and  <ref>c). Next, he observed that although the median CRPSS was significantly above zero, the minimum value was negative. Upon inspecting the dots adjacent to the box plots in the Comparison View, he observed that while most dots clustered around the median line, there were a few outliers with negative CRPSS values.

Hence, the scientist sought to understand temporal patterns to identify if any specific hour of the day contributed to the negative values. Consequently, he navigated to the Patterns View within to analyze the CRPSS value distribution in the heatmap at a 20% solar penetration level (Figure <ref>d). Consistent with observations from the Comparison View, most heatmap boxes displayed varying shades of blue, indicating superior model performance compared to the reference across most months and hours. On closer look, the scientist noted a box-like pattern in the heatmap, revealing enhanced performance during morning hours (8 am to 10 am) from April to September, followed by a decline during midday (11 am to 4 pm), and another spike in the evening (4 pm to 6 pm) during these months. This pattern suggested that the model effectively captured diurnal variations in net load data and adjusted predictions accordingly (T2). While similar box-like patterns were evident across other solar penetration levels, the intensity of model performance varied with increased solar penetration. This insight holds practical significance for model selection during deployment, as it suggests the potential use of different models or model ensembles tailored to distinct times of the day, leveraging their respective performances on diurnal patterns. Thus, the power scientist was satisfied that could yield valuable insights regarding the model, aiding informed decision-making. A short demonstration video is available at https://bit.ly/trustvisthis link.


§ CONCLUSION

Efficient model selection for net load forecasting plays a pivotal role in energy planning and grid operations. In this work, we delve into this process, integrating visual analytics with input from domain experts to identify key tasks for comparative model selection. Subsequently, we translate these tasks into an interactive interface, enabling users to assess model behavior across various factors such as solar penetration levels, data resolution, and time of day.


Throughout this endeavor, we gained valuable insights into the model behavior and the challenges posed during the multi-way comparison of models. This collaborative effort underscored the need for interactive tools that facilitate the seamless translation of model insights into actionable decisions. We can argue that our application is a first step towards this direction. As a next step, our plan is to incorporate multiple net load forecasting models into the application and integrate additional metrics for effective comparison of their performance.

Looking ahead, we aim to enhance our application in several ways. Incorporating economic planning and analysis options will allow stakeholders to assess the cost-benefit ratio before model selection, thereby enhancing trust in the outcomes. Additionally, as the energy landscape evolves, our application's flexibility will be pivotal in effectively comparing and selecting forecasting models for real-world applications. In summary, our collaborative endeavor has produced a robust tool for multi-faceted model comparison and has paved the way for informed decision-making through visual analytics in energy planning.








abbrv-doi




 Paper Title: MambaCapsule: Towards Transparent Cardiac Disease Diagnosis with Electrocardiography Using Mamba Capsule Network.


1
.001


 





mode = title]MambaCapsule: Towards Transparent Cardiac Disease Diagnosis 
with Electrocardiography Using Mamba Capsule Network                      




























1]Yinlong Xu[
orcid=0009-0008-1987-7562
]






22321336@zju.edu.cn








[1]
    organization=College of Computer Science & Technology, Zhejiang University,
    city=Hangzhou,
    country=China
[2]
    organization=Department of Gastroenterology, First Hospital of Quanzhou Affiliated to Fujian Medical University,
    city=Quanzhou,
    country=China
[3]
    organization=School of Public Health, Zhejiang University,
    city=Hangzhou,
    country=China
[4]
    organization=State Key Laboratory of Transvascular Implantation Devices of The Second Affiliated Hospital and Liangzhu Laboratory, Zhejiang University School of Medicine,
    city=Hangzhou,
    country=China


[5]
    organization=Innovation Institute for Artificial Intelligence in Medicine, Zhejiang University,
    city=Hangzhou,
    country=China
    
[6]
    organization=School of Medicine, Stanford University,
    country=USA
    
[7]
    organization=School of Computer Engineering and Science, Shanghai University,
    city=Shanghai,
    country=China

[8]
    organization=College of Future Industry, Gachon University,
    country=Korea


  

2]Xiaoqiang Liu
liuxiaoqianghusina@163.com


1]Zitai Kong
kongzitai@zju.edu.cn

3]Yixuan Wu
wyx_chloe@zju.edu.cn

4]Yue Wang
ywang2022@zju.edu.cn


6]Yingzhou Lu

lyz66@stanford.edu

7,8]Honghao Gao
gaohonghao@shu.edu.cn
honghaogao@gachon.ac.kr

3]Jian Wu
wujian2000@zju.edu.cn

5]Hongxia Xu
Einstein@zju.edu.cn
[1]



[cor1]Corresponding author

















§ ABSTRACT 
Cardiac arrhythmia, a condition characterized by irregular heartbeats, often serves as an early indication of various heart ailments. With the advent of deep learning, numerous innovative models have been introduced for diagnosing arrhythmias using Electrocardiogram (ECG) signals. However, recent studies  solely focus on the performance of models, neglecting the interpretation of their results. This leads to a considerable lack of transparency, posing a significant risk in the actual diagnostic process. To solve this problem, this paper introduces MambaCapsule, a deep neural networks for ECG arrhythmias classification, which increases the  explainability of the model while enhancing the accuracy.Our model utilizes Mamba for feature extraction and Capsule networks for prediction, providing not only a confidence score but also signal features. Akin to the processing mechanism of human brain, the model learns signal features and their relationship between them by reconstructing ECG signals in the predicted selection. The model evaluation was conducted on MIT-BIH and PTB dataset, following the AAMI standard. MambaCapsule has achieved a total accuracy of 99.54% and 99.59% on the test sets respectively. 
These results demonstrate the promising performance under the standard test protocol.


















Mamba Capsule ECG Classification Explainable AI



[
    [
    August 5, 2024
==================





§ INTRODUCTION

Cardiac arrhythmia refers to the disruption of the cardiac rhythm caused by abnormal cardiac electrical activity <cit.>. For the reason that arrhythmia serves as an indicator to advent heart consequence, its accurate detection and classification are imperative for mitigating potential risks and complications. Electrocardiogram(ECG) is widely used for recording the electrical cardiac activity<cit.>. It provides critical information about the cardiac function and health by measuring the cardiac electrical signals during each heartbeat cycle, which makes it a reliable tool for diagnosing heart health.

Recent researches have shown great advantages of deep learning in arrhythmia classification<cit.>, leveraging the information contained in ECG signals. The development of powerful and effective deep learning methods has the potential to significantly improve the accuracy and efficiency of arrhythmia diagnosis, and ultimately leads to better patient outcomes.  

Although the numerous previous models have made substantial progress in arrhythmia classification, most of them solely focus on the performance  neglecting the interpretation of their results, which leads to a considerable lack of transparency and poses a low reliability in the actual diagnostic process<cit.>.

To alleviate this limitation, we proposed a encoder-decoder based neural network called MambaCapsule in this paper. Inspired by the mechanism of brain processing<cit.>, which tells what features exactly the eyes see and tells the reason why the brain chooses to classify one item to this label not the other one, we novelly designed our model to implement both processes by using the main architecture for prediction and a reconstruct part for explaination. Our evaluation was carried on MIT-BIH dataset<cit.> and PTB<cit.> dataset, reached 99.54% and 99.59% accuracy. By applying a reconstruct network, we could intuitively know what the model sees and why it makes that decision.

The main contributions of this paper are as follows:


=0pt

  ∙ We introduced a novel encoder-decoder based network architecture based on Mamba<cit.> and Capsule networks<cit.>, changing the form of the output from the probability sequence to capsule vector sequence, which contains both the classification and feature attribute.

  ∙ We applied a downstream network to reconstruct the ECG signal. The reconstructed signal can be used to explain what our model learns and the reason why our model makes its prediction.

  ∙ We applied a fusion states SSM architecture in process of feature extraction. The evaluation indicators show the feature extracted has considerable advantages on the MIT-BIH and PTB datasets.
  



§ RELATED WORK



 §.§ Mamba Nerual Network

Mamba has made excellent performance in numerous traditional fields since it was proposed by <cit.>. Different from traditional and the most popular structure Transformer, the advantages of Mamba are mainly reflected in the computational efficiency and the ability to deal with long sequences. For the reason above, many researches have been carried to replace traditional components with Mamba. <cit.> proposed Graph-Mamba, which improved the remote context modeling ability of attention mechanism by integrating Mamba block and node selection mechanism, and solved the problems of high computing cost and limited scalability of attention mechanism. 
<cit.> proposes a new model-based meta-reinforcement learning method, achieving higher returns and better sample efficiency (up to 15x) with little need for hyper-parameter tuning.
<cit.> proposes a multi-modal large language model VL-Mamba based on state space model, demonstrating competitive performance on a variety of multi-modal benchmark tasks and the great potential of state-space models in multi-modal learning.

Mamba also show its advantages in vision tasks <cit.>. <cit.> proposed a generic vision backbone with bidirectional Mamba blocks (Vim) to solve the location sensitivity and global context requirements of visual data. Vim performs well in various tasks, with significant improvements in compute and memory efficiency. <cit.> introduced VideoMamba into video understanding task. Its linear-complexity operator enables efficient long-term
modeling, which is crucial for high-resolution long video understanding.




 §.§ Capsule Neural Network

Capsule network was first proposed in a 2D handwritten digital image classification task and has excellent performance in the view-invariance <cit.>. Later on, it was widely applied to point cloud <cit.>, image processing <cit.>, and tabular learning <cit.>, graph learning <cit.>. Specifically, <cit.> proposed a multi-scale capsule network that is more robust and efficient for feature representation in image classification, demonstrating a competitive performance on FashionMNIST
and CIFAR10 datasets. <cit.> introduce
a capsule network that can detect various kinds of attacks. The model uses many fewer parameters than traditional convolutional neural networks with similar performance.

Capsule neural network also has a intrinsic advantage for explaining its results. For instance, <cit.> produced a sentiment capsule to identify the informative logic unit and the sentiment based representations in user-item level for rating prediction. By introducing capsules in different layers, <cit.> demonstrated the possibility of transforming deep networks into transparent networks. This work helps make DNNs' internal processes more understandable and provides the ability to explain their decisions.




 §.§ ECG based Cardiac Disease Diagnosis

Numerous researches have been carried in the field of ECG signal classification and the most challenging part of it is to extract the features from the signal. The feature extraction methods have experienced from the manual features and automatic features. Manually designed features extracting usually applied by traditional ML methods like Support Vetor Machine <cit.> and Random Forest <cit.>.
<cit.> proposed an effective methods to extract the abnormal beat eigenvectors of low-dimensional ECG and classify them using support vector machine. <cit.> proposed a ML model using multi-domain features combined with least-square support vector machine (LS-SVM), which extracts the time and frequency domain features and selects the most relevant ones.

With the success of deep learning in vision and NLP fields, its ability has been admitted and has been applied to many other fields, including the ECG signal classification <cit.>. 
<cit.> proposed a 1D residual CNN based on deep residual networks with a 31-layer one-dimensional residual convolutional neural network combined with 2-lead ECG signals for the classification.
<cit.> proposed a novel model combining residual networks, compressed activation blocks and bidirectional short-duration memory networks. It was evaluated on multiple databases and the results showed that the framework performed well especially for a small number of categories.

Although these methods have their own way to explain the process of the prediction, their lack of accuracy and inadequacy of features extracting hinder their practical applications. Some efforts also focused on the class imbalance problom on ECG datasets. For instance, <cit.> introduced a deep learning model called ECGTransForm ,which combines multi-scale convolution, channel calibration module and two-way Transformer mechanism to improve the ability to capture spatio-temporal features in ECG data and solves class imbalance problem by introducing context-aware loss function. However, these methods were employed through simple data reinforcement using statistical methods, the reinforced data benefits to model performance but has no specific physiological information through the process of the reinforcement.
 


§ METHODS




 §.§ Mamba

State space models (SSMs) <cit.> are defined as 1D-time-invariant systems, which serve as the predecessor of Mamba. These systems aim to map a 1D input sequence x(t)∈ℝ^L to a 1D output sequence y(t)∈ℝ^L through a hidden state h(t)∈ℝ^N, which can be represented as the following equations:

    h(t) = Ah(t)+Bx(t)
       y(t)=Ch(t)


where A∈ℝ^N× N, B∈ℝ^N× L, C∈ℝ^L× N. Considering that deep neural networks operate in discrete space, researchers applied a zero-order preservation on the matrix A and B, turning the equations to the form as:

    h_t=Ah_t-1+Bx_t
       y_t=Ch_t


Where A=exp(△ A) and B=(△ A)^-1(exp(△ A)-I)· (△ B) for△ represents the time step in the continuous space. SSMs also code the remote dependency by initializing matrix A as HiPPO<cit.> and allow training in the form of a convolution kernel to accelerate the training process. However, the matrix A, B and C do not change whatever the input data are, leading to problem of low sensitivity. To solve the problem, Mamba applies a selective method to SSMs, which makes the matrix B and C and △ adapt themself based on the input as follows:

    B = s_B(x)
       C = s_C(x)
       △ = τ_△(Parameter+s_A(x))


Where s_B, s_C and s_A are linear projections, and τ_△ is a SoftPlus function. The matrix A does not depend on the input directly, it is influenced by the △ in the process of the zero-order preservation, making all main parameter inputs adaptable<cit.>.






 §.§ Capsule Networks

The whole network contains three parts: the convolution network, the capsule network and the dynamic routing. The difference between the capsule network and the traditional neural network is that the unit of the neural network is a scalar neuron x∈ℝ, while the unit of the capsule network is a capsule, which is a vector x∈ℝ^N.

In a capsule network, there are many capsules in one layer, and each capsule contains two types of information: the given vector's orientation and the vector's length as the probability of the capsule, the next capsule first applies a linear transformation of the previous capsule output as:

    û_j|i,L+1=W_ij, Lu_i, L


where u_i,L∈ℝ^D is the i-th output capsule at layer L, W_ij,L∈ R^D̂× D serves as the transformation matrix and the û_j|i, L+1∈ℝ^D̂ represents the contributes from the parent i-th capsule to the j-th one at layer L+1. The model then gets the weight of each contributes through a clustering like routing algorithm. At present, the popular routing algorithms include dynamic routing algorithm and self-routing algorithm. This paper adopts dynamic routing algorithm, which obtains the final weight coefficient through several rounds of iteration. In each iterations, the weight coefficient c is:

    c_ij=exp(b_ij)/∑_k exp(b_ik)


In the first iteration the b_ij is initialized to 0 and will be updated at the end of each iteration. Then the coefficient is used to calculate the temp capsule:

    s_j=∑_i c_ijû_j|i


Where s_j∈ℝ^D̂ serves as a temp result for an iteration, the capsule network designs a special activation function to transform the length of the capsule into [0-1] as:

    v_j = ||s_j||^2/1+||s_j||^2s_j/||s_j||


Where v_j∈ℝ^D̂ serves as the real output capsule in each iteration. The equation <ref> separates the long and short length of the capsules in the dim of 0 to 1, making the long capsule close to 1 and short capsule close to 0. Then the b_ij is updated as:

    b_ij = b_ij + v_jû_j|i


The equation <ref> serves as an clustering-like algorithm for after each iteration the b_ij grows bigger in capsule whose position is  closest to the others. And after the setting iterations have done, the v_j will serve as the output of the capsule networks.



 §.§ Model Architecture


In this section, we present a classification model based on Mamba and Capsule networks. The overall framework, as shown in Figure <ref>, consists of three main parts: the feature network, the capsule network and the reconstruction network.  Before be fed into the feature network, the input ECG signal
is firstly up-projected to contain scope knowledge not a signal sequence, the input shape changes from x∈ℝ^batch, length to x̂∈ℝ^batch, length, dim.

Then N Mamba Layers constitute a pipeline to extract features from the x̂ and each Layer contains a Mamba Block and a Layer Norm. Previous work typically apply a single state Mamba as the encoder backbone, however, these fails to extract wider and narrower features. To solve this problem, we propose a fusion states SSM to construct the Mamba Block. For the i-th Block, it first applies a 1D-convolution on the upstream i-1-th feature sequence x_i, which does not change the dimension of the sequence. Then the sequence will be handled by m different SSM Blocks. The different SSM Blocks focus on different scales of time step and have different convolution shape to obtain more abundant and wider features. Then the features will be down sampled and then linked together to add the x_i in the same shape. The whole process can be formulated as:

    x̂_i = Conv_1D(x_i)
        u_j = DownSample_j(SSM_j(x̂_i))
        x_i+1 = Concat(∑_j^mu_j) + x_i


Where u_j ∈ℝ^batch, length, dim/mand the m is the factor of the dim to assure x_i+1∈ℝ^batch, length, dim.

The feature network extracts the signal features and sends the features to the capsule network. The capsule network outputs it in the form of capsules in the end. 




The loss of our model consists of two parts, the marginal loss of focusing on model output and the label, and the reconstruct loss focusing on the reconstructed signal and the original signal.

Since our output is in the form of a capsule network, we must first take the geometric length of the output as the probability of the classification label, and then the difference between the set positive label threshold and negative label threshold is made. The final form of margin loss function is as follows:


    Loss_margin =    ∑_k T_kmax(0, m^+-||v_k||)^2+
       λ(1-T_k)max(0, ||v_k||-m^-)^2


Where v_k∈ℝ^D is the k-th output capsule represents one class of the classification, T_k=1 if the label of the input is k, and m^+=0.9, m^-=0.1 at the beginning. λ is used to prevent the lengths of vectors from shrinking too much, which is set to 0.5 in the beginning.

Reconstruct loss needs to choose the label with the highest probability or we choose it for other reasons. Then we put the output into the reconstruct model, which is a 3 fully connection layers model with Relu activation. After that, calculate the MSEloss of the reconstructed signal output of the reconstruct model and the original input signal, the reconstruct loss is formulated as follows:

    x̂_reconstruct = Model(v_choose)
        Loss_reconstruct=MSELoss(x̂_reconstruct, x_origin)




§ MODEL SETUP AND EVALUATION



 §.§ Training Strategy

Since the output probability density of the capsule network is not the same as that of the traditional neural classification network, and the bias can directly affect the output, we adjusted bias by cosine schedule during the training process, so that the threshold of bias kept increasing. At the same time, due to the slow training speed of capsule network in the initial stage of training, the cosine annealing algorithm after linear increase is adopted to change the learning rate. The changes of bias and learning rate are shown in the figure.<ref>





 §.§ Performance Metrics

To evaluate the model, we applied five performance indexes: accuracy(ACC), sensitivity (SEN), F1-score(F1), precision (PPV) and specificity (SPEC). These indexes are defined as follows:


    ACC=TP+TN/TP+TN+FP+FN
       SEN=TP/TP+FN
       F1=2×ACC× SEN/ACC+SEN
       PPV=TP/TP+FP
       SPEC=TN/TN+FP


In these formulas, TP represents the number of true positives (correctly labeled positive); TN represents the number of true negatives (correctly labeled negative); FP represents the number of false positives (incorrectly labeled positive); and FN represents the number of false negatives (incorrectly labeled negative).


 

 §.§ Dataset

 Our training and testing datasets are preprocessed in the same routine and are randomly splited in a ratio of 0.8. The detailed description of the dataset is in Table<ref>:


  §.§.§ MIT-BIH Arrhythmia Dataset

The MIT-BIH arrhythmia dataset <cit.> is widely used for arrhythmia classification in recent researches. The dataset contains 48 half-hour two-channel ECG recordings digitized at 11 bit resolution in 10 mV range and a rate of 360 samples per second per channel. Two cardiologists annotated each record separately (about 110,000 annotations in total). MIT-BIH has annotated 41 categories which can be converted to five-classes classification problems following the ANSL/AAMI EC57: 2012 (The Association for the Advancement of Medical Instrumentation) standard. These distinctive classes are: Normal Sinus Rhythm (N), Supraventricular Premature or Ectopic Beat (S), Ventricular Premature or Ectopic Beat (V), Fusion of Ventricular and Normal Beat (F), Unknown Beats (Q). Some researches were carried on three or four classes due to imbalanced sample size, making the comprehensive evaluation difficult. In this paper, we chose to adapt to all the five classes.



  §.§.§ PTB Diagnostic ECG Database

The PTB diagnostic ECG database is a popular ECG recording database collected by the National Metrology Institute of Germany<cit.>. The dataset contains 549 records from 290 subjects (aged 17 to 87, mean 57.2; 209 men, mean age 55.5, and 81 women, mean age 61.6; ages were not recorded for 1 female and 14 male subjects). Each subject is represented by one to five records. Each record includes 15 simultaneously measured signals: the conventional 12 leads (i, ii, iii, avr, avl, avf, v1, v2, v3, v4, v5, v6) together with the 3 Frank lead ECGs (vx, vy, vz). Each signal is digitized at 1000 samples per second, with 16 bit resolution over a range of ± 16.384 mV. The PTB dataset consists of two main classes, i.e., (1) Normal (N), which represents normal ECG recordings, and (2) the Myocardial Infarction (M), which represents ECG recordings that exhibit signs of myocardial infarction, indicating the presence of a heart attack.









§ EXPERIMENTAL RESULTS



 §.§ Performance Evaluation


Many previous work focused on only four or less classes performance, aiming to reduce the perplexity of the problem or concentrate on specific arrhythmia types. However, it leads to great difficulties to compare the ability and universality of different models. In our work, we operate our evaluation on five full classes.



Table <ref> shows the confusion matrix of our best model prediction. And the detailed evaluations on each class of the MIT-BIH dataset are presented in Table<ref>. The result shows our model has excellent ability of discrimination, especially in class N, V and Q the Sensitive, Precision and Specificity reached almost 100%, which can be attribute the large training sample size. The performance of Supraventricular Premature or
Ectopic Beat (S) is not so outstanding, which can be the result of lack of training samples and the Fusion of Ven-tricular. The average performance of Normal Beat (F) can be the reason its original features resemble to the Normal class (N).





Table <ref> and Table <ref> also show our model great performance on PTB dataset. The confusion matrix indicates that our model predicts nearly all of positive and Negative samples, with an macro-avg accuracy of 99.59%.















The performance of our model is also compared with several previous studies, which is presented in Table <ref>, presenting comparative advantages over the other architectures. And the performance of each label can be seen in Table <ref>, which can be observed from the result that our model has considerable advantages compared with others in majority classes.



 §.§ Ablation Study

The ablation experiment was carried out to test the effectiveness of our model structure. Since Mamba was mainly used as the encoder and capsule networks as the decoder in our model, the ablation experiment was carried out in these two aspects on MIT-BIH dataset.



  §.§.§ Encoder Structure

This experiment aims to compare the performance of different model structures, including the number of stacked mamba layers, the number of mamba states, the steps per layer, whether to adopt the layernorm and dropout structures, and the use of different models as encoders.

In order to assess the number of mamba states and steps in each layer, we adjusted these parameters while keeping the encoder layers fixed at 4. The experimental results depicted in the figure<ref>demonstrate a significant impact of varying numbers of states and steps on the outcomes. Moreover, a layer composed of diverse mamba blocks outperforms a single layer due to its ability to focus on different ECG data features across various time scales. Consequently, this wider temporal coverage achieved by combined blocks leads to enhanced performance. However, if the states are too wide it will miss significant information.

To assess the impact of encoder layers on model performance, we held the parameters of each layer constant and varied the number of encoder layers from 2 to 8 while considering whether to implement layer normalization and dropout. The corresponding results are depicted in the figure <ref>. From our analysis of training time and memory usage for a four-layer encoder, it is evident that the metric increases with additional layers; however, model performance does not improve proportionally with increased depth. In fact, the optimal performance is achieved at six layers due to limitations in information capacity exceeding what can be learned from available data as well as increased computational demands associated with larger models.

In order to evaluate the effect of different types of encoder, we have also tried to use resnet, transformer and cnn as encoders. The experimental results are shown in the figure<ref>. The results show that mamba has the best effect when it is used as encoder.





  §.§.§ Decoder Structure

The purpose of this experiment was to compare the changes in the model training and the performance caused by using capsule instead of traditional decoder.

We tried to replace the capsule network with MLP after freezing encoder parameters. Experiments showed that there was no significant difference between the capsule network and MLP under the same encoder with F1-score 93.50 compared to 93.62, but the training time of capsule network was longer than that of MLP. This is determined by the routing structure of the capsule network and the number of parameters. 



 §.§ Explainable Study

The interpretability of this paper is unique from other recent papers on ECG diagnosis, mainly through the reconstruction of ECG signals in different ways to explain what knowledge the model has learned on MIT-BIH dataset. For the reason of the limitation of the memory, we only consider P-wave and R-wave in our reconstruction, discarding the T-wave which is far from the center of the signal.



  §.§.§ Feature Relation Mining


When processing the ECG graphs, the human brain not only attends to focus on individual wave peaks but also considers the interrelationship between them. The view invariance of capsule network aligns remarkably well with this characteristic, which is why we selected it as the decoder. In our experiment, we conducted forward and backward disturbance on the input data to observe the disturbance reconstruction of ECG signals using our model and DAE model. The experimental results are depicted in the figure <ref>, where left and right shift represent forward and backward shift respectively. Repeated reconstructions of ECG signals by MambaCapsule exhibit remarkable consistency, indicating that Mambacapsule has learned not only the intrinsic features but also their interrelationships. Conversely, this characteristic is absent in the reconstructed ECG signal produced by DAE model.



  §.§.§ Cardiac Disease Manifestation Type Inspection


When human brain categorize an item, it can discern the reasons for its classification and differentiation. Due to the inherent output characteristics of Capsule network, the output not only encompasses the probability density of each category but also encapsulates the distinctive features associated with that particular category. Consequently, by reconstructing ECG signals from unified inputs belonging to different categories, we are able to precisely comprehend why the model deems a certain category more probable. The experimental results are depicted in Figure <ref>. In Figure <ref>, both the ECG signal and model prediction are labeled as N, where P, R wave can be observed clearly and can be distinguished it represents a normal signal. However in Figure <ref>, which is reconstructed by the label S and the length of the capsule is adjusted to be equal to the capsule of label N. The false reconstruction represents a vanish in P-wave, which is a symbol of Supraventricular Premature or
Ectopic Beat. The reconstructions of N and P labels demonstrate an ablity of knowledge explaination and thus, each capsule can be seen as the information collection of different labels and the length of it is how much the information is concluded in the signal.



§ CONCLUSION


In this work, we have proposed an effective-while-explainable solution to ECG classification by introducing MambaCapsule, which emphasizes the capability of multi-scale Mamba in feature extracting and the interpretability of the form of Capsule network. By applying evaluation and reconstruction, the result of our model reveals great advantages in both classification score and mechanism explainablity. However, it's necessary to realize the limitation of our model, a the long training time and big memory occupied. In our future work, we aim to address these challenges by applying pre-trained parameters and optimize routing structure in Capsule network. As an automatic heart rhythm abnormality detection system, our model can offer doctors and patients accurate and reliable results, which have a wide range of application prospects.






cas-model2-names









 Paper Title: Hardware Limitations of Dynamic Metasurface Antennas in the Uplink: A Comparative Study.














Hardware Limitations of Dynamic Metasurface Antennas in the Uplink: A Comparative Study
    Maryam Rezvani1, Raviraj Adve1, Amr El-Keyi2, Akram bin Sediq2
	1Dept. of Electrical and Computer Engineering,
		University of Toronto, 
		Toronto, Canada 
	
	2Ericsson Canada, 
		Ottawa, Canada, 

		emails: {mrezvani, rsadve}@ece.utoronto.ca, {amr.el-keyi, akram.bin.sediq}@ericsson.com

    
====================================================================================================================================================================================================================================================================================================



§ ABSTRACT 
Dynamic Metasurface Antennas (DMAs) have emerged as promising candidates for basestation deployment in the next generation of wireless communications. While overlooking the practical and hardware limitations of DMA, previous studies have highlighted DMAs' potential to deliver high data rates while maintaining low power consumption. In this paper, we address this oversight by analyzing the impact of practical hardware limitations such as antenna efficiency, power consumed in required components, processing limitations, etc. Specifically, we investigate DMA-assisted wireless communications in the uplink and propose a model which accounts for these hardware limitations. To do so, we propose a concise model to characterize the power consumption of a DMA. For a fair assessment, we propose a wave-domain combiner, based on holography theory, to maximize the achievable sum rate of DMA-assisted antennas. We compare the achievable sum rate and energy efficiency of DMA antennas with that of a partially connected hybrid phased array. Our findings reveal the true potential of DMAs when accounting for the limitations of both designs.



Hybrid multiple-input multiple output (MIMO), Dynamic metasurface antenna (DMA), Holography theory, Uplink wireless communications.




§ INTRODUCTION


High throughput, energy efficiency, and low power consumption are key demands for the next generation of wireless communications <cit.>. While extra-large multiple-input multiple-output (XL-MIMO) <cit.> with each antenna connected to a radio frequency (RF) chaindigital phased arrays (DPAs), can provide high throughput, they would result in excessive power usage at basestations (BSs). 

In contrast, hybrid MIMO antennas <cit.> and dynamic metasurface antennas (DMAs) <cit.> connect a large number of antennas/unit-cells to fewer RF chains reducing power consumption. Specifically, while an N-element DPA uses N RF chains, in hybrid MIMO systemshybrid phased arrays (HP), using an analog combiner/precoder enables connecting N antennas to M RF chains, where N ≫ M <cit.>. One variant of HPs is a partially connected HP (PCHP). In PCHPs,
each antenna is connected to a phase shifter (PS), and the captured signal from N_s antenna-PS pairs, is combined via a Wilkinson combiner <cit.> before being fed to an RF chain. 

On the other hand, DMAs introduce a groundbreaking design by incorporating metasurfaces into the antenna structure, offering a wave-domain (WD) precoder/combiner. In DMAs, a large number of unit-cells are loosely coupled <cit.> to a few microstrip lines (or other waveguide types). Each unit-cell influences the impinging wave through a weighting factor, w. This factor can follow either Lorentzian phase modulation (LPM), expressed as w= + e^θ/2 θ∈ [0,2π) or binary amplitude modulation (BAM), denoted as w∈{0,1} <cit.>. Notably, each unit-cell couples its signal to the neighboring microstrip line, resembling the structure of PCHPs. In DMAs, one end of each microstrip line connects to an RF chain while the other end is terminated to prevent standing waves.

Since their introduction, several aspects of DMA have been studied. For example, <cit.> studied the performance of a DMA in uplink wireless communications and showed that a LPM-constrained DMA can achieve a sum rate comparable to that of an unconstrained DMA. Furthermore, in <cit.>, the authors show that a DMA offers higher energy efficiency than a DPA or fully-connected HP.

When comparing antenna types, it is crucial to consider the hardware limitations across designs. Examples of these limitations are the antenna efficiency, the power consumed by the device components, and limited beam control due to the LPM constraint. While the  DMA literature often overlooks antenna efficiency and limitations, recent work <cit.> has considered the efficiency of DMAs and the effect of sampling from the guided mode. The authors propose a tri-hybrid architecture where a DMA is used in front of a PCHP in a downlink scenario. However, the full-wave simulation used in <cit.> provides accurate modeling for the specific simulated DMA design but lacks generalizability to other DMA variants. 

Notably, the impact of DMA’s hardware limitations on uplink performance remains unexplored. In the uplink, where users contend with limited transmitted power, it is imperative to account for antenna efficiency and hardware limitations since these factors significantly impact achieving optimal performance and reliability. Equivalently, for a fair comparison, one must design effective signal processing techniques to maximize the potential performance of the chosen system.

The primary limitation of using a DMA in the uplink stems from the loose coupling between unit-cells and the underlying microstrip line <cit.>. This loose coupling ensures that, in transmit mode, sampling from the guided mode and the introduced decay rate do not prevent the guided mode signal from reaching all unit-cells. The decay rate is controlled by the geometry and weighting factor of each unit-cell <cit.>, respectively. However, the static control of the decay factor significantly impacts DMA performance in the uplink. It restricts the amount of power coupled to the microstrip line, subsequently affecting the signal power at the RF chain and reducing the received signal-to-noise ratio (SNR).

Contributions: In this paper, we aim to precisely describe the hardware limitations of DMAs. Our goal is to compare the achievable sum rate when using a DMA with when using a PCHP in uplink wireless communications. Additionally, we propose a method to account for DMA’s limitations independently of its specific design. Notably, our approach requires characterizing a single unit-cell, rather than the entire DMA structure, through full-wave simulations. We propose a model to calculate the power consumption of different designs while accounting for the power consumed by the RF chain as well as the required driving circuitry for the unit-cell's configuration. 

For the signal processing, we propose an algorithm based on holography theory to design a WD combiner which maximizes achievable sum rate in the uplink of DMA-assisted antenna systems. We compare the achievable sum rate and energy efficiency of using DMAs with using PCHP under equal aperture area and equal number of RF chains. This comparison is in single- and multi-user scenarios in rich scattering and realistic 3GPP channels while accounting for hardware limitations in both designs. Our results reveal the shortcomings of DMA-assisted antennas in comparison with using PCHP.

Notation: Italics denote scalars. We use lowercase and uppercase boldface letters to denote vectors and matrices, respectively. (·)^† and (·)^H denote pseudo inverse and conjugate transpose, respectively. We represent the vector fields by A⃗. Also,  is defined as √(-1) and (x) = sin(π x)/π x. 𝒞𝒩(μ, 𝐑) represents the complex Gaussian distribution with mean μ and covariance matrix 𝐑.



§ SYSTEM AND DMA MODEL

System Model: We consider a BS equipped with a DMA comprising N_μ microstrip lines with N_e unit-cells on each microstrip linea total number of unit-cells N=N_μ N_e. The BS serves K users. The number of RF chains, M, is equal to N_μ, since each microstrip line connects to an RF chain at one end. The decoded signal at the BS is given by

    x = gW^D(G^DMA(√(P_T)Hs+z_ant)+z_RF)

where x∈ℂ^K × 1 is the received symbol vector at the BS, P_T is the transmitted signal power of each user (we assume equal power transmission), H∈ℂ^N × K is channel matrix, z_ant∈ℂ^N × 1 denotes the noise captured by the antenna from the environment <cit.>. Furthermore, g and z_RF∈ℂ^M× 1 are gain and noise terms introduced by the RF chain where z_RF∼𝒞𝒩(0,σ^2_RF). Also, via G^DMA∈ℂ^M × N and W^D ∈ℂ^K × M we, respectively, model the effect of the DMA's WD and digital combiner on the received signals.

DMA model: To model the DMA, we begin by pointing out its hardware limitations and the maximum weighting factor of each unit-cell.

As shown in Fig.<ref>, a DMA comprises N_μ microstrip lines separated by a  half-wavelength. Each line feeds the signal to (in transmit mode) or extracts the signal from (in receive mode) N_e unit-cells. Each unit-cell is loosely coupled to the microstrip line to prevent extensive decay of the propagating wave in the microstrip line <cit.> in transmitting mode. This requirement leads to the loss of a large portion of signal power transmitted by users in receiving mode. The coupling factor of each unit-cell is dictated by the dynamic state of the cell, average coupling factor, dielectric losses of unit-cells as well as their geometry and spacing <cit.>. Therefore, while one can maximize the dynamic component of the coupling factor of each unit-cell by setting it “on” (in BAM mode) or by setting its weighting factor to 1 (in LPM mode), there is a maximum coupling value dictated by the losses and geometry of the unit-cell. In what follows, we model this aspect of the coupling factor.

In a DMA, each unit-cell is modeled as a magnetic dipole with a magnetic polarizability defined by Lorentzian resonance <cit.> described as 

    α_m(ω) = F_mω^2/ω_0^2 - ω^2 +Γ_m ω,

where ω_0 and ω are the resonance and working angular frequencies, F_m is the coupling factor, and Γ_m is the damping factor =ω_0/2Q_m where Q_m is the quality factor of the unit-cell.

By ignoring the coupling between unit-cells (well-accepted as an accurate approximation <cit.>) the magnetic dipole formed by the i-th unit-cell is described as

    m⃗_i = α_m,iH⃗_i
    

where H⃗_i is the impinging magnetic field at the i-th unit-cell <cit.>. In the uplink, the transmitted signal by the users excites each unit-cell and hence, the i-th unit-cell magnetic dipole is proportional to the received  electromagnetic (EM) plane-wave described as <cit.>[Here we used the reciprocity theorem and described the user's transmitted EM field to be equal to the EM field radiated by the DMA as if the system is in the downlink. This ensures that the EM field coupled to the microstrip line is the dominant quasi-transverse EM (TEM) mode.]. For a single user

    H⃗_i = √(P_T)k_0^2 η_0 f(r⃗)ϕ̂

where P_T is the transmitted signal power, and k_0 and η_0 are free space wavenumber and impedance, respectively. In (<ref>) f(r⃗) captures the dependency of the magnetic field on the channel between the user and BS where r⃗ is their radial distance, for example in free space f(r⃗)= e^-k⃗·r⃗/4πr⃗.

Combining (<ref>)-(<ref>), the maximum magnetic dipole moment, m^DMA_max, occurring at the unit-cell's resonance frequency, is equal to

    m^DMA_max = 2 Q_m F_m k_0^2 η_0.

For example, in <cit.>, for a DMA with 30 unit-cells on each microstrip line and designed to operate at 3.5 GHz, F_m = 3.0× 10^-9 [m^3] and Q_m = 10.0. In this case, the maximum amplitude of the magnetic dipole formed by each unit-cell would be 0.09 [J/T].
which ensures the feed signal would reach all 30 unit-cells in the underlying microstrip line. This value is consistent with the maximum value of a DMA's weighting factor in BAM mode used in <cit.>. Note that F_m and Q_m can be extracted from the unit-cell design using the method introduced in <cit.>.

Note that considering the maximum magnetic dipole moment of DMA is crucial when designing signal processing techniques or comparing it with other antenna designs. This accounts for the practicality of the proposed technique and ensures a fair comparison.

Microstrip Model: As in <cit.>, we capture the effect of EM field propagation in the microstrip line and weighting factor of each unit-cell in matrices P^DMA and W^DMA, respectively. The DMA weighting matrix W^DMA is given by

    W^DMA = blkdiag(w^1, …, w^M),
    










where W^DMA is of size N_μ× N, N=N_μ N_e is the total number of unit-cells in DMA, w^i = [w^i_1, …,w^i_j, …, w^i_N_e] is a row vector representing the weight factors of unit-cells on the i-th microstrip line, and w^i_j is the weight factor of its j-th unit-cell. Each non-zero element of W^DMA follows either a BAM or LPM constraint:

    w^i_j∈𝒲^DMA≜{m^DMA_max +e^ϕ/2, ϕ∈ [0,2π]}   LPM
    {0, m^DMA_max}   BAM.


The propagation matrix P^DMA which captures the changes in EM fields due to the propagation in the microstrip line, is described by a diagonal matrix defined as

    P   ^DMA_(i-1)N_e+j = e^-(α_w + β_w)d_i,j ,
       ∀ (i,j) ∈{(i,j)|i ∈{1, …, N_μ}, j ∈{1, …, N_e}},

where d_i,j is the distance of the j-th element on the i-th microstrip line from the RF chain, α_w is the waveguide attenuation coefficient, and β_w is the microstrip line wavenumber.

Combining the two matrices, the WD combiner of DMA is described as

    G^DMA= W^DMAP^DMA.
    




Power consumption model: The consumed power by the DMA comprises two terms related to the RF chains and the needed power to configure unit-cells. We begin with the RF chains. Accounting for the main components in RF chainslow noise amplifier (LNA), mixer (Mix), clock distribution network (Clk), in-phase and quadrature demodulator (IQD), analog-to-digital converter (ADC), and its driver (drv), as shown in Fig. <ref>, the power consumed, P_RF for one RF chain is given by[A complete block diagram of RF chain can be found in <cit.>.]

    P_RF = P_LNA+P_Mix+P_Clk+P_IQD+2(P_ADC+P_drv)


Note that while P_LNA is usually considered to be fixed <cit.>, it contains a static term due to LNA's internal circuitry, P_sLNA, and a dynamic term depending on the input power, P_in, and power-added-efficiency, η_LNA <cit.>. Therefore, we model an LNA's power consumption as P_LNA = max (P_sLNA, G_LNA-1/η_LNAP_in) where G_LNA is the LNA gain.

To calculate the needed power to configure the DMA unit-cells, we need to carefully account for components in driving circuitry. The properties of each unit-cell are controlled via the voltage applied to the implemented varactor diode. The digital-to-analog converter (DAC) driven varactor diodes provide modulation of the unit-cell's weighting factor by translating the applied voltage to an adjustable capacitance. However, more than the power consumed by the DAC, P_DAC, the data line driving the DAC, P_Ctrl determines the power consumed.  Finally, there is a static power consumption by the core of the DMA's field programmable gate array (FPGA), P_FPGA. We assume that all the processing is handled in the main FPGA/processor and the DMA's FPGA only drives the varactors (via the DAC) consuming limited power. Hence, the total power consumption of the configuration is 

    P_DMA =     N(P_DAC+P_Ctrl)+P_FPGA

where N is the number of unit-cells. 

We note that the power consumed by the RF chains and configuring circuitry of the DMA, as defined in (<ref>) and (<ref>) is independent of the DMA state, and only depends on the received power by the antenna via P_LNA.





§ MAXIMIZING THE ACHIEVABLE SUM RATE


The achievable sum rate, R, for a DMA-assisted antenna in the uplink is defined as R = ∑_k=1^K log_2(1+γ_k) where γ_k is the signal-to-interference-plus-noise ratio (SINR) for user k
    γ_k =P_TG_k,:h_:,k^2/∑_j≠ k^K P_TG_k,:h_:,j^2 + z,

where we defined G = W^D G^DMA and z=σ_ant^2(G_k,:Σ_rx(G_k,:)^H)+σ_RF^2 W^D_k,:^2 represents the noise terms comprising external noise with power σ_ant^2, colored by the WD combiner, and the thermal noise added by the RF chain of power σ_RF^2 <cit.>. 

To maximize the achievable sum rate, we are interested in solving the following optimization problem:

	
	
    max_W^D, W^DMA R
    s.t.    W^DMA = blkdiag(w^1, …, w^M)
       w^i_j ∈𝒲^DMA.


where 𝒲^DMA is defined in (<ref>). Solving (<ref>) is not trivial due to the non-convexity of objective function and constraints. To obtain an effective solution, adopting holography theory <cit.>, we first find a desired combiner, G^des assuming the existence of a virtual DPA at the location of the DMA. This is a trivial task using the massive MIMO literature; for example, in single-user and multi-user scenarios, a good choice for G^des is maximum ratio combining (MRC) and zero-forcing (ZF), respectively, when noise in the system is white. Knowing G^des, we are interested in finding W^DMA and W^D that are solutions to the following optimization problem.

    min_W^D, W^DMA   G^des - W^D W^NWW^DMAP^DMA
    s.t.    (<ref>), (<ref>)

where, since G^des is a good choice for systems with added white noise and in DMA we are dealing with colored noise, W^NW = Σ_z^-1/2 is introduced as a noise whitening filter where Σ_z is the noise correlation matrix.
While (<ref>) is still non-convex jointly in W^D and W^DMA, adopting an alternative optimization approach, similar to <cit.>, we find an optimum answer for W^D and W^DMA. 

WD combiner: Given W^D, various solutions have been proposed to obtain W^DMA. Here, we use the non-zero mapper (NZM) and the closed-form mapper (CFM) as introduced in <cit.> and <cit.>. Specifically, defining G^D = W^D W^NW, NZM finds the closest WD combiner to G^des by normalizing the Frobenious norm between (G^D)^†G^des(P^DMA)^-1 and W^DMA entry-wise. On the other hand, in CFM, knowing G^D a closed-form solution for elements of W^DMA that minimizes Frobenious distance between G = G^des(P^DMA)^-1 and G^D W^DMA is calculated as w^i_j = (G^D)_i^H (G)_i,j/(G^D)_i^2 where (G)_i,j and G^D_i denote the j-th column of the i-th M× N_e block of G and i-th column of G^D. Then w^i_j is first mapped to the unit-circle followed by projection to the Lorentzian circle to obtain its final value.

Conversely, given W^DMA to calculate the digital combiner, we adopt the following approaches:

    
  * Single-user case: In this case, with W^DMA known, the best solution is applying a noise whitening filter W^NW followed by match filtering defined as W^D = (W^NWW^DMAP^DMAH)^H
    
  * Multi-user case: Knowing W^DMA, an optimal choice for W^D is the least-squares solutionW^D = G^des(W^NWW^DMAP^DMA)^†.


The process of alternatively optimizing between W^DMA and W^D continues until convergence.



§ NUMERICAL STUDY




In this section, we compare the sum rate when using a DMA with using a PCHP, considering the hardware issues described above. We examine equal aperture areas, RF chains M, single-user and multi-user scenarios, in an outdoor wireless communication system where K users communicate with a base station at 3 GHz carrier frequency with 20 MHz bandwidth. As the BS is in the far-field of users, it receives a plane wave with Poynting vector magnitude equal to P_T  E⃗_user^2/2η_0 [W/m^2] where P_T is the power transmitted by users. 

We assume both PCHP and DMA have physical apertures equal to 2λ_0 × 8λ_0 and M=4 RF chains. The PCHP is an array of patch antennas separated by λ_0/2 with 90% efficiency <cit.>. Also, the unit-cells of the DMA are of size λ_0/2 ×λ_0/6 <cit.>. This configuration translates to N_s and N_e equal to 16 and 48, respectively. Furthermore, the RF chain is at room temperatureT_BS = 290  [K]. For a microstrip line as described in <cit.> α_w and β_w are set 0.13 and 113.8 [m^-1], respectively. Finally, g and the noise figure of the RF chain are set to be 12.5 [dB] and 18.8 [dB], respectively. Also, to form a clear picture of the effect of hardware limitations, we report the performance of DPA with N = M=4. We repeat the simulation for 10 instances of user placements and 100 channel realizations per instance.

The power captured by each antenna is equal to P_T A_eff where A_eff is the antenna effective area <cit.>. Each unit-cell is approximated by a continuous current sheet, hence, A_eff,UC is equal to its physical area <cit.>. In the case of PA, using Matlab's Antenna Designer app <cit.>, a typical patch antenna is designed and its effective area is calculated.

The noise power captured by the antenna is given by 

σ_ant^2 = KT_EBA_eff/λ_0^2ΔΩ <cit.> 

where K, T_E, and ΔΩ are Boltzmann's constant, environment temperature (290  K) <cit.>, and the solid angle of noise sources equal to 2 π.

For rich scattering environments, we use the Kronecker Rayleigh fading model where H = Σ_rxĤΣ_tx, Ĥ∼𝒞𝒩(0,I_NK), and Σ_tx and Σ_rx are transmitters'users', and receivers'antennas', correlation matrices, respectively. We assume independent channel between usersΣ_tx = diag(β_1, …, β_K) where β_k is channel path loss of k-th user computed using the Winner II urban microcell (UMi) non-line of sight (NLoS) scenario <cit.>. To calculate Σ_rx, we assume scatterers are distributed in 3D space and hence, ()_(i,j) = (2d_i,j/λ_0) <cit.> where d_i,j is the distance between i-th and j-th unit-cells. Also, to evaluate the achievable sum rate of different antenna designs in realistic 3GPP channels, we use channels generated by QuadRiGa <cit.>. In realistic channels the external noise correlation matrix is equal to that in a rich scattering environmentz_ant∼(0,σ_ant^2  ).

In single-user and multi-user scenarios, MRC and ZF combiners are used for the DPA. The PCHP's combiner is obtained using the algorithm in <cit.>. Finally, to account for hardware limitations in PCHP, we assume a k-way Wilkinson combiner is formed by cascading ⌈N/M⌉ 2-way Wilkinson combiners <cit.>. The insertion loss of the Wilkinson power combiner and phase shifter (PS) is 3.9 [dB] <cit.> and 5 [dB] <cit.>, respectively.

Single-user scenario: Depicted in Fig. <ref> is the achievable sum rate of different designs in the single-user case. As shown in Fig. <ref>, when hardware limitations are not factored in, the PCHP and DMA achievable sum rate surpasses that of DPA with equal M. In PCHP, PSs can fully compensate for the phase differences in signal captured by different antennas, while in DMA, due to LPM (or BAM) constraints, signals captured by different unit-cells are not completely in phase. This explains the performance gap between DMA and PCHP. In contrast, when hardware limitations are accounted for in rich scattering channels, as illustrated in Fig. <ref>, the achievable sum rate of the PCHP is close to that of the DPA and the DMA's performance drops dramatically. The superior performance of PCHP with respect to DPA is mainly due to its larger physical aperture. which leads to higher signal power captured from the environment. 

Due to the small coupling factor between unit-cells and microstrip lines in the DMA structure, most of the signal power illuminating the antenna's aperture is not coupled to microstrip lines leading to poor performance of the DMA in comparison to DPA and PCHP.

Finally, the achievable sum rate of different designs in realistic 3GPP channels is demonstrated in Fig. <ref>. In these selective channels, while the performance of all designs drops, DPA experiences the most degradation in its achievable sum rate due to its smaller aperture area. Furthermore, the achievable sum rate of the BAM-constrained DMA case is extremely low which makes this design inappropriate.


Multi-user scenario: The performance of different antenna designs in a multi-user scenario where BS serves 2 users is depicted in Fig. <ref>. Similar to the single-user case, when hardware limitations are not accounted for, as shown in Fig. <ref>, both PCHP and the DMA achieve higher sum rates than the DPA due to their larger number of antennas and available flexibility to form more complex patterns. Also, the DMA's performance in LPM mode using NZM surpasses that of PCHP mainly due to dense unit-cells in the DMA structure and the flexibility it provides to nullify inter-user interference. Moreover, the achievable sum rate of the DMA in BAM mode has the same trajectory as that of PCHP. In contrast, when accounting for hardware limitations, in rich scattering channels, the DPA's achievable sum rate surpasses other designs due to its lower signal losses and, hence, higher SINR. On the other hand, in realistic 3GPP channels, PCHP's performance is better than the DPA due to its larger effective aperture and increased signal power. Here, the DMA's sum rate is close to that of the DPA, due to its large aperture area but lower than PCHP due to higher losses in the system. 

Energy Efficiency: Energy efficiency (EE) is defined as EE = R/P. The power consumption of the DPA and PCHP is due to M RF chains calculated using (<ref>). In contrast, DMA power consumption is due to RF chains and unit-cell's configuring circuitry as described in (<ref>). To calculate consumed power in each case, P_sLNA <cit.>, P_Mix <cit.>, P_IQD <cit.>, P_drv <cit.>, P_ADC <cit.>, P_Clk <cit.>, P_DAC <cit.>, P_Ctrl <cit.> and P_FPGA <cit.> are set as 0.75 [W], 0.4 [W], 2.2 [W], 0.15 [W], 0.725 [W], 0.1 [W], 0.002 [W], 0.0006 [W], and 0.1 [W], respectively. Finally, η_LNA is 0.12 <cit.>. Note that in calculating P_LNA, we assume the power received by each antenna is equal to β_A P_T A_eff where β_A is the average path-loss of the users. Also, in a DMA/PCHP, the total power captured by N_s/N_e antennas enters each LNA while in DPA, the power captured by one antenna forms P_in of the LNA.





























Depicted in Fig. <ref>, is the EE in rich scattering channels. As seen, the EE shows the same trend as the achievable sum rate. This is because the major contributor to the power consumption of the antenna is the number of RF chains, M, which is equal across all compared designs. Also, the DMA's power consumption is slightly larger than the other two antenna designs due to the required circuitry to configure the state of unit-cells. Note that combining a DMA with HPs, as in <cit.>, would result in a more energy-efficient design. 


































§ CONCLUSION

In this paper, we studied the impact of the hardware limitations of a DMA in uplink wireless communications. Our results show that the hardware limitations of DMA drastically affect its achievable sum rate in comparison with PCHP and DPA with an equal number of RF chains. This underlines the importance of accounting for the practical limitations of DMA, indeed any other antenna design, when used in a wireless communication scenario.



§ ACKNOWLEDGMENTS


This work was supported by the Ericsson Canada. 


IEEEtran




 Paper Title: Learning to Embed Distributions via Maximum Kernel Entropy.




Floquet Engineering of Polaritonic Amplification in Dispersive Photonic Time Crystals
    Vladimir M. Shalaev
    August 5, 2024
=====================================================================================




§ ABSTRACT 
Empirical data can often be considered as samples from a set of probability distributions. Kernel methods have emerged as a natural approach for learning to classify these distributions. Although numerous kernels between distributions have been proposed, applying kernel methods to distribution regression tasks remains challenging, primarily because selecting a suitable kernel is not straightforward. Surprisingly, the question of learning a data-dependent distribution kernel has received little attention. In this paper, we propose a novel objective for the unsupervised learning of data-dependent distribution kernel, based on the principle of entropy maximization in the space of probability measure embeddings. We examine the theoretical properties of the latent embedding space induced by our objective, demonstrating that its geometric structure is well-suited for solving downstream discriminative tasks. Finally, we demonstrate the performance of the learned kernel across different modalities.




§ INTRODUCTION

Most discriminative learning methods conventionally assume that each data point is represented as a real-valued vector. In practical scenarios, however, data points often manifest as a 'set' of features or a 'group' of objects. A quintessential example is the task of predicting a health indicator based on multiple blood measurements. In this case, the single data point of a patient has multiple, or a distribution of, measurements. One approach to accommodate such cases involves representing each input point as a probability distribution. Beyond mere convenience, it is more appropriate to model input points as distributions when dealing with missing data or measurement uncertainty, as often encountered when facing the abundance of data, which commonly presents a challenge for data-rich fields such as genetics, neuroscience, meteorology, astrophysics, or economics.


The task of regressing a mapping of probability distributions to a real-valued response is known as distribution regression. Distribution regression has been successfully applied in various fields, such as voting behavior prediction <cit.>, dark matter halo mass learning <cit.>, human cancer cells detection <cit.>, brain-age prediction <cit.>, among others <cit.>. The versatility and effectiveness of this framework underscore its power in solving complex problems <cit.>. Kernel methods have become a widely used approach for solving distribution regression tasks by exploiting a kernel between distributions  referred to as a distribution kernel. Despite the multitude of proposed kernels, the practical application of kernel methods remains challenging due to the nontrivial choice of the appropriate kernel. While some efforts focus on identifying kernels with broad applicability and favorable statistical properties <cit.>, others aim to tailor kernels to the geometric characteristics of specific input spaces <cit.>. Remarkably, the question of learning data-dependent kernels has received limited attention. This study is thus driven by a fundamental question: What are the underlying principles that facilitate the unsupervised learning of an effective kernel, one that optimally encapsulates the data properties and is well suited for discriminative learning on distributions?

In this work, we leverage a key insight: an appropriate selection of the distribution kernel enables the embedding of a set of distributions into the space of covariance operators. Building on this theoretical idea, we claim that quantum entropy maximization of the corresponding covariance operator is a suitable guiding principle to learn data-dependent kernel. This, combined with a careful design of kernel parametrization, let us to devise a differentiable optimization objective for learning data-specific distribution embedding kernels from unlabeled datasets (i.e. unsupervised) (<Ref>). We show that the entropy maximization principle facilitates learning of the latent space with geometrical configuration suitable for solving discriminative tasks <cit.>. We empirically demonstrate the performance of our method by performing classification tasks in multiple modalities.

In summary, our unsupervised data-dependent distribution kernel learning framework introduces a theoretically grounded alternative to the common practice of hand-picking kernels. Such framework could be further leveraged for generalizing existing learning approaches <cit.> catalyzing the use of distribution-based representations within the broader scientific community.



§ PRELIMINARIES

We first introduce the main concepts necessary to formalize our learning framework: kernel mean embeddings and covariance operator embeddings.



 §.§ Kernel Embeddings of Distributions


Consider an input space 𝒳 and a positive-definite (p.d.) kernel k: 𝒳×𝒳→ℝ. Let ℋ be the corresponding reproducible kernel Hilbert space (RKHS) induced by such kernel. Consider a probability distribution P∈𝒫(𝒳). The kernel mean embedding map embeds the distribution P as a function in Hilbert space:


    μ_P≡μ(P) ∫_𝒳 k(x, ·)   dP(x) = ∫_𝒳ϕ(x)    dP(x) ,

where ϕ: 𝒳→ℋ is a feature map such that ϕ(x) = k(x, ·).

Importantly, if the kernel k is characteristic <cit.>, the mapping μ: 𝒫(𝒳) →ℋ is injective, implying that all information about the original distribution is preserved in ℋ. This last property underscores much of power under recent applications of kernel mean embeddings <cit.>. 
The natural empirical estimator for the kernel mean embedding approximates the true distribution with a finite sum of Dirac delta functions:

    μ̂_P1/N∑_i=1^N ϕ(x_i) ∈ℋ

where x_1, …, x_N ∼P are N empirical i.i.d. samples. The estimator has a dimension-free sample complexity error rate of 𝒪(N^-1/2) <cit.>.

Additionally, we denote d_k: 𝒳×𝒳→ℝ_≥ 0 a kernel metric induced by a kernel k,

    d_k(x, x^') = ϕ(x) - ϕ(x^') _ℋ  .

Note that d_k is a metric on 𝒳 if feature map ϕ is injective.



 §.§ Covariance Operators and Entropy

A second way of mapping a probability distribution to a Hilbert space can be defined by means of a covariance operators. For a given feature map ϕ(x) = k(x, ·): 𝒳→ℋ[Assuming k to be a continuous positive definite (p.d.) kernel and 𝒳 to be compact.], and a given probability distribution P∈𝒫(𝒳), the covariance operator embedding is defined as:

    Σ_P∫_𝒳ϕ(x) ⊗ϕ(x)   dP(x)

where ⊗ is a tensor product. Σ_P is a self-adjoint positive semi-definite (p.s.d.) operator acting on ℋ. Such operator can be seen as a mean embedding w.r.t. the feature map x ↦ϕ(x) ⊗ϕ(x) and therefore, for a universal kernel k, the map P↦Σ_P is injective (see <cit.>).

Similarly to <Ref>, the natural empirical estimator is:

    Σ̂_P = 1/N∑_i=1^Nϕ(x_i) ⊗ϕ(x_i)

where x_1, …, x_N ∼P are N i.i.d. samples.

For a translation-invariant kernel k(x, x^') = ψ(x - x^') normalized such that k(x, x) =  1, the covariance operator Σ_P is a density operator <cit.>. Henceforth, entropy measures can be applied to it, and the quantum Rényi entropy of the order α can be defined as:

    𝒮_α(Σ_P) 1/1 - αlogtr[ (Σ_P)^α]  = 1/1 - αlog∑_iλ_i^α

where {λ_i }_i are the eigenvalues of Σ_P.
The Von Neumann entropy can be seen as a special case of Rényi entropy in the limit α→ 1. However, in our work, we focus primarily on the second-order case of Rényi entropy, i.e. α = 2 (see <cit.> for an in-depth overview of the properties and theory of quantum entropies).



§ UNSUPERVISED DISTRIBUTION KERNEL LEARNING



 §.§ Distribution Regression


In this section, we discuss the key topics in distribution regression, including problem setup, the notion of a 2-stage sampling process, and the common solutions to regression employing kernel methods.

Distribution regression extends the common regression framework to the setup where covariates are given as probability distributions available only through samples. Formally, consider the task of finding a regressor f: 𝒫(𝒳) →𝒴 from the dataset of samples Γ_M = {(P_i, y_i)}_i=1^M where P_i ∈𝒫(𝒳) are distributions provided as a set of i.i.d. empirical samples x_1, …, x_N_i∼P_i (see <cit.> for a comprehensive analysis). A viable approach to solving this problem is to define a kernel K: 𝒫(𝒳) ×𝒫(𝒳) →ℝ that is universal in 𝒫(𝒳). By setting up such a kernel K, we can utilize kernel-based regression techniques, with SVM often being the preferred method for classification tasks <cit.> or Kernel Ridge Regression (KRR) when the space 𝒴 is continuous <cit.> [As pointed out in <cit.>, under mild conditions, KRR can also be used for classification problems.]. To this end, several kernels have been proposed over time (see details in <Ref>). 

One possibility, proposed by <cit.>, is to introduce a kernel in the input space, the so called embedding kernel k_emb: 𝒳×𝒳→ℝ and exploit the induced mean embeddings μ_emb: 𝒫(𝒳) →ℋ_emb to map input distributions to points in RKHS. Subsequently, to define a second level kernel, distribution kernel K_distr: ℋ_emb×ℋ_emb→ℝ between points in the RKHS ℋ_emb. The simplest choice for such a distribution kernel is the linear kernel:

    K_l(μ_P, μ_Q) ⟨μ_P, μ_Q⟩_ℋ_emb
    = ∬_𝒳×𝒳 k(x, x^')   d P(x)   d Q(x^') .

A standard alternative to the linear kernel is a Gaussian kernel with bandwidth parameter λ > 0:

    K_RBF(μ_P, μ_Q) exp( - λ/2‖μ_P - μ_Q‖_ℋ_emb^2 ) = exp( - λ/2 d_k_emb(μ_P, μ_Q)^2 )   ,

which was shown to be universal in 𝒫(𝒳) <cit.>. The Gaussian kernel K_RBF can be computed from the linear kernel K_l as d_k_emb(μ_P, μ_Q)^2 = K_l(P, P) + K_l(Q, Q) - 2 K_l(P, Q).

In practice, we only have access to a finite number of samples of each input distribution, thus the distribution kernel is approximated using the natural estimator for the kernel mean embedding. The related excess risk for the regression solution is analyzed in <cit.>.





 §.§ Dataset Embedding


Instead of using standard kernels designed to encapsulate the geometry of the input space, we consider learning a data-dependent kernel, tailored to the specific properties of the dataset. In a similar vein, <cit.> proposed learning an optimal kernel (or equivalently, a feature map) jointly with the classifier to address the text modality. In this work, we focus on an unsupervised problem, aiming to learn a data-dependent kernel between probability distributions without access to classification labels.

We first introduce proper parametrization to ensure both expressivity and robustness followed by the definition of the optimization objective. Leveraging the idea of 2-level kernel setup, we define the embedding kernel as

    k_θ: 𝒳×𝒳→ℝ = k_emb(f_θ(x), f_θ(x^')) .

where f_θ is a trainable encoder function f_θ: 𝒳→𝒵, 𝒵 is a latent encoding space, and k_emb is a kernel defined on the latent space k_emb: 𝒵×𝒵→ℝ. The encoder function f_θ transforms every input probability distribution P∈𝒫(𝒳) into a distribution over the latent space ℙ_θ∈𝒫(𝒵) [By the definition of the encoding process, P_θ is a push-forward measure. For an empirical probability distribution q=∑_iδ(x_i)∈𝒫(𝒳) and a measurable map f: 𝒳→𝒵, the push-forward measure f # q ∈𝒫(𝒵) is defined as ∑_iδ(f(x_i)).] (<Ref>). Furthermore, we denote RKHS corresponding to the kernel k_emb as ℋ_emb and the kernel mean embedding map as μ_emb (see <Ref>).

    P∈𝒫(𝒳)  P_θ∈𝒫(𝒵)  μ_emb(P) ∈ℋ_emb .

These transformations define mean embeddings for each input probability distributions through the first level, embedding kernel k_emb (<Ref>). The second level, distribution kernel K_distr: ℋ_emb×ℋ_emb→ℝ is defined over the mean embeddings μ_emb(P)'s. We can now consider the embedding dataset 𝒟_M = {P_i }_i=1^M as an empirical covariance operator (see <Ref>), i.e.

    𝒟_M Σ̂_𝒟 = 1/M∑_P∈𝒟_MK_distr(μ_emb(P), ·) ⊗ K_distr(μ_emb(P), ·)  .

As Σ̂_𝒟 encapsulates information about the entire dataset, we term it dataset embedding. With the assumption that the dataset 𝒟_M is sampled i.i.d. from the (unknown) true meta-distribution 𝒟, Σ̂_𝒟 is a natural estimator to approximate the true covariance operator Σ_𝒟. To simplify the notation we use Σ_𝒟 in place of Σ_𝒟 unless required by the context.

Both the embedding kernel k_emb and the distribution kernel K_distr remain fixed throughout the training, learning happens by adjusting the parametrization of the latent space encoder f_θ. Such separation ensures expressivity while conforming to all technical requirements for a distribution kernel. Throughout the paper, we make the following assumptions on the latent space 𝒵 and embedding kernel k.

Latent space 𝒵 is a compact subset of ℝ^d. Kernel k_emb: 𝒵×𝒵→ℝ is a p.d. characteristic translation-invariant kernel k_emb(z, z^') = f( z - z^'^2) such that -f^' is completely monotone on (0, ∞) (see Definition 2.2.4 of <cit.>) and ∀ z ∈𝒵:   k_emb(z, z) = 1.



To facilitate our optimization process (which will be explained shortly), we opt for 𝒵=𝕊^d-1 (the d-dimensional hypersphere) and Gaussian kernel both for k_emb and K_distr (see <Ref>). We retain other suitable choices as potential avenues for future research.



 §.§ Unsupervised Optimization Objective


This dataset level representation depends on the choice of first and second level kernels k, K and, in turn, on the trainable function f_θ parameterized by the set of parameters θ. In this work, we propose learning the parameters θ to maximize quantum entropy of the dataset embedding, i.e.,

    θ = {𝒮_2(Σ_𝒟)  - logtr[ (Σ_𝒟)^2 ] } .

As we will describe in brief, optimizing this target has clear benefits inherited from the underlying geometry of the setup. But, first, we show how to empirically compute 𝒮_2(Σ_𝒟). Building upon previous work <cit.>[See the proof of Proposition 6 in <cit.>.], we exploit the following property of the covariance estimator:

    tr[ (Σ_𝒟)^2 ] = tr[ ( 1/M K_𝒟)^2 ]

where K_𝒟∈ℝ^M × M is the distribution kernel matrix, with [K_𝒟]_ij = K_distr(μ_P_i, μ_P_j).  This equation follows directly from the fact that Σ_𝒟 and 1/M K_𝒟 share the same set of eigenvalues. Leveraging this relationship, we can define tractable unsupervised training loss, which we term Maximum Distribution Kernel Entropy (MDKE), with respect to the parameters of the encoder f_θ:

    ℒ_MDKE(θ)   - 𝒮_2(Σ_𝒟) = logtr[ ( 1/M K_𝒟)^2 ]  = log∑_i=1^Mλ_i^2 ( 1/M K_𝒟)  = log1/M K_𝒟_F^2

where the latter relies on the fact that the Frobenius norm A _F^2 = ∑_iλ_i^2(A), where λ_i(A) are eigenvalues A.

The MDKE objective is differentiable w.r.t. θ for commonly used kernels, provided that the encoder f_θ is differentiable as well. While the entropy estimator 𝒮_2(Σ_𝒟) is convex in the kernel matrix K_𝒟, the objective as a whole is generally not convex in θ. However, in practice, as we show in <Ref>, mini-batch Stochastic Gradient Descent (SGD) proves to be an effective method for optimizing this objective. The effectiveness of this optimization process is significantly influenced by the parameters of the Gaussian kernels. We elaborate on the methodologies for kernel bandwidth selection in <Ref>.

The Frobenius norm formulation in the loss <Ref> significantly reduces computational complexity. However, as we have observed in some of our experiments, it can lead to the collapse of small eigenvalues of K_𝒟, particularly near the optimal value of the objective. To address this challenge we introduced a regularized version of the loss ℒ_MDKE-R that incorporates optional regularization, based on the determinant K_𝒟 inspired by the connection with Fekete points (see details in <Ref>).



 §.§ Geometrical Interpretation
 

The optimization objective is specifically designed to minimize the variance within each distribution (inner-distribution variance) while simultaneously maximizing the spread of distributions over the compact latent space 𝒵=𝒮^d-1. This shaping of the distributions embeddings in the latent space facilitates easier separation in downstream tasks. In this section we show that the geometry of the optimal (w.r.t. the MDKE loss) configuration of mean embeddings in the RKHS attains describe properties. For doing so we leverage the notion of distributional variance  𝕍_ℋ (Definition 1 in <cit.>). 


For a set of M probability distributions 𝒟_M, distributional variance 𝕍_ℋ(𝒟_M) of the mean embeddings in the RKHS ℋ is given by

    𝕍_ℋ(𝒟_M) 1/Mtr[ G ] - 1/M^2∑_i=1^M∑_j=1^MG_ij ,

where G is the M × M Gram matrix of mean embeddings in ℋ, i.e. G_ij = ⟨μ_P_i, μ_P_j⟩_ℋ <cit.>.


Here we show that the distributional variance 𝕍_ℋ can be equally reformulated into two separate contributions:

    𝕍_ℋ(𝒟_M) ≡1/M∑_i=1^M‖μ_P_i‖_ℋ^2  - ‖μ_P̅‖_ℋ^2 ,

where P̅ denotes mixture distribution with elements of 𝒟_M being uniformly weighted mixture components (see proof in the <Ref>).

The relevance of distributional variance for MDKE objective is established by the following result. 


For a set of M probability distributions 𝒟_M, the second-order Rényi entropy 𝒮_2 of the empirical covariance operator embedding Σ̂_𝒟 induced by the choice of Gaussian distribution kernel K_RBF over points in the RKHS ℋ_emb, - as defined in <Ref>, - is upper bounded by the distributional variance 𝕍_ℋ_emb(𝒟_M), i.e.,


    1/2 γ𝒮_2(Σ̂_𝒟) ≤𝕍_ℋ_emb(𝒟_M)


where γ is the bandwidth of the distribution kernel K_RBF.

The proof of this proposition is provided in <Ref>. This result formalizes the fact that our objective increases distributional variance, pushing up the average squared norm of mean embedding of input distributions while minimizing squared norm of the mean embedding of the mixture. We further explore the geometrical implications of such optimization by formalizing connection between the variance of the distribution and the squared norm of the its mean embedding in RKHS.


Under <Ref>, the maximum norm of kernel mean embedding is attained by Dirac distributions {δ_z }_z ∈𝒵.

This result is trivial due to the fact that the set of mean embeddings is contained in the convex hull of { k_emb(z, ·) }_z ∈𝒵, and, under <Ref>, ∀ z ∈𝒵:  k_emb(z, ·) _ℋ_emb^2 = k_emb(z, z) = 1.


Under <Ref>, uniform distribution 𝒰(𝒵) is a unique solution of

    _P ∈𝒫(𝒵){μ_emb(P) _ℋ_emb^2 ≡∬_𝒵×𝒵 k_emb(z, z^')   d P(z)   d P(z^') }.


The key intuition here comes from the fact that minimization of the squared norm of mean embedding in RKHS could be seen as minimization of total interaction energy over the given surface where the potential is defined by the kernel k. Thus <Ref> is a special case of Theorem 6.2.1 of <cit.>. Similar setup was used w.r.t. Gaussian potential over the unit hypersphere in Proposition 1 from <cit.>. The reformulation in <Ref> together with <Ref> immediately suggests that the framework could be seen as an extension of the Hyperspherical Uniformity Gap <cit.> to infinite-dimensional spaces of probability distributions This extension maintains the goal of reducing variance among input distributions while maximizing the separability between their means. See <Ref> for a broader explanation of the connection.

More generally, utilizing the fact that under <Ref>, the kernel metric d_k_emb (see <Ref>) is a monotonically increasing function of Euclidean metric on the latent space, we establish a precise connection between the generalized variance <cit.> and the norm of the  mean embedding. Further details can be found in <Ref>.

An attempt to directly optimize 𝕍_ℋ_emb using SGD resulted in significantly weaker outcomes. While a thorough mathematical explanation necessitates further investigation, we contend that this issue aligns with the recurring challenge reported across various studies regarding direct optimization over Maximum Mean Discrepancy (MMD). We hypothesize that optimal solutions exhibit similar geometric configurations, while the entropy of the covariance operator providing a smoother objective. Nonetheless, 𝕍_ℋ_emb retains its value as an insightful and intuitive measure for describing the geometric configuration of the learned system.



 §.§ An Illustrative Example

We use a simple example to illustrate the connection between geometrical configurations of embedded distributions and distribution kernel entropy 𝒮_2(Σ_𝒟) (see <Ref>). We sample a number of points from 6 different Gaussian distributions and project on a sphere 𝕊^2 varying their projected variance γ. As γ decreases, the distributional variance of the overall distribution of Gaussians increases (<Ref>). For very small γ each distribution converges to a point (a Dirac distribution). This results in the entropy interpolating between lower and upper bounds, demonstrating how entropy behaves in response to changes in distribution variance. <Ref> showcases the behavior of two terms comprising distributional variance (<Ref>): the average kernel norm of the distributions alongside the kernel norm of the mixture. The increase in entropy and variance corresponds to a 'flattening' effect on the spectrum of the distribution kernel matrix. This example provides a simplified picture of how input distributions configurations influence kernel entropy.




 §.§ Limitations


Runtime complexity. The applicability of a data-dependent distribution kernel to solving discriminative tasks relies on the structure of the dataset being well-suited for distribution regression modeling. The model performs best when the number of input distributions is relatively small (e.g., thousands rather than millions), while the number of samples per distribution is large. It is crucial to note that the computational complexity of the proposed method, which is a common concern in practical applications, is most favorable for the tasks described. A detailed analysis of runtime complexity can be found in <Ref>.

Broader impact. We wish to emphasize that the distribution regression framework has emerged as a powerful tool for analysis and predictive modeling, especially in domains where traditional methods face challenges, including social science, economics, and medical studies. We urge researchers and practitioners applying distribution regression in these areas to give special consideration to issues such as bias, fairness, data quality, and interpretability, -  aspects that are currently under-researched in the context of distributional regression, largely due to the relative novelty.



§ RELATED WORK





 §.§ Distribution Regression

Distribution regression was introduced in <cit.>, while the seminal work of <cit.> provides a comprehensive theoretical analysis of this regression technique. A natural approach to solving distribution regression problems involves using kernels between measures. Notable examples include the Fisher kernel <cit.>, Bhattacharyya kernel <cit.>, Probability product kernel <cit.>, kernels based on nonparametric divergence estimates <cit.>, and Sliced Wasserstein kernels <cit.>. <cit.> proposed leveraging the mean embedding of measures in RKHS, and <cit.> provided theoretical guarantees for learning a ridge regressor from distribution embeddings in Hilbert space to the outputs. Distribution kernels have been successfully applied in various kernel-based methods, such as SVM <cit.>, Ridge Regression <cit.>, Bayesian Regression <cit.>, and Gaussian Process Regression <cit.>. They have also been adapted for different modalities like distribution to distribution regression <cit.>, sequential data <cit.>, and more. For an in-depth exploration of the diverse methodologies employed in distributional regression settings, we invite readers to consult <ref>.



 §.§ Matrix Information Theory

Quantum entropy, including Rényi entropy, is a powerful metric to describe information in a unique way (see <cit.> for foundational insights). <cit.> designed the measure of entropy using operators in RKHS to mimic Rényi entropy's behavior, offering the advantage of direct estimation from data. <cit.> applied von Neumann entropy of the density matrix to the covariance operator embedding of probability distributions, thereby defining an information-theoretic framework utilizing kernel methods. In machine learning, especially within self-supervised learning (SSL) setups, entropy concepts have recently found novel applications. Our study builds on most recent developments <cit.> by applying quantum Rényi entropy to the covariance operator in RKHS.



§ EXPERIMENTS


We here demonstrate that our proposed method successfully performs unsupervised learning of data-dependent distribution kernel across different modalities. The experimental setup is divided into two phases: unsupervised pre-training and downstream regression classification using the learned kernel.

For each dataset, we select a hold-out validation subset with balanced classes, while the remainder of the dataset is utilized for unsupervised pre-training. We use mini-batch ADAM <cit.> with a static learning rate of 0.0005. We report mini-batch based (instead of epoch based) training dynamics as our tasks do not require cycling over the entire dataset to converge to the optimal loss value. All experiments use Gaussian kernel both as an embedding kernel and distribution kernel, the hyperparameter selection is performed as described in <Ref>.

Once the samples encoder f_θ is learned, we employ it to compute distribution kernel Gram matrix, used as an input to the Support Vector Machine (SVM) for solving downstream classification tasks. A grid search with 5 splits (70/30) is conducted to optimize the strength of the squared l_2 regularization penalty C, exploring 50 values over the log-spaced range {10^-7, …, 10^5}. The best estimator is then applied to evaluate classification accuracy on the validation subset, which we report.

Additional experiments exploring the application of data-dependent distribution kernels in domains where distribution regression models are less common, such as image and text, are presented in <Ref>.



 §.§ Flow Cytometry



Flow Cytometry (FC) is a widely used technique for measuring chemical characteristics of mixed cell population. Because population-level properties are described through (randomized) sampling of cells, FC is used as a canonical setup of distribution regression. For this study we used a dataset <cit.> where more than 100.000 cells are measured per each patient (subject). For each cell a total of ten parameters are reported, hence, we treated each subject as an empirical distribution over ℝ^10. We considered downstream classification tasks on two different sets of labels. The first ('Tissue' classification) contains peripheral blood (pB) and bone marrow (BM) samples from N=44 subjects. The second ('Leukemia' classification) presents healthy and leukemia BM cell samples, N=50. Classes were balanced in both cases.
l8.2cm
 Distribution regression accuracy on Flow Cytometry datasets. Code available at https://github.com/hidden-for-review.






Model     2cTissue     2cLeukemia 
 2-5 
                   Acc.     Var.        Acc.     Var.    
 
GMM-FV             93.07%            ± 0.308            94.80%          ± 0.186                

SW1                87.10%            ± 0.530            95.07%          ± 0.111                

SW2                81.71%            ± 0.341            95.30%          ± 0.224                

MMD Linear         82.42%            ± 0.840            90.57%          ± 0.208                

MMD Gaussian       81.71%            ± 0.574            92.23%          ± 0.216                

MMD Cauchy         81.57%            ± 0.662            93.77%          ± 0.080                

MMD IMQ            82.89%            ± 0.698            91.43%          ± 0.217                

MDKE Rand        77.50%                ± 0.002                    89.50%                ± 0.003                

MDKE No Reg.    95.30%                ± 0.002                    92.46%                ± 0.002                
 
MDKE Reg.        98.89%                ± 0.010                    94.57%                ± 0.005                
 








We sampled 16 subjects for Tissue classification and 20 subjects for Leukemia for training. Unsupervised learning was performed over the entire dataset. The encoder f_θ was parametrized by a 2-layers neural network (NN) with ReLU nonlinearity and l_2 normalized output (on the unit hypersphere 𝕊^9). Per each subject we sampled a small percentage of cells, and we report performance for the sample size of 200. We repeated each training and testing phase for 100 times to track the variance induced by this aggressive subsampling.

To demonstrate the impact of unsupervised pre-training, we compared several methods across multiple configurations (<Ref>):

a) Kernels on distributions. This group includes Fisher kernels applied to parametric Gaussian Mixture Model (GMM) estimates, as suggested in <cit.>, along with Sliced Wasserstein-1 and Sliced Wasserstein-2 kernels <cit.>.

b) MMD kernels. Here, we employ a Gaussian embedding kernel for mean embeddings, marked as 'MMD' (Maximum Mean Discrepancy) in the table. This category includes various options for the distribution kernel, such as linear, Gaussian, Cauchy, and inverse multiquadrics.

c) MDKE. We explore various configurations of the encoder optimized with the MDKE objective. We report performance for randomly initialized encoder, and for unsupervised pre-trained encoder with and without regularization. Random initialization happens only once, and all subsequent accuracy measurements are taken using the same encoder.

The variance reported for each model is measured across multiple runs to demonstrate the effect of the sampling. Importantly, the optimization of the MDKE objective results in embedding kernels with significantly lower variance compared to non-data-specific kernels.




§ CONCLUSION

In this work, we presented an unsupervised way of learning data-dependent distribution kernel. While previous studies in distribution regression predominantly relied on hand-crafted kernels, our work, in contrast, demonstrates that entropy maximization can serve as a powerful guiding principle for learning adaptable, data-dependent kernel in the space of distributions. Our empirical findings show that this technique can not only serve as a pre-training step to enhance the performance of downstream distribution regression tasks, but also facilitate complex analyses of the input space. The interpretation of the learning dynamics induced by the proposed objective relies on a theoretical link between the quantum entropy of the dataset embedding and distributional variance. This theoretical link, which we have proven, enables us to approach the optimization from a geometrical perspective, providing crucial insights into the flexibility of the learned latent space encoding.

We hope that theoretically grounded way of learning data-dependent kernel for distribution regression tasks will become a strong alternative to the common practice of hand-picking kernels. More broadly, our results present a methodology for leveraging the distributional nature of input data along side the novel perspective on the encoding of complex input spaces. This highlights the potential to extend the application of more advanced learning methods, embracing the ever-increasing complexity of data by going beyond more conventional vector-based representations.



  
*Acknowledgments. 
We thank the Allen Institute for Brain Science founder, Paul G. Allen, for his vision, encouragement, and support.

abbrvnat







§ PROOFS


In this section, we present the proofs for the propositions outlined in our study. To ensure clarity, we will first restate the setup and introduce necessary concepts.

We define the input space as 𝒳, and 𝒫(𝒳) represents the space of probability distributions over 𝒳. Consider a dataset of M probability distributions, denoted as 𝒟_M = {P_i ∈𝒫(𝒳) }_i=1^M. With a p.d. characteristic embedding kernel k: 𝒳×𝒳→ℝ, the corresponding RKHS ℋ, and the feature map ϕ: 𝒳→ℋ, we define the mean embedding map μ: 𝒫(𝒳) →ℋ such that μ_P = μ(P) ∫_𝒳ϕ(x)   d P(x). A p.d. translation-invariant characteristic distribution kernel K: 𝒫(𝒳) ×𝒫(𝒳) →ℝ is defined using the mean embeddings of corresponding distributions. For simplicity, μ_i denotes μ(P_i).

Additional concepts essential for our proofs include the Gram matrix of mean embeddings G ∈ℝ^M × M, representing the inner products of mean embeddings in the dataset, i.e., G  [⟨μ_i, μ_j ⟩_ℋ]_ij. The kernel matrix K_𝒟∈ℝ^M × M with respect to the distribution kernel K is denoted as K_𝒟 [K(P_i, P_j)]_ij. We also recall the definition of distributional variance 𝕍_ℋ (see <Ref>):

    𝕍_ℋ(𝒟_M) 1/Mtr[G] - 1/M^2∑_i=1^M∑_j=1^MG_ij

These definitions and notations will be referenced throughout the proofs.



 §.§ Kernel Norms Gap and Distributional Variance


For both cases of input distributions being empirical probability distributions or continuous densities, we define mixture distribution P̅, with a slight abuse of notation:

    P̅(x) 1/M∑_i=1^M P_i(x)


For the mixture distribution P̅ and the Gram matrix G, the following relationship holds:

    μ_P̅_ℋ^2 = 1/M^2∑_i=1^M∑_j=1^MG_ij



Proof. We begin by recalling that the inner product between mean embeddings μ_i and μ_j in ℋ is given by:

    ⟨μ_i, μ_j ⟩_ℋ = ∬_𝒳×𝒳k(x, x^')   d P_i(x)   d P_j(x^')


Substituting this into the expression for the Gram matrix, we have:

    1/M^2∑_i=1^M∑_j=1^MG_ij   = 1/M^2∑_i=1^M∑_j=1^M∬_𝒳×𝒳k(x, x')    d P_i(x)   d P_j(x')
       = ∬_𝒳×𝒳 k(x, x')    d ( 1/M∑_i=1^MP_i(x) )   d ( 1/M∑_j=1^MP_j(x') ) 
       = ∬_𝒳×𝒳 k(x, x')    d P̅(x)   d P̅(x')  = μ_P̅_ℋ^2


This completes the proof of <Ref>.

By incorporating <Ref> into the definition of distributional variance (see <Ref>), and noting that the trace of G is the sum of squared norms of input distributions (i.e.,  tr[G] = ∑_i=1^M μ_P_i_ℋ^2), we obtain:

    𝕍_ℋ(𝒟_M) ≡1/M∑_i=1^Mμ_P_i_ℋ^2 - μ_P̅_ℋ^2

This result offers a new and intuitive perspective on distributional variance, conceptualizing it as the difference between the average squared norm of individual input distributions and the squared norm of the mixture distribution.



 §.§ Pairwise Distance and Distributional Variance




For a dataset 𝒟_M of M probability distributions, we define the average pairwise distance between kernel mean embeddings in ℋ as 𝕁_ℋ(𝒟_M), given by:

    𝕁_ℋ(𝒟_M) 1/M^2∑_i=1^M∑_j=1^M||μ_i - μ_j||_ℋ^2



For the distributional variance 𝕍_ℋ of the dataset 𝒟_M, the following relationship holds:

    𝕍_ℋ(𝒟_M) ≡1/2·𝕁_ℋ(𝒟_M)



Proof. Starting with the definition of 𝕁_ℋ(𝒟_M):

    𝕁_ℋ(𝒟_M)    = 1/M^2∑_i=1^M∑_j=1^M||μ_i - μ_j||_ℋ^2
       = 1/M^2∑_i=1^M∑_j=1^M⟨μ_i - μ_j, μ_i - μ_j ⟩_ℋ
       = 1/M^2∑_i=1^M∑_j=1^M( ⟨μ_i, μ_i ⟩_ℋ + ⟨μ_j, μ_j ⟩_ℋ - 2⟨μ_i, μ_j ⟩_ℋ) 
       = 1/M^2( 2 M ∑_i=1^M⟨μ_i, μ_i ⟩_ℋ - 2∑_i=1^M∑_j=1^M⟨μ_i, μ_j ⟩_ℋ) 
       = 2 ( 1/M∑_i=1^M⟨μ_i, μ_i ⟩_ℋ_diagonal of G - 1/M^2∑_i=1^M∑_j=1^M⟨μ_i, μ_j ⟩_ℋ)

Recognizing that ∑_i=1^M⟨μ_i, μ_i ⟩_ℋ = tr[G], we can equate expression in the brackets to 𝕍_ℋ(𝒟_M), leading to:

    𝕍_ℋ(𝒟_M) ≡1/2·𝕁_ℋ(𝒟_M)

<Ref> will be instrumental in further proofs, establishing a crucial link between distributional variance in ℋ and quantum entropy of the covariance operator embedding Σ_𝒟.



 §.§ Distribution Kernel Entropy Upper-Bound


In this section, we provide the proof for the key theoretical result stated in <Ref>.

Consider a dataset 𝒟 consisting of probability distributions {P∈𝒫(𝒳) }_i sampled i.i.d. from an unknown meta-distribution 𝔻. We assert that the second-order Rényi entropy 𝒮_2 of the empirical covariance operator embedding Σ_𝒟, induced by the choice of Gaussian distribution kernel K_RBF over points in the RKHS ℋ, is upper-bounded by the distributional variance 𝕍_ℋ(𝒟):

    1/2 γ𝒮_2(Σ̂_𝒟) ≤𝕍_ℋ(𝒟)

where γ is the bandwidth of the distribution kernel K_RBF.

Starting from the properties of 𝒮_2(Σ̂_𝒟) stated in  <Ref>:

    𝒮_2(Σ̂_𝒟) = 𝒮_2 ( 1/M K_𝒟) = -log∑_i=1^M∑_j=1^M( 1/M K_ij) ^2  = -log( 1/M^2∑_i=1^M∑_j=1^MK_ij ^2)


Applying Jensen's inequality, considering the concavity of the log, to <Ref>, we have:

    𝒮_2(Σ̂_𝒟)    ≤ - 1/M^2∑_i=1^M∑_j=1^MlogK_ij^2
       = - 1/M^2∑_i=1^M∑_j=1^Mlogexp(-γ/2μ_i - μ_j_ℋ^2)^2
       = - 1/M^2∑_i=1^M∑_j=1^Mlogexp(-γμ_i - μ_j_ℋ^2)
       =  γ( 1/M^2∑_i=1^M∑_j=1^Mμ_i - μ_j_ℋ^2) 
       = γ·𝕁_ℋ(𝒟) 
       = 2 γ·𝕍_ℋ(𝒟)

<Ref> uses the definition of K_RBF from <Ref>, <Ref> is derived from the definition of the average pairwise distance in <Ref>, and <Ref> follows from <Ref>. This completes the proof of <Ref>, establishing the upper bound for the second-order quantum Rényi entropy of the covariance operator embedding Σ_𝒟 in terms of the distributional variance in RKHS ℋ. 

Technical Note: It is important to note that 𝒮_2 is typically measured using the base-2 logarithm (log_2) rather than the natural logarithm. However, the proof remains accurate with a proper re-scaling to account for the change in the logarithmic base.



 §.§ Generalized Variance in RKHS



In this section we prove the connection between generalized variance and norm of the kernel mean embedding.

So far we established the squared norm of mean embedding is maximized by Dirac points (zero variance) and is minimized by uniform distribution (max variance). We now formalize the intuition that larger squared norm in embedding kernel RHKS corresponds to smaller variance in the latent space. We first note that under  <Ref>, the kernel metric d_k_emb (see <Ref>) induced by the choice of k_emb is a monotonically increasing function of Euclidean metric on the latent space, thus is representative of the geometry of encoded input distributions. Which let us use a generalized notion of variance here.



(<cit.>) Let X be a random variable which takes values in a Fréchet space ℱ equipped with seminorm ·_α. And suppose that X is square-integratable, in a sense that 𝔼 X _α^2 < ∞. Let μ∈ℱ be a Pettis integral of X (i.e. generalization of the mean). Generalized variance of X w.r.t. seminorm ·_α is defined as following

    Var_α[X] 𝔼 X - μ_α^2   .



Note that for a random variable in RKHS defined as a push-forward of a probability distribution P∈𝒫(𝒵) over the latent space, i.e. X = z ∼ϕ#P satisfies conditions of <Ref> with ℱ = ℋ, ·_α = ·_ℋ_emb, and a Pettis integral being a kernel mean embedding. Denote the described variance as Var_ℋ_emb[P]. We now show that


Under <Ref>, for every P∈𝒫(ℤ),

    Var_ℋ_emb[P] = 1 - μ_emb(P) _ℋ_emb^2.





  
Proof.

    Var_ℋ_emb[P]    = 𝔼_x ∼ Pϕ(x) - μ_P _ℋ_emb^2 
       = ∫ϕ(x) - μ_P _ℋ_emb^2   d P(x) 
       = ∫⟨ϕ(x) - μ_P, ϕ(x) - μ_P ⟩_ℋ_emb  d P(x) 
       = ∫( ϕ(x) _ℋ_emb^2 + μ_P _ℋ_emb^2 - 2 ⟨ϕ(x), μ_P ⟩_ℋ_emb)   d P(x) 
       = ∫ϕ(x) _ℋ_emb^2   d P(x)  + ∫μ_P _ℋ_emb^2   d P(x)  - 2 ∫⟨ϕ(x), μ_P ⟩_ℋ_emb  d P(x) 
       = ∫ 1   d P(x)  + μ_P _ℋ_emb^2 ∫ 1   d P(x)  - 2  ⟨∫ϕ(x)   d P(x) , μ_P ⟩_ℋ_emb
       = 1 + μ_P _ℋ_emb^2 - 2 μ_P _ℋ_emb^2 
       = 1 - μ_P _ℋ_emb^2

Which concludes the proof.

The proposition demonstrates a motivation behind training the encoder to minimize variance in the latent space by maximizing the (average) squared norm of the mean embeddings.



§ PRACTICAL ASPECTS OF LEARNING





 §.§ Kernel Hyperparameter Selection


In practice, the effectiveness of the optimization process is significantly influenced by the parameters of the Gaussian kernels. Setting the bandwidth parameter γ either too low or too high can hinder the model's ability to learn effectively. This issue is a well-known challenge in working with kernel methods, where determining the optimal kernel bandwidth has been an area of extensive study.

In our work, we have employed an empirical approach, which involves adjusting γ based on the idealized structure of the dataset after its projection onto the latent space. Specifically, we set γ such that 1/γ equals 10 times the average distance to the nearest neighbor in the set of points sampled uniformly on 𝕊^d-1 (inspired by the experimental approach in <cit.>). The number of points is chosen to be the number of distributions in the training set for the distribution kernel. For the embedding kernel, it is set to the number of distributions in a batch multiplied by the number of samples per distribution used for unsupervised training.



 §.§ Optional Regularization




As the theoretical maximum of 𝒮_2 entropy is attained when the spectrum of 1/M K_𝒟 is uniform, the optimal encoder with respect to ℒ_MDKE also tends to maximize the determinant of the kernel Gram matrix K_𝒟. This relationship is intuitive when considering that the determinant of a matrix is the product of its eigenvalues. The points that maximize such determinant K_𝒟 are theoretically known as Fekete points <cit.> and, in our case, are relative to the distribution kernel. Fekete points have been shown by <cit.> to be an optimal configuration for learning kernel interpolants, making them particularly suitable for downstream tasks framed as kernel regression. As such, encoders optimized under the ℒ_MDKE objective facilitate more accurate and robust performance in subsequent regression tasks.

In practice, we found that optimizing the MDKE objective posed certain numerical challenges, particularly due to the tendency of too small eigenvalues in the distribution kernel matrix K_𝒟 to collapse near the optimal value of the objective when using large batch size for training. To mitigate this issue and prevent undesirable optimization behavior, we have introduced a regularization term to the original objective. This term is inspired by the concept of Fekete points configuration, leading to the following loss function:

    ℒ_MDKE-R(θ)     -𝒮_2(Σ_𝒟) + ϵ·Ω(Σ_𝒟) 
       = log1/M K_𝒟_F^2 - ϵ·log| 1/M K_𝒟|


Here, ϵ serves as a hyperparameter to control the strength of regularization. The regularization term Ω(Σ_𝒟) is designed to stabilize the optimization process by counteracting the effects of the collapse of small eigenvalues. The empirical evidence supporting the effectiveness of regularization in stabilizing the training dynamics for the sentence representation learning experiment (as detailed in <Ref>) is showcased in <Ref>.



 §.§ Runtime Complexity


Scalability of the kernel methods is typically a key concern when it comes to practical applications. The runtime complexity of the proposed method could be decomposed into two components:


  * Computation of the distribution kernel Gram matrix. This involves computing O(N^2) inner products between distributions where N is a number of distributions. Each inner product involves computing matrix of pairwise distances between samples from each distribution, in the case of computing the Gaussian kernel over points on the hypersphere, the runtime complexity is the complexity of multiplying matrices ℝ^M × d where M is a number of samples. All computations could be efficiently parallelized.

  * Solving regression using distribution kernel Gram matrix. The complexity comes from matrix inverse and is typically estimated to be O(N^3) with a range of methods proposed to reduce the complexity <cit.>.

With this being said, we want to highlight that the runtime complexity of the proposed method is most favorable exactly for those tasks and datasets where distributional regression is an appropriate modeling approach. A significant  number of samples per  distribution ensures a high accuracy in approximating the kernel between a pair of distributions, and, at the same time, a relatively small number of distributions in the dataset alleviates issues related to storing the distribution kernel Gram matrix in memory and performing computations on it




§ CONNECTION TO OTHER FRAMEWORKS



 §.§ Distributional Regression Landscape



As the distributional regression task differs from common Machine Learning (ML) setups where inputs are given as vectors, the effective practical solution requires unique considerations.

The most obvious approach would be to ignore the fact that inputs are given as distributions and learn classifier on the space of samples, with a proper aggregation of posteriors (e.g. with a simple sum over histograms). This approach, while being simple, has been shown not to yield practically useful results, and was explicitly excluded from reported performance on different tasks by different authors <cit.>.


Viable approaches to solving distributional regression could be, approximately, split up into the following categories.



  
Discriminate generative models. The idea is to fit each input distribution to a parametric family, e.g. Gaussian Mixture Model (GMM), to use available closed-form solutions to compute kernel or similarity or distance between distributions. This group of methods originated from the work on hidden Markov models and it's applications to processing sequence modalities, like text, DNA, proteins, and more. Early work <cit.> on learning discriminative classifier for generative models leveraged the fact that parametric models forms a Riemannian manifold with local metric given by Fisher information, they derived kernel function termed Fisher kernel suitable for running SVM between generative models. Driving motivation was classification between hidden Markov models, with the developed method being applied to DNA and protein sequence analysis. Following the same modeling approach of analysing DNA sequences with discriminative models between Markov models, <cit.> proposed to use Bhattacharyya distance between distributions from exponential families to derived so-called Expected Likelihood Kernel. <cit.> explored the method of computing kernel between distributions as the integral of the product of pairs of distributions, termed Probability Product Kernel. A new family of kernels was applied to the same setup of discriminating task on text modality, hidden Markov models for biological data. The model was also successfully applied for analysis of linear dynamical systems for time series data. Critical advantage of this work was access to computationally effective way of computing kernel for the distributions without having access to analytical closed-form parametrization (only relying on samples). In <ref> we used fitting distributions to GMM with Fisher Kernel applied to learn parameters as a baseline for the performance on the task.



  
Point clouds. This group includes methods that model each input distribution as a set of points (also known as 'point cloud' or 'feature group' in computer vision) and use kernel or similarity function defined on sets. Large portion of the methods in this group arised in computer vision (CV) field when local features extractor were widely employed to pre-process images yielding either per-image histograms or sets of low dimensional vectors. The group includes kernels based on nonparametric divergence estimates, quantized set kernels, and so-called 'nearest-neighbor' set kernels to name a few <cit.>. Such kernel were employed to many CV tasks though the successfully application required not only a good kernel but also high-quality feature extractor. Special attention goes to methods leveraging  Wasserstein distance (including both kernel-based and similarity-based solutions). Wasserstein distance as a metric-aware discrepancy measure for probability distributions is a natural choice of deriving kernels or similarity functions between point clouds <cit.>. While being computationally problematic for large-scale problems, Sliced Wasserstein kernels were succesfully adopted <cit.> as they provided both reliable way for the point set comparison with a reduced cost leveraging sliced formulation. In <ref> we compared performance of Sliced Wasserstein-1 and Sliced Wasserstein-2 kernels, with the latter yielding significantly lower performance. Such a behavior is consistent with theoretical analysis stating that Sliced Wasserstein-1 has favoriable properties when compared to kernel based on Wasserstein-2 distance.



  
Kernel Mean Embeddidngs. <cit.> proposed to leverage  kernel mean embedding of measures in RKHS, so that the distributional regression could now be casted to a regression the corresponding Hilbert space. While the original work leveraged kernel between RKHS embeddings to train SVM, <cit.> provided theoretical guarantees for the  for learning Ridge regressor from distribution embeddings in the Hilbert space to the outputs. <cit.> applied the same approach in Bayesian regression settings, and <cit.> used it for Gaussian Process Regression. The setup gives a great deal of flexibility by choosing the kernel in RKHS, with a few being tested in practice. Gaussian kernel is a common choice, due to it's universality <cit.>. Among others, inverse multiquadric (IMQ) is a popular choice, typically paired with random features to improve runtime complexity of the algorithm. Linear kernel in RKHS, despite not being universal, was reported to produce competitive results in multiple practical setups. In <ref> we compared performance for 4 different kernels in RKHS, namely linear, Gaussian, Cauchy, and IMQ, -  all defined as functions of Maximum Mean Discrepancy (MMD), which gives as a way to compute the value of the kernel from finite number of samples with high accuracy. While all kernels demonstrated different performance in terms of the test accuracy, the difference reported is not that substantial.

Distribution kernels were successfully applied in different kernel-based methods, such as , as well as in different modalities, like distribution to distribution regression <cit.>, distribution regression for sequential data <cit.>, and more. In cases where input data is naturally represented as a probability distribution but not immediately applicable for existing distribution regression solutions, pre-processing or encoding of inputs is required. <cit.> proposed learning a latent space representation for input data to apply distribution regression in the text modality, with the encoder being trained jointly with the classifier. To our best knowledge, methods for learning data-dependent kernel for solving distributional regressions were not previously reported.



 §.§ Hyperspherical Uniformity Gap



The link between entropy maximization in the space of distribution, the gap between average norm and norm of the mixture distribution marginalized over the dataset (i.e. 'average' distribution), and properties of distributions that minimize and maximize kernel mean embedding norm (as described in <Ref>) unveils a subtle yet significant connection to the concept of the Hyperspherical Uniformity Gap (HUG), introduced in <cit.>. HUG has been developed to generalize the phenomenon of neural collapse observed in supervised classification settings. In our approach, working directly with samples from input distributions grants us explicit access to the 'grouping' of points in the input space. Even though the HUG framework setup does not have a notion of 'grouping', we can close the gap by noting that class labels provided with the dataset implicitly create 'groupings' of points, which can be interpreted as empirical samples drawn from latent probability distributions (one for each class).

However, a notable distinction lies in the dimensional aspect: HUG focuses on the distribution of points on the surface of a finite-dimensional hypersphere, whereas our work encompasses an infinite-dimensional hyperball setting, given the use of points in kernel induced RKHSs. Furthermore, the loss function introduced in our study presents a unified optimization objective that weaves together both the inner-group and inter-group dynamics while being derived from first principles. In the HUG framework these two aspects are addressed separately.

Establishing a more formal connection between these frameworks emerges as a promising direction for future research. Such an endeavor could offer a novel perspective on supervised classification, particularly by conceptualizing class prototypes as probability distributions rather than mere vectors. This exploration might bridge the gap between these distinct approaches, enriching our understanding of classification paradigms in high-dimensional spaces.




§ ADDITIONAL EXPERIMENTS



In the following section, we present additional experiments on learning data-dependent distribution kernels in the image and text domains. While representing text as empirical samples from the finite space of tokens (i.e., words from the dictionary) is quite common, the choice to model images as histograms over pixel positions is more subtle. We demonstrate that, in both scenarios, unsupervised pre-training of the encoder yields distribution kernel that achieves strong performance on downstream classification tasks, showcasing the versatility of the proposed learning framework in scenarios where distribution regression formulations are uncommon.



 §.§ Image Classification Tasks


MNIST <cit.> and Fashion-MNIST <cit.> consist of 28 × 28 pixel grayscale images divided into 10 classes. We considered each individual image to be a probability distribution (via rescaling pixel intensities so that l_1(image)=1) over the discrete space of pixel positions (i.e., histogram). Given this support space, the encoder f_θ is a discrete map which we implemented as a table lookup (i.e., embeddings) from pixel indices to the points on the hypersphere 𝒮^31. Embeddings were initialized by sampling points uniformly. Gradients of the MDKE objective with respect to the embeddings parameters were computed via automatic differentiation using projected gradient steps to ensure that the embeddings remain on the hypersphere. The small size of the support space enables the exact computation of the inner product between kernel mean embeddings of input distributions <Ref> and, subsequently, the distribution kernel <Ref> during both training and evaluation. This ensured a lower variance of the accuracy for the downstream classification.

Performing MNIST classification upon pre-training with our unsupervised encoder significantly improves the baseline (random initialization of latent embeddings) accuracy of 85.0% by reaching a plateau at 92.15%.

To understand the data-dependency of our encoding procedure, we analyzed the latent spaces of MNIST and Fashion-MNIST datasets.  The visualization of pixel-level interaction, computed using Gaussian kernel Gram matrix, reveals complex, dataset-specific interactions (<Ref>, <ref>). Spectral clustering using the kernel Gram matrix provides deeper insight into the pixel interaction landscape. The chart shows clusters of pixels with correlated intensities (<Ref>, <ref>), with the number of clusters set to 10 empirically.






 §.§ Text Classification Tasks


To assess our method's performance in a larger discrete support space, we utilized the "20 Newsgroups" <cit.>, a multi-class text classification dataset. We reduced the size of the dataset to 5 classes (resulting in 2,628 sentences and 38,969 unique words) by subsampling both training and test subsets. We treated sentences as empirical distributions over words, assuming word sets to be enough for topic classification, despite no positional info.

The encoder f_θ mirrored the setup used in the MNIST case (<Ref>), with l_2 normalized word embeddings on 𝕊^31.

However, while in the MNIST case embeddings computations were performed exactly, here considering the entire embedding kernel Gram matrix is impractical due to its large size. Instead, we optimized embeddings by randomly sampling 20 words per sentence, making the inner product between embeddings a stochastic approximation. This setup is meant to confirm that the optimization of the proposed MDKE objective yields a solution that is robust w.r.t. the excessive risk induced by first-level subsampling. 

During the unsupervised pre-training phase, we observed a steady decrease in the training loss (<Ref>) despite the small batch size (50 sentences with total of 1,000 words). Importantly, for this experiment we employed a regularized version of the objective (MDKE-R). We discuss the rationale and empirical evidence supporting the use of this regularization scheme in <Ref>. During evaluation, to reduce computational complexity, we sample 2,000 sentences for the train and 1,000 for the test split, keeping classes balanced. The maximum classification accuracy achieved was approximately 89.3%, while the random initialization performance averaged at 37. 5% (<Ref>). The framework’s high accuracy in downstream classification showcases its prowess in learning potent latent representations, even when dealing with large, finite input distributions.





§ IMPLEMENTATION DETAILS



In this section, we provide an example illustrating the implementation of the proposed method using the PyTorch framework. All experiments were performed on a single machine with 1 GPU and 6 CPUs.

Functions to compute distribution kernel Gram matrix:


def pairwise_kernel(x, gamma1):
    B, T = x.size(0), x.size(1)
    X_unroll = x.reshape(B*T, -1)
    dist = (X_unroll[:, None, :] - X_unroll[None, :, :])**2
    dist = torch.sum(dist, dim=2)
    G = F.avg_pool2d(
        dist[None, : , :],
        kernel_size=(T, T),
        stride=(T, T)
    )
    return torch.exp(-(gamma1/2.) * G.squeeze(0))

def distribution_kernel_gram(x, gamma1, gamma2):
    Gxy = pairwise_kernel(x, gamma1=gamma1)
    Gx = Gy = torch.diag(Gxy)
    G = Gx[:, None] + Gy[None, :] - 2*Gxy
    K = torch.exp(-(gamma2/2.) * G)
    return Gxy, K


Distribution kernel entropy estimator and the MKDE loss:


def distribution_kernel_entropy(K):
    B = K.size(0)
    Cov = (1/B) * K
    return -(Cov ** 2).sum().log2()

def mkde_loss(encoder, X, gamma1, gamma2):
    # X.shape is (n_distributions, n_samples, d_input)
    Z = encoder(X)
    # Z.shape is (n_distributions, n_samples, d_latent)
    _, K = distribution_kernel_gram(Z, gamma1, gamma2)
    # K.shape is (n_distributions, n_distributions)
    loss = -distribution_kernel_entropy(K)
    return loss


The code presented here follows the setup presented in <Ref> using Gaussian kernel both as embedding and as a distribution kernel. Other kernels could be used by adjusting implementation of both helper functions accordingly. When training on large datasets, <cit.> might be used to avoid memory overflow in average pooling.




 Paper Title: Investigating Brain Connectivity and Regional Statistics from EEG for early stage Parkinson's Classification.



The Inevitable Quark Three-Body Force and its Implications for Exotic States
    Su Houng Lee
    August 5, 2024
============================================================================



§ ABSTRACT 
We evaluate the effectiveness of combining brain connectivity metrics with signal statistics for early stage Parkinson's Disease (PD) classification using electroencephalogram data (EEG). The data is from 5 arousal states - wakeful and four sleep stages (N1, N2, N3 and REM). Our pipeline uses an Ada Boost model for classification on a challenging early stage PD classification task with with only 30 participants (11 PD , 19 Healthy Control). Evaluating 9 brain connectivity metrics we find the best connectivity metric to be different for each arousal state with Phase Lag Index achieving the highest individual classification accuracy of 86% on N1 data. Further to this our pipeline using regional signal statistics achieves an accuracy of 78%, using brain connectivity only achieves an accuracy of 86% whereas combining the two achieves a best accuracy of 91%. This best performance is achieved on N1 data using Phase Lag Index (PLI) combined with statistics derived from the frequency characteristics of the EEG signal. This model also achieves a recall of 80 % and precision of 96%. Furthermore we find that on data from each arousal state, combining PLI with regional signal statistics improves classification accuracy versus using signal statistics or brain connectivity alone. Thus we conclude that combining brain connectivity statistics with regional EEG statistics is optimal for classifier performance on early stage Parkinson's. Additionally, we find outperformance of N1 EEG for classification of Parkinson's and expect this could be due to disrupted N1 sleep in PD. This should be explored in future work.






Parkinson's Disease, Classification, Time Series, Connectivity





§ INTRODUCTION
 
Parkinson's Disease is the second most prevalent neurodegenerative disease and is characterised by the loss of dopaminergic neurons in the substantia nigra <cit.> <cit.>. Extensive loss of these neurons culminates in movement-related symptoms including muscle stiffness, instability and a tremor.  However, prior to the development of motor symptoms, PD is very challenging to diagnose, complicating early detection and intervention.
The progression of the disease can be rated accorded to a clinical rating scale such as the HY (Hans and Yahr) scale <cit.>. This scale defines broad categories of motor function in Parkinson's and has 5 stages with stage 1 being the earliest. There is a need for earlier diagnosis of Parkinson's and Electroencephalogram (EEG)  has shown promise for this. EEG is non-invasive, relatively inexpensive and records the electrical activity of cortical neurons in the brain with a high temporal resolution.

Limited work has been done on the earliest stage (HY 1) Parkinson's classification. Limited work has also been done on the fusion of brain connectivity with signal statistics and exploration on whether the information in each is supplementary or redundant. Prior work on general PD classification uses methods such as deep learning (e.g. Convolutional Neural Networks), signal statistics input to a machine learning model and measures of brain connectivity (see related work <ref>) input to a machine learning model. Machine learning models used in this previous work include Support Vector Machines, Neural Networks and tree based models such as Random Forest and Ada Boost classifiers <cit.>.

In this work we evaluate the effectiveness of combining signal statistics with brain connectivity for early stage PD classification (mean HY 1.55) using EEG recorded across 5 arousal states. The arousal states include wakeful data and the four sleep stages (N1, N2, N3 and REM). We investigate 9 brain connectivity metrics individually and in combination with regional EEG statistics. Best classifier performance is achieved by combining brain connectivity with signal statistics from multiple brain regions. This result suggests that brain connectivity metrics and signal statistics provide supplementary information for classification of early stage PD and best performance is achieved by combining both.



§ RELATED WORK
 



 §.§ Parkinson's Classification with EEG

Prior work has found frequency based statistics of the EEG signal input to machine learning models to be effective for classification. We build on work regarding peak frequencies and band powers from EEG. Band powers refers to EEG signal strength within defined frequency ranges whereas peak frequency refers to the highest strength frequency <cit.>.  In <cit.> the authors achieve 78% accuracy on PD classification with a balanced data set using EEG band power values, median and peak frequency as features. In  <cit.> an accuracy of 86% is achieved with a support vector machine using frequency based EEG features. In  <cit.> the authors combine coherence (functional connectivity measure) with wavelet coefficients achieving an accuracy of 94%. More complex spectral features have also been used such as in <cit.> where authors achieve 99% accuracy using features of the third-order spectra of EEG (bispectrum). However, a limitation of this study is that participants were not in the early stage of PD - they were in stage II and III according to the HY scale. Other work such as  <cit.> achieved lower performance (85% accuracy) using complex spectral features. Deep Learning methods have also shown promise such as in <cit.> where the authors achieve 96.9% accuracy using a 'convolutional-recurrent' neural network architecture. To summarise, EEG classification of Parkinson's shows promise. However current work on early stage Parkinson's is limited and limited work has been done to explore different connectivity metrics across different arousal states and how performance changes when fusing brain connectivity information with regional statistical EEG features.  



 §.§ Brain Connectivity
 
Measures of connectivity across the brain provide valuable information about brain activity and are thus useful for EEG classification. Different types of connectivity include structural connectivity, functional connectivity and effective connectivity. Structural connectivity corresponds to anatomical connectivity between brain regions, functional connectivity characterises statistical dependencies or correlations in activity and effective connectivity explores causal or directional influences between regions<cit.>. A number of brain connectivity measures have been defined in the EEG literature <cit.>. For this work we explore 9 brain connectivity measures including  Coherence (coh) , Imaginary Coherence (imcoh) , Phase-Locking Value (PLV), corrected imaginary PLV (ciplv) , Pairwise Phase Consistency (PPC) , Phase Lag Index (PLI), Directed Phase Lag Index (DPLI), Weighted Phase Lag Index (WPLI) and debiased weighted phase lag index (DWPLI).  

For conciseness we define PLI in detail only as this metric was evaluated in most detail in our experiments (see section <ref>). Phase lag index measures the asymmetry in the distribution of phase differences over time between two signals. The PLI ranges between 0 and 1 with 0 indicating no coupling or instaneous coupling and 1 indicating that there is a true lagged interaction. By definition if the distribution of phase differences between two signals are given by Δϕ(t_k) for k = 1 … N then the phase lag index is given by <cit.> : PLI = | ⟨sign[Δϕ(t_k)] ⟩|] 

Connectivity metrics such as PLI can be calculated for two signals or for components of each signal. For example, one can calculate the PLI between the delta band (0.5-4Hz) of one signal with the delta band of another signal. In <cit.> the authors use Phase Lag Index to classify Mild cognitive impairment and find phase lag index in the alpha band to be correlated with a cognitive assessment score.

PLI as a connectivity metric has the benefit of being insensitive to volume conduction effects <cit.>. Volume conduction occurs when there is a single source of neuronal activity in the brain but it's electrical signal is measured simultaneously at multiple electrodes. This can lead to a false positive for connectivity between electrodes or brain regions. 




 §.§ Frequency based signal statistics
 
Frequency based statistics are commonly used in EEG classification pipelines. Power spectral density (PSD) refers to the distribution of power vs. frequency for a signal and is computed with the Fourier Transform. Research has shown the strength of the EEG signal in specific frequency power bands to be informative of brain activity. These power bands are defined as delta (0.5–4Hz), theta (4–8Hz), alpha (8–12Hz), sigma (12–16Hz), beta (16–30Hz) and gamma (30–40Hz) <cit.>. The peak frequency (highest power frequency) in the PSD aswell as the power in that frequency are also informative statistics for classification. For longer EEG signals (over 5 minutes) enhanced classification performance can be achieved by calculating distributions of these frequency based statistics over time windows across the signal. <cit.>





§ METHOD



The data used in this work is from a yet unpublished dataset containing 57 channel wakeful and sleep EEG from Parkinson's and Healthy participants. The data set is described in further detail in <ref>.



 §.§ Preprocessing

The EEG data for each participant was scored manually in 30-second epochs in line with AASM guidelines <cit.>.  Epochs were labelled as Wake, N1, N2, N3 or REM. The EEG data was downsampled from the recording frequency of 512Hz to 256Hz and a 0.25 - 40Hz bandpass filter applied. Next, Independent Component Analysis and manual inspection of the EEG signal and power spectral density was completed for further artefact removal. Bad channels were interpolated and the data re-referenced using the REST referencing technique <cit.>.



 §.§ Regional Statistics Classifier
  
For the regional classification pipeline the 57 EEG electrodes were grouped into 13 brain regions of interest (see figure <ref>) . 



Per electrode and per region, statistics of 15 key frequency based features were calculated. These 15 statistics were the 6 band powers as defined in <ref>, total signal power aswell as the four peak frequencies and their corresponding power values. These 15 statistics are calculated per 6 second window resulting in 15 feature distributions. Key statistical features of these feature series including length, maximum, absolute maximum, minimum, mean, median, standard deviation, variance, root mean square and sum are calculated and input to a machine learning model (Ada Boost). The pipeline is shown in figure <ref>.




The statistical model is fit using features from single brain regions, two brain regions together at a time and all brain regions together.



 §.§ Connectivity Classifier
 
For the connectivity classification pipeline. Connectivity is calculated for every electrode-electrode pair. For 57 electrodes, this results in 1596 features calculated per sample. An example electrode-electrode connectivity grid is shown in Figure <ref>. The x-axis and y-axis contain the electrode names and between each electrode a connectivity value is calculated. The grid is symmetrical as electrode 1 connectivity with electrode 2 is the same as electrode 2 connectivity with electrode 1. The diagonal is blank as the connectivity of an electrode is not calculated with itself.



The calculated connectivity features are input to a machine learning model (AdaBoost) for classification. This is done for 9 connectivity metrics as outlined in experimental procedure (section <ref>).





 §.§ Combined Classifier

The combined classifier simply combines the regional statistical features with brain connectivity features for classification. Both sets of features are input to our machine learning model (Ada Boost classifier).



§ EXPERIMENTS AND DISCUSSION





 §.§ Data
 
The dataset includes 57-channel EEG data for 19 Healthy Control and 11 Parkinson's participants.  The data includes wakefulness EEG and recordings of the four sleep stages (N1, N2, N3, REM). In home recording was conducted on 2 or 3 nights for all participants in line with the AASM standards for polysomnography  <cit.>. The demographics of PD and Healthy participants are similar, see table <ref>



The number of samples exceeds the number of participants as EEG data was recorded on multiple nights. Those recordings of a low quality were dropped. Samples per class per arousal state aswell as mean sample length in epochs (where an epoch is 30s for sleep data and 20s for wakeful) are shown in Table  <ref>.





 §.§ Experimental Procedure
 
All experiments were run with two random seeds using 5-fold cross validation. The mean across the two sets of 5-fold cross validated experiments was calculated to determine the classifier performance. Where multiple samples were present from a single participant they were kept in the same fold to avoid information leakage. 

The regional statistics classification pipeline (see <ref>) was deployed using features from each of the 13 brain regions individually, using features from two brain regions at a time (78 combinations) and using features from all 13 brain regions at once for each arousal state. 

The connectivity pipeline (see <ref> ) was deployed for 9 connectivity metrics for each arousal state. Each connectivity metric was calculated for each band power delta (0.5–4Hz), theta (4–8Hz), alpha (8–12Hz), sigma (12–16Hz), beta (16–30Hz), gamma (30–40Hz) and for the entire EEG signal. Thus there were 7 results for each connectivity metric for each arousal state. The 9 metrics used in the pipeline were Coherence (CoH), Imaginary Coherence (Imcoh), Phase-Locking Value (PLV), corrected imaginary PLV, Pairwise Phase Consistency (PPC) , Phase Lag Index (PLI), Unbiased Phase Lag Index, Directed Phase Lag Index (DPLI), Weighted Phase Lag Index (WPLI) and debiased weighted phase lag index (DWPLI).  

Finally, a combined pipeline was deployed for the best performing overall connectivity metric combined with each set of statistical features for each arousal state.



 §.§ Results
 

The regional statistical features pipeline had best performance using features from two brain regions at a time. We present results from this best performing statistical approach. For this classification method a best accuracy of 78.3% was achieved on Wakeful data and 77.5 % accuracy on  N1 data. Worse performances (less than 65 % accuracy) were achieved on N2, N3 and REM data (see column 2 in table <ref>).

The best performing connectivity metric was Phase Lag Index on the gamma power band using N1 data which achieved a performance of 86.2% classification accuracy. Across other arousal states other metrics performed better. Table <ref> below shows the best performing metric per arousal state with their corresponding performance. 




With PLI as the highest performing connectivity metric on an individual arousal state (86.2 % N1) we combined PLI with statistical features across arousal states. Results are in table <ref> below. The table also shows results for the best individual PLI only models and best statistical features only models to highlight the effect of combination.





  §.§.§ Analysis

Per table <ref> the best performing connectivity metric varied per arousal state. The best overall performance was with N1 data where Phase Lag Index on the gamma power band achieved an accuracy of 86.2%. Phase Lag Index measures the consistency of phase differences between two signals and is insensitive to volume conductive effects (see <ref>). Other metrics derived from PLI include Weighted Phase Lag Index which de-weights the value of small phase differences and the debiased estimator of squared weighted phase lag index (DWPLI). With PLI having highest performance for N1 this suggests that volume conduction effects are likely to be present in the data as well as true lagged interaction at small phase differences. This would explain why WPLI and the debiased estimator of squared WPLI do not outperform PLI on N1 data. Across other arousal states different metrics perform best. This suggests that the patterns of connectivity discriminating Healthy Controls from those with Parkinson's likely varies across arousal states. This should be explored in further detail for future work. 



Table <ref> shows per arousal state the best accuracy achieved with using Statistical Features only, Connectivity (PLI) only and Statistical Features combined with Phase Lag Index. For every arousal state combining statistical features with PLI leads to an increase in mean accuracy. This suggests that the statistical features and PLI provide useful complementary information for classification of Parkinson's. Therefore if performance is the main priority over computational efficiency then one should always include connectivity data to complement statistical features for classification. 

The overall best performing classifier on N1 data used Statistical Features combined with Phase Lag Index. This model achieved an accuracy of 91 %. For the classification of Parkinson's disease precision and recall are also important. This model achieved a recall of 80 % and precision of 96% (see figure <ref> for confusion matrix).  



This demonstrates the effectiveness in combining connectivity and statistical features. This also indicates N1 data as being very discriminatory for this classification task. This could be due to the Parkinson's disease participants having more disrupted light sleep during the N1 phase and should be explored in future work.









 §.§ Future Work

Future work should explore the differences between connectivity metric performance across data types for Parkinson's classification. This could help to highlight the differences in the Parkinson's brain activity vs Healthy controls across wakefulness and the sleep stages. 

Secondly, future work should work on optimal methods for fusion between connectivity metrics and statistical features. In this work we used simple feature concatenation (feature level fusion) to demonstrate the effect of using the respective features individually and together. For optimal classifier performance data fusion techniques such as late fusion (fusion at the decision level), model level fusion (at the representation level) or a hybrid between methods can be implemented <cit.>.

Finally, work should be expanded to other data sets as our data set is relatively small with 30 participants (19 Healthy Control, 11 Parkinson's). Replication of our results should be done on additional data sets to show generalisability of the result.




§ CONCLUSION
 

We evaluate the effectiveness of EEG signal statistics and brain connectivity metrics across 5 arousal states (Wakeful, N1, N2, N3 and REM) for early stage Parkinson's classification. Brain connectivity metrics are evaluated on their own and in combination with regional signal statistics using feature level fusion. Phase Lag Index on the gamma band power on N1 data was found to be the highest performing connectivity metric (86 % accuracy) with different metrics performing best on each arousal state. Phase Lag Index was then combined with signal statistics across all arousal states. The combination of PLI with regional signal statistics increased mean accuracy of the classifier across every arousal state. This suggests that fusion of connectivity with regional signal statistics is important for best performance. The best overall classifier (PLI and regional statistics combined) was on N1 data achieving 91 % accuracy, 96% precision and 80% recall. Future work should explore how connectivity metric performance varies across arousal state and optimal methods for fusion of EEG signal statistics with brain connectivity information.







IEEEbib




 Paper Title: Finding Defective Elements in Intelligent Reflecting Surface via Over-the-Air Measurements.







































 


Finding Defective Elements in Intelligent Reflecting Surface via Over-the-Air Measurements
    
   Ziyi Zhao1,
                     Zhaorui Wang2,
                     Shuowen Zhang1,
                     and Liang Liu1
   1
                    Department of Electrical and Electronic Engineering, The Hong Kong Polytechnic University, Hong Kong SAR, China,

                     E-mail: ziyii.zhao@connect.polyu.hk, {shuowen.zhang,liang-eie.liu}@polyu.edu.hk
   2
                     School of Science and Engineering, The Chinese University of Hong Kong, Shenzhen, Shenzhen, China 

                     E-mail: wangzhaorui@cuhk.edu.cn
 The work was supported in part by the National Key R&D Project of China under Grant No. 2022YFB2902800; by the National Natural Science Foundation of China with Grant No. 62101474; and by the Research Grants Council, Hong Kong, China, with Grant No. 15203222 and Grant No. 15230022.
    August 5, 2024
===============================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================



remarkRemark
exampleExample










§ ABSTRACT 
Due to circuit failures, defective elements that cannot adaptively adjust the phase shifts of their impinging signals in a desired manner may exist on an intelligent reflecting surface (IRS). Traditional way to find these defective IRS elements requires a thorough diagnosis of all the circuits belonging to a huge number of IRS elements, which is practically challenging. In this paper, we will devise a novel approach under which a transmitter sends known pilot signals and a receiver localizes all the defective IRS elements just based on its over-the-air measurements reflected from the IRS. The key lies in the fact that the over-the-air measurements at the receiver side are functions of the set of defective IRS elements. Based on this observation, we propose a bisection based method to localize all the defective IRS elements. Specifically, at each time slot, we properly control the desired phase shifts of all the IRS elements such that half of the considered regime that is not useful to localize the defective elements can be found based on the received signals and removed. Via numerical results, it is shown that our proposed bisection method can exploit the over-the-air measurements to localize all the defective IRS elements quickly and accurately.




§ INTRODUCTION

The recent revolution in software-controlled surfaces using metamaterials has stimulated a flurry of research activities in using intelligent reflecting surface (IRS) to improve the performance for wireless communication <cit.>. The key is to adaptively adjust the phase shifts of IRS elements based on the channel state information (CSI) so as to enhance the channel quality of the mobile users. Previously, tremendous works have been done to enable low-overhead CSI acquisition <cit.> and efficient IRS phase shift design based on CSI <cit.>, assuming that all IRS elements are in the normal state and able to induce the desired phase shifts via properly controlling their circuits. However, in practice, the circuits of IRS elements are prone to failures <cit.>. Therefore, it is very likely that some defective elements that cannot adaptively achieve the desired reflecting patterns exist on the IRS. For example, if there is a circuit failure in a controller chip such that it cannot consistently send the desired phase shifts to its associated IRS elements, the phase shifts of these defective elements will be stuck at a fixed state. This motivates us to study the diagnosis of the IRS in this paper.

A conventional approach for a diagnosis of the IRS is to have a thorough check of all its circuits. However, this approach is practically challenging, because there are a huge number of sophisticated circuits to control hundreds of or even thousands of IRS elements. In this paper, we propose a novel approach that is able to find all the defective IRS elements just based on the over-the-air measurements. To enable an over-the-air diagnosis of the IRS, a transmitter and a receiver are deployed around the IRS. Under this system, the transmitter keeps transmitting known pilot signals, the IRS elements keep changing their phase shifts in a pre-designed pattern, and the receiver measures its received signals. Because of the defective IRS elements, the received signals can be quite different from the expected ones. The key observation is that the abnormal received signals are functions of the set of the defective IRS elements. This motivates us to apply the bisection method to estimate this set based on the received signals. Specifically, an efficient way to adaptively control the desired phase shifts of all the IRS elements is designed such that we are able to iteratively find a sub-regime on the IRS that is not useful to localize the defective IRS elements. After iteratively removing these sub-regimes based on the bisection method, we can localize all the defective IRS elements. Because the bisection method can cut off half of the remaining regime at each iteration, our proposed method is very efficient to localize the defective elements from a huge number of IRS elements.

It is worth noting that a related topic, hardware impairment, has been investigated in the literature <cit.>. In these works, it is assumed that an unknown phase error is added to the desired phase shift of each IRS element. Based on this model, various signal processing techniques have been proposed to estimate these phase errors such that calibration can be conducted to compensate for these phase errors. However, circuit failure considered in this paper is quite different from hardware impairment considered in the above works. Specifically, under the hardware impairment model, the phase shifts of all IRS elements are still close to the desired values, just subject to some errors. However, under the circuit failure model, some IRS elements are totally in the irregular state, and the phase shifts of these defective elements usually have nothing to do with the desired values, e.g., they are stuck at fixed values. Due to this reason, calibration is not possible under our considered model. We have to localize all the defective IRS elements, even though we do not know their irregular reflecting patterns, such that we can repair their circuits. This is a new problem.



§ SYSTEM MODEL

Consider an IRS consisting of N=N_h× N_v elements, which are laid out in a grid pattern with N_h columns and N_v rows. Without loss of generality, we assume that N_h=2^m_h and N_v=2^m_v with some integers m_h and m_v. Define 𝒩_h={1,…,N_h} and 𝒩_v={1,…,N_v}. Moreover, we index the element at the n_h-th column and the n_v-th row of the IRS as element (n_h,n_v), n_h∈𝒩_h, n_v∈𝒩_v.

Because of the complicated circuit to control the reflecting patterns, the elements on an IRS are prone to failures. The defective IRS elements may not reflect the signals in the pre-designed pattern, leading to poor communication performance. If an IRS is in an irregular state with some defective elements, our objective is to find the indices of all the defective elements.



 §.§ IRS Failure Model

In this paper, we consider a clustered failure model of the IRS <cit.>, where all the defective elements are located within a continuous rectangular region of the IRS,[If the defective elements are located within a continuous but non-rectangular region of the IRS, the defective element set ℰ defined in (<ref>) actually is the minimum rectangular region that contains all the defective elements. After such a small region is found based on the over-the-air approach that is later proposed in the paper, we can have a quick diagnosis of circuits of all the elements in ℰ to further localize the defective elements.] as shown in Fig. <ref>. Such a model is valid in many scenarios, e.g., when a controller chip fails to send control signals to its associated elements, when faults in some interconnects leave a clustered region of the IRS isolated from the remaining region, etc. Let (n_h, min,n_v, min) and (n_h, max,n_v, max) denote the indices of the elements at the lower left corner and the upper right corner of this defective region, respectively, as shown in Fig. <ref>. Therefore, under our considered clustered failure model, the set consisting of the indices of all the defective elements on the IRS is given by

    ℰ={(n_h,n_v)|    n_h∈{n_h, min,…,n_h, max}, 
         n_v∈{n_v, min,…,n_v, max}}.
For simplicity, we also define the set consisting of the indices of all the IRS elements in the normal state as

    𝒲={(n_h,n_v)|(n_h,n_v)∉ℰ}.




Let ϕ_n_h,n_v,t∈ (0,2π] and e^jϕ_n_h,n_v,t denote the desired phase shift and the corresponding reflecting coefficient of IRS element (n_h,n_v) at time slot t, ∀ n_h∈𝒩_h, n_v∈𝒩_v. For each IRS element (n_h,n_v)∈𝒲 that is in the normal state, it will adaptively reflect the signals with different phase shifts at different time slots. However, for any defective IRS element (n_h,n_v)∈ℰ, we assume that it is stuck at a constant but unknown state over different time slots <cit.>. This can occur when a controller chip fails to send control signals to its associated elements, such that these elements cannot adaptively update their reflecting coefficients as desired. Under this error pattern, the constant phase shift and the corresponding reflecting coefficient of a defective element (n_h,n_v)∈ℰ across all the time slots are defined as β_n_h,n_v∈ (0,2π] and e^jβ_n_h,n_v, respectively. In practice, the real reflecting coefficient of IRS element (n_h,n_v) at time slot t, which depends on whether this element is defective or not, is given by

    θ_n_h,n_v,t={[  e^jϕ_n_h,n_v,t, if  (n_h,n_v)∈𝒲,;    e^jβ_n_h,n_v, if  (n_h,n_v)∈ℰ, ].  ∀ t.
Note that under the above reflecting model, β_n_h,n_v and ℰ (also 𝒲) are unknown. Therefore, θ_n_h,n_v,t is also unknown.




 §.§ Signal Model

In this paper, we aim to propose an over-the-air approach to estimate the values of n_h, min, n_h, max, n_v, min, and n_v, max such that we can find the indices of all the defective elements on the IRS, i.e., ℰ. To enable the over-the-air diagnosis of the IRS, we deploy a radio transmitter and a radio receiver around the IRS. It is assumed that the transmitter is equipped with 1 antenna, while the receiver is equipped with M≥ 1 antennas. Define h∈ℂ^M× 1 as the channel from the transmitter to the receiver, u_n_h,n_v as the channel from the transmitter to IRS element (n_h,n_v), and r_n_h,n_v∈ℂ^M× 1 as the channel from IRS element (n_h,n_v) to the receiver. We assume that the transmitter and the receiver are deployed very close to the IRS such that they only have line-of-sight (LOS) channels to each other, which can be known in advance given their positions. Moreover, define g_n_h,n_v=u_n_h,n_vr_n_h,n_v as the cascaded channel from the transmitter to IRS element (n_h,n_v) to the receiver. At time slot t, the received signal at the receiver side is then given by

    y_t
    =    hx_t+∑_(n_h,n_v)∈ℰe^jβ_n_h,n_vg_n_h,n_vx_t 
        +∑_(n_h,n_v)∈𝒲e^jϕ_n_h,n_v,tg_n_h,n_vx_t+z_t,  ∀ t,
where x_t denotes the known pilot transmitted at time slot t, and z_t∼𝒞𝒩(0,σ^2I) denotes the additive white Gaussian noise (AWGN) of the receiver at time slot t. Note that the received signals given in (<ref>) are functions of the unknown set ℰ. Under the over-the-air approach, we aim to properly design ϕ_n_h,n_v,t, ∀ n_h,n_v,t, such that we can localize all the defective elements in ℰ just based on the received signals y_t, ∀ t, without checking the circuit of the IRS.




















§ THREE-PHASE BISECTION METHOD FOR LOCALIZING DEFECTIVE IRS ELEMENTS

The bisection method belongs to the class of the cutting-plane methods, and is widely used to solve the one-dimension optimization problem. In this paper, we will show that the bisection method can also be exploited to localize the defective IRS elements under the over-the-air scheme. For convenience, we will focus on the design of the bisection-based method to estimate n_h, min and n_h, max in the horizontal dimension, while the same approach can be utilized to estimate n_v, min and n_v, max in the vertical dimension.

The basic idea of our proposed scheme is as follows. Starting with the regime [1,N_h], we keep cutting off half of the regime that does not contain n_h, min (n_h, max) until the remaining regime is sufficiently small such that the middle point of this remaining regime can serve as the estimation of n_h, min (n_h, max). In the following, we first show that given any boundary defined by c_h=n̅, how to determine whether we should cut off the regime on the left hand side of this boundary with n_h<n̅ or the regime on the right hand side of this boundary with n_h>n̅, just based on the over-the-air measurements. Then, we will introduce our proposed bisection method to iteratively update n̅ and estimate n_h, min and n_h, max.



 §.§ Determining the Cutting-Plane

Given any boundary defined by c_h=n̅, our goal is to determine whether n_h, min (n_h, max) is on the left hand side or on the right hand side of this boundary. Depending on the location of the defective IRS elements defined in (<ref>), there are three cases that may occur.

Case 1: All the defective IRS elements defined in (<ref>) are on the left hand side of the boundary defined by c_h=n̅, i.e., n_h, min≤ n_h, max< n̅, as shown in Fig. <ref> (a).

Case 2: All the defective IRS elements defined in (<ref>) are on the right hand side of the boundary defined by c_h=n̅, i.e., n_h, max≥ n_h, min> n̅, as shown in Fig. <ref> (b).

Case 3: The defective IRS elements defined in (<ref>) are on both the left hand side and the right hand side of the boundary defined by c_h=n̅, i.e., n_h, min< n̅ and n_h, max>n̅, as shown in Fig. <ref> (c).



In the following, we propose an efficient detection scheme such that given any boundary defined by c_h=n̅, we can determine which of the above three cases is true.

Under our proposed scheme, there is an initialization stage which consists of two time slots, denoted by time slots 0_- and 0_+ for convenience. In time slots 0_- and 0_+, we respectively set a common desired phase shift for all the IRS elements as

    ϕ_n_h,n_v,0_-=ϕ̅_0_-,  ∀ n_h,n_v, 
       ϕ_n_h,n_v,0_+=ϕ̅_0_+≠ϕ̅_0_-,  ∀ n_h,n_v.
Define

    g_ e=∑_(n_h,n_v)∈ℰe^jβ_n_h,n_vg_n_h,n_v, 
       g_ w=∑_(n_h,n_v)∈𝒲g_n_h,n_v.
According to (<ref>), the received signals in time slots 0_- and 0_+ are

    y_0_- = hx_0_-+g_ ex_0_-+g_ we^jϕ̅_0_-x_0_-+z_0_-, 
       y_0_+ = hx_0_++g_ ex_0_++g_ we^jϕ̅_0_+x_0_++z_0_+.
Because z_0_-, z_0_+∼𝒞𝒩(0,σ^2I) are independent noise, we have

    p(y_0_-,y_0_+|g_ e,g_ w) 
    
        =    1/(πσ^2)^2Me^-∑_i∈{0_-,0_+}‖y_i-hx_i-g_ ex_i-g_ we^jϕ̅_ix_i‖^2 /σ^2.
Then, the maximum-likelihood (ML) estimators of g_ e and g_ w that maximize the above conditional probability are given as

    g̅_ e=
    e^jϕ̅_0_+x_0_+
    (y_0_--hx_0_-)
    -e^jϕ̅_0_-x_0_-
    (y_0_+-hx_0_+)/x_0_-x_0_+(e^jϕ̅_0_+-e^jϕ̅_0_-), 
       g̅_ w=x_0_-
    (y_0_+-hx_0_+)-
    x_0_+
    (y_0_--hx_0_-)/x_0_-x_0_+(e^jϕ̅_0_+-e^jϕ̅_0_-).




After the above initialization stage, we make some boundary defined by c_h=n̅_t at each time slot t≥ 1, which can divide the whole IRS into two parts. At time slot t, we respectively set a common desired phase shift for all the IRS elements on the left hand side of the boundary and one for all the IRS elements on the right hand side of the boundary as

    ϕ_n_h,n_v,t= ϕ̅_t, l,   if  n_h∈ [1,⌊n̅_t⌋],
       ϕ_n_h,n_v,t= ϕ̅_t, r,   if  n_h∈ [⌈n̅_t⌉,N_h],   t≥ 1,
where ϕ̅_t, l≠ϕ̅_t, r. According to (<ref>) and (<ref>), the received signal at time slot t≥ 1 is

    y_t =    hx_t+g_ ex_t+∑_(n_h,n_v)∈𝒲_t, lg_n_h,n_ve^jϕ̅_t, lx_t+ 
        +∑_(n_h,n_v)∈𝒲_t, rg_n_h,n_ve^jϕ̅_t, rx_t+z_t,   t≥ 1,
where

    𝒲_t, l={(n_h,n_v)| (n_h,n_v)∈𝒲, n_h< n̅_t}, 
       𝒲_t, r={(n_h,n_v)| (n_h,n_v)∈𝒲, n_h> n̅_t},
denote the sets of the normal IRS elements that are on the left hand side and on the right hand side of the boundary defined by c_h=n̅_t, respectively. Based on the signal received in time slot t as well as the ML estimators made based on the signals received at time slots 0_- and 0_+, our goal is to detect which of Case 1, Case 2, and Case 3 is true at time slot t.

Suppose Case 1 is true. This indicates that all the IRS elements on the right hand side of the boundary c_h=n̅_t are in the normal state, i.e., 𝒲_t, r= {(n_h,n_v)|n_h>n̅_t}. In this case, define

    g_t, r^ C1=∑_n_h>n̅_t∑_n_v∈𝒩_vg_n_h,n_v, 
       g_t, l^ C1=g̅_ w-g_t, r^ C1,
as the estimations of ∑_(n_h,n_v)∈𝒲_t, rg_n_h,n_v and ∑_(n_h,n_v)∈𝒲_t, lg_n_h,n_v in (<ref>) under Case 1. Because z_t∼𝒞𝒩(0,σ^2I), the probability to receive y_t at time slot t given the estimations g̅_ e, g_t, l^ C1, and g_t, r^ C1 is

    p(y_t|g̅_ e,g_t, l^ C1,g_t, r^ C1) 
    
        =    1/(πσ^2)^Me^-‖y_t-hx_t-g̅_ex_t-g_t, l^ C1e^jϕ̅_t, lx_t-g_t, r^ C1e^jϕ̅_t, rx_t‖^2 /σ^2.
Note that based on the received signals at time slots 0_- and 0_+ given in (<ref>) and (<ref>), we estimate 2M variables in g_ e and g_ w via 2M equations. Therefore, it is expected that the ML estimators given in (<ref>) and (<ref>) are quite accurate. If Case 1 is true, then g_t, r^ C1 and g_t, l^ C1 given in (<ref>) and (<ref>) are also very accurate estimators of ∑_(n_h,n_v)∈𝒲_t, rg_n_h,n_v and ∑_(n_h,n_v)∈𝒲_t, lg_n_h,n_v. To summarize, the conditional probability given in (<ref>) will be quite high if Case 1 is true. Otherwise, if Case 1 is not true, then  g_t, r^ C1 and g_t, l^ C1 given in (<ref>) and (<ref>) are poor estimators. Therefore, a very large noise power of z_t can make the equation in (<ref>) hold given the above poor estimators. To summarize, the conditional probability given in (<ref>) will be quite low if Case 1 is not true. Consequently, we claim that Case 1 is true if and only if

    p(y_t|g̅_ e,g_t, l^ C1,g_t, r^ C1)>p̅_1,
where p̅_1 is a pre-determined threshold.



Next, if Case 2 is true, all the IRS elements on the left hand side of the boundary c_h=n̅_t are in the normal state, i.e., 𝒲_t, l= {(n_h,n_v)|n_hn̅_t}. Define

    g_t, l^ C2=∑_n_hn̅_t∑_n_v∈𝒩_vg_n_h,n_v, 
       g_t, r^ C2=g̅_ w-g_t, l^ C2,
as the estimations of ∑_(n_h,n_v)∈𝒲_t, lg_n_h,n_v and ∑_(n_h,n_v)∈𝒲_t, rg_n_h,n_v in (<ref>) under Case 2. Similar to (<ref>), the probability to receive y_t at time slot t given the estimations g̅_ e, g_t, l^ C2, and g_t, r^ C2 is

    p(y_t|g̅_ e,g_t, l^ C2,g_t, r^ C2) 
    
        =    1/(πσ^2)^Me^-‖y_t-hx_t-g̅_ex_t-g_t, l^ C2e^jϕ̅_t, lx_t-g_t, r^ C2e^jϕ̅_t, rx_t‖^2 /σ^2.
We claim that Case 2 is true if and only if

    p(y_t|g̅_ e,g_t, l^ C2,g_t, r^ C2)>p̅_2,
where p̅_2 is a pre-determined threshold.

At last, we claim that Case 3 is true if and only if both (<ref>) and (<ref>) do not hold,[If both (<ref>) and (<ref>) hold, it indicates that all the IRS elements are detected to be in the normal state, i.e., a diagnosis of the IRS is no longer needed. Because this paper assumes that there exist defective elements on the IRS, the above case is very unlikely to occur if p̅_1 and p̅_2 are properly selected.] i.e.,

    p(y_t|g̅_ e,g_t, l^ C1,g_t, r^ C1)≤p̅_1, 
        p(y_t|g̅_ e,g_t, l^ C2,g_t, r^ C2)≤p̅_2.













 §.§ Three-Phase Bisection Method



After introducing how to make the cutting plane, in the following, we propose a novel three-phase bisection method to estimate n_h, min and n_h, max in the horizontal dimension, based on the over-the-air measurements.

Specifically, define n_h, min^ lb,0=1 (n_h, max^ lb,0=1) and n_h, min^ ub,0=N_h (n_h, max^ ub,0=N_h) as the initial lower bound and upper bound of n_h, min (n_h, max). In time slots 0_- and 0_+, we set desired IRS phase shifts according to (<ref>) and (<ref>), keep a record of the received signals shown in (<ref>) and (<ref>), and make ML estimations of g_ e and g_ w based on (<ref>) and (<ref>). After the above initialization stage, we conduct the t-th iteration of the bisection method based on the signal received at time slot t via applying the detection method proposed in the previous subsection. Let n_h, min^ lb,t (n_h, max^ lb,t) and n_h, min^ ub,t (n_h, max^ ub,t) denote the lower bound and the upper bound of n_h, min (n_h, max) obtained after the t-th iteration of our bisection method. Depending on which bounds are updated, we divide the whole process of our proposed bisection method into three phases.


  §.§.§ Phase I

Phase I is the phase where the bounds of n_h, min and n_h, max are updated together at one iteration. Specifically, in the first a few iterations of the bisection method, n_h, min and n_h, max have the same lower bound and the same upper bound. For example, at the initialization stage, we have n_h, min^ lb,0=n_h, max^ lb,0=1 and n_h, min^ ub,0=n_h, max^ ub,0=N_h. If n_h, min^ lb,t=n_h, max^ lb,t and n_h, min^ ub,t=n_h, max^ ub,t are true at the t-th iteration, then at the (t+1)-th iteration, the boundary for estimating n_h, min, which is defined by the middle point of the regime [n_h, min^ lb,t,n_h, min^ ub,t], i.e., n_h, min^ md,t+1=(n_h, min^ lb,t+n_h, min^ ub,t)/2, and the boundary for estimating n_h, max, which is defined by the middle point of the regime [n_h, max^ lb,t,n_h, max^ ub,t], i.e., n_h, max^ md,t+1=(n_h, max^ lb,t+n_h, max^ ub,t)/2, are the same, i.e., n_h, min^ md,t+1=n_h, max^ md,t+1. Therefore, we can simultaneously update the bounds of n_h, min and n_h, max at the (t+1)-th iteration as follows. At time slot t+1, we set the boundary as c_h=n̅_t+1=n_h, min^ md,t+1=n_h, max^ md,t+1 and the phase shifts of IRS elements based on (<ref>) and (<ref>). Based on the signal received at time slot t+1, i.e., y_t+1, if Case 1 is true, i.e., (<ref>) holds, then we update the new bounds as n_h, min^ lb,t+1=n_h, max^ lb,t+1=n_h, min^ lb,t and n_h, min^ ub,t+1=n_h, max^ ub,t+1=⌊ n_h, min^ md,t+1⌋. If Case 2 is true, i.e., (<ref>) holds, then we update the new bounds as n_h, min^ lb,t+1=n_h, max^ lb,t+1=⌈ n_h, min^ md,t+1⌉ and n_h, min^ ub,t+1=n_h, max^ ub,t+1=n_h, min^ ub,t. If Case 3 is true, i.e., both (<ref>) and (<ref>) hold, then we update the new bounds as n_h, min^ lb,t+1=n_h, min^ lb,t, n_h, min^ ub,t+1=⌊ n_h, min^ md,t+1⌋, n_h, max^ lb,t+1=⌈ n_h, max^ md,t+1⌉, and n_h, max^ ub,t+1=n_h, max^ ub,t.

Note that if Case 1 or Case 2 is true, we still have n_h, min^ lb,t+1=n_h, max^ lb,t+1 and n_h, min^ ub,t+1=n_h, max^ ub,t+1 at the (t+1)-th iteration. Otherwise, if Case 3 is true, we have n_h, min^ lb,t+1≠ n_h, max^ lb,t+1 and n_h, min^ ub,t+1≠ n_h, max^ ub,t+1 at the (t+1)-th iteration. Define t^ I as the index of the iteration where Case 3 occurs for the first time. Then, Phase I of our proposed bisection method ends at the t^ I-th iteration, when the boundary passes through the cluster of defective IRS elements for the first time such that the bounds of n_h, min and n_h, max are different.

One example about the update of the bounds in three cases of Phase I can be found in Fig. <ref>.



  §.§.§ Phase II

Phase II starts at time slot t^ I+1 and is the phase where merely the bounds of n_h, min are updated until n_h, min is found. At iteration t+1 with t≥ t^ I, the new boundary is c_h=n_h, min^ md,t+1=(n_h, min^ lb,t+n_h, min^ ub,t)/2. Based on the signal received at time slot t+1, if Case 2 is true, we set n_h, min^ lb,t+1=⌈ n_h, min^ md,t+1⌉ and n_h, min^ ub,t+1=n_h, min^ ub,t. If Case 3 is true, we set n_h, min^ lb,t+1=n_h, min^ lb,t and n_h, min^ ub,t+1=⌊ n_h, min^ md,t+1⌋. Define t^ II as the index of the iteration where n_h, min^ lb,t^ II=n_h, min^ ub,t^ II. Then, Phase II ends at the t^ II-th iteration, and n_h, min is estimated as n_h, min^ lb,t^ II.


  §.§.§ Phase III

Phase III starts at time slot t^ II+1 and is the phase where merely the bounds of n_h, max are updated until n_h, max is found. At iteration t+1 with t≥ t^ II, the new boundary is c_h=n_h, max^ md,t+1=(n_h, max^ lb,t+n_h, max^ ub,t)/2. Based on the signal received at time slot t+1, if Case 1 is true, we set n_h, max^ ub,t+1=⌊ n_h, max^ md,t+1⌋ and n_h, max^ lb,t+1=n_h, max^ lb,t. If Case 3 is true, we set n_h, max^ lb,t+1=⌈ n_h, max^ md,t+1⌉ and n_h, max^ ub,t+1=n_h, max^ ub,t. Define t^ III as the index of the iteration where n_h, max^ lb,t^ III=n_h, max^ ub,t^ III. Then, Phase III ends at the t^ III-th iteration, and n_h, max is estimated as n_h, max^ lb,t^ III.





In the following, we provide an example to illustrate how the above three-phase bisection approach works to iteratively find n_h, min and n_h, max in the horizontal domain. In this example, we assume that on an IRS consisting of 4× 4 elements, all the defective IRS elements are located at the regime ℰ={(n_h,n_v)|n_h∈{2,3}, n_v∈{2,3}}, as shown in Fig. <ref>. Specifically, at the initial stage, we have n_h, min^ lb,0=n_h, max^ lb,0=1 and n_h, min^ ub,0=n_h, max^ ub,0=N_h=4. As shown in Fig. <ref>, at the first time slot, which is Phase I, the boundary to cut in the horizontal domain is c_h=2.5, which is useful for updating the bounds for both n_h, min and n_h, max. Based on the detectors shown in (<ref>)-(<ref>), Case 3 should be detected. In this case, we update n_h, min^ ub,1=2, n_h, max^ lb,1=3, and keep n_h, min^ lb,1=n_h, min^ lb,0=1 and n_h, max^ ub,1=n_h, max^ ub,0=4. Because Case 3 has occurred, Phase I ends after the first iteration.
As shown in Fig. <ref>, at the second time slot, which is Phase II, the boundary to cut in the horizontal domain is c_h=1.5, which is useful for checking whether n_h, min>1.5 or n_h, min<1.5. Based on the detectors shown in (<ref>)-(<ref>), Case 2 should be detected. In this case, we update n_h, min^ lb,2=2, and keep n_h, min^ ub,2=n_h, min^ ub,1=2. Because n_h, min^ lb,2=n_h, min^ ub,2=2, Phase II ends and n_h, min=2 can be determined.
As shown in Fig. <ref>, at the third time slot, which is Phase III, the boundary to cut in the horizontal domain is c_h=3.5, which is useful for updating the bounds for n_h, max. Similarly, based on the detectors shown in (<ref>)-(<ref>), Case 1 should be detected. In this case, we update n_h, max^ ub,3=3, and keep n_h, max^ lb,3=n_h, max^ lb,1=3. Because n_h, max^ lb,3=n_h, max^ ub,3=3, Phase III ends and n_h, max=3 can be determined. After three iterations, the values of n_h, min and n_h, max are known. We can also apply the above approach to estimate n_v, min and n_v, max in the vertical domain. 














§ NUMERICAL RESULTS

In this section, we provide a numerical example to verify the effectiveness of the proposed method for localizing the defective IRS elements based on over-the-air measurements. We assume that there are N=1024 elements on an IRS, with N_h=N_v=32. Moreover, we assume that the transmitter, the receiver, and the IRS are deployed in the near-field regime of each other. Therefore, h, r_n_h,n_v, and u_n_h,n_v are generated based on the near-field LOS channel model <cit.>. In each realization, we randomly generate the locations of the transmitter, the receiver, the IRS, and the defective elements on the IRS. We claim that the diagnosis is accurate in one realization only when n_h, min, n_h, max, n_v, min, and n_v, max are all correctly estimated. Fig. <ref> shows the performance of our proposed scheme. It is observed that the diagnosis accuracy is around 95% when M=1, and very close to 100% when M≥ 3. This is because with more receive antennas, we have more observations to localize the defective elements. This numerical example (together with many other examples that are not shown here due to space limitation) verifies the possibility to perform over-the-air diagnosis of the IRS.   





§ CONCLUSION

In this paper, we investigated the possibility to find the defective IRS elements based on the over-the-air measurements. To achieve this goal, we designed a novel three-phase bisection method, under which we can iteratively remove half of the interested regime that does not contain the boundary of the defective IRS elements. Our results provided a new approach to perform diagnosis of the IRS.

IEEEtran





 Paper Title: i-CardiAx: Wearable IoT-Driven System for Early Sepsis Detection Through Long-Term Vital Sign Monitoring.









i-CardiAx: Wearable IoT-Driven System for Early Sepsis Detection Through Long-Term Vital Sign Monitoring


    
Kanika Dheman1,
Marco Giordano1,
Cyriac Thomas1,
Philipp Schilk1,
Michele Magno1


1{dhemank, mgiordano, cythomas, schilkp, magnom}@ethz.ch
1Center for Project Based Learning, Dept. of Information Technology and Electrical Engineering, ETH Zürich, Switzerland

    August 5, 2024
=========================================================================================================================================================================================================================================================================



§ ABSTRACT 
Sepsis is a major cause of premature mortality, high healthcare costs, and disability-adjusted life years. Digital interventions such as continuous cardiac monitoring solutions can help to monitor the patient's status and provide valuable feedback to clinicians to detect early warning signs and provide effective interventions. This paper presents i-CardiAx, a wearable sensor based on low-power high-sensitivity accelerometers that measures vital signs essential for cardiovascular health monitoring, namely, heart rate (HR), blood pressure (BP), and respiratory rate (RR). A dataset has been collected from 10 healthy subjects with the i-CardiAx wearable chest patch to develop low-complexity, lightweight vital sign measurement algorithms and evaluate their performance. The experimental evaluation demonstrates high-performance vital sign measurement for RR (-0.11±0.77 breaths per minute), HR ( 0.82± 2.85 beats per minute), and systolic BP (-0.08 ± 6.245 mm of Hg). The proposed algorithms are embedded on the ARM Cortex-M33 processor supporting Bluetooth Low Energy (BLE). Estimation of HR and RR achieved an inference time of only 4.2 ms and 8.5 ms for BP. 
Moreover, a multi-channel quantized Temporal Convolutional Neural (TCN) Network has been proposed and trained on the open-source HiRID dataset to have a large number of patients with data for sepsis ground truth. The model has been trained and evaluated using only digitally acquired vital signs as input-data that could be collected by i-CardiAx to detect the onset of Sepsis in a real-time scenario. The TCN has been fully quantized to 8-bit integers and deployed on i-CardiAx.The network showed a median predicted time to sepsis of 8.2 hours with an energy per inference of 1.29mJ.
i-CardiAx has a sleep power of 0.152 mW and an averages a power of 0.77 mW for always-on sensing and periodic on-board processing and BLE transmission. With a small 100 mAh battery, the operational longevity of the wearable has been estimated at two weeks (432 hours) for measuring the three cardiovascular parameters (HR, BP and RR) at a granularity of 30 measurements per hour per vital sign, running inference every 30 minutes. Thus, the wearable i-CardiAx system can provide a method to monitor the cardiovascular parameters of patients with energy-efficient, high-sensitivity sensors to provide predictive alerts for life-threatening adverse events of sepsis, over a long period of time.



cardiovascular parameter monitoring, wearable, low power sensor nodes, continuous monitoring





§ INTRODUCTION

Continuous monitoring of vital physiological parameters is crucial within intensive care units (ICUs), particularly for high-risk situations like premature birth or infants undergoing surgery. Sepsis, a potentially fatal syndrome resulting from infection-induced organ dysfunction, poses a significant burden in terms of morbidity and mortality, contributing to approximately 11 million annual deaths<cit.>. Its presentation varies across patients, making early identification challenging in different patient phenotypes<cit.>. The prevalence of sepsis differs based on a country's economic demographic, but the global estimate stands at around 30 million cases each year, with a notable incidence among neonates and children, accounting for approximately 3 million and 1.2 million cases per year, respectively<cit.>. Patient management is highly time-sensitive as delayed treatment increases the probability of mortality by 0.42% per hour of delay to administer antibiotics in sepsis patients <cit.>. Hence, identifying the onset of sepsis early and rapid initiation of antibiotics and supportive management are critical as no specific therapy is available to clinicians. 


The current approach to sepsis screening primarily depends on monitoring vital signs and utilizing standardized scores such as the Systemic Inflammatory Response Syndrome (SIRS) score, Early Warning Score (EWS), and the Quick Sepsis-Related Organ Failure Assessment (qSOFA)<cit.>. Nevertheless, these scoring systems are episodic and necessitate repeated measurements by medical personnel. Numerous factors impede effective sepsis monitoring of patients. Ongoing challenges in clinical implementation encompass the diverse clinical presentations of sepsis, the continuous and real-time tracking of vital signs, delays in accessing electronic medical records (EMR) due to third-party validation, a lack of medical decision support, alert fatigue, and information overload. In neonatal and pediatric ICUs, monitoring vital signs is further complicated by the presence of multiple hard-wired and inflexible connections to the delicate and developing skin of infants. These monitoring platforms pose significant risks of iatrogenic skin injuries, hinder skin-to-skin contact between the neonate and parent, complicate basic clinical tasks, and are incompatible with magnetic resonance and x-ray imaging.


In addition, pre-existing co-morbidities, such as cardiovascular disease (CVD) could also increase the risk of sepsis significantly <cit.>. Unfortunately, non-communicable diseases (NCD) are the dominant health challenge in the twenty-first century and account for about two-thirds of all reported global deaths <cit.>. Cardiovascular diseases, in turn, account for 50% of these global NCD-related deaths and are a major barrier to sustainable health development <cit.>. The United Nations has set a goal under Sustainable Development Goal 3 to reduce premature mortality due to NCDs, and specifically CVDs by 30% until 2030 <cit.>. This necessitates the development and implementation of interventions that are cost-effective, unobtrusive, and comfortable to continuously monitor the major risk factors. In this regard, digital health solutions can provide an invaluable contribution to reducing adverse CVD outcomes, as shown by recent research <cit.>. 


To effectively track the onset of sepsis, particularly with a focus on monitoring CVD parameters, the most crucial vital signs are blood pressure, heart rate, and respiratory rate to prevent the occurrence of adverse events<cit.>. Currently, wearable cardiac monitoring solutions, including Holter monitors, event recorders, ECG patches, wristbands, and smart textiles, predominantly offer heart rate measurements and electrocardiograms  <cit.>. These monitoring devices have limitations as they either do not provide all essential cardiovascular parameters (HR, RR, and BP) simultaneously, require multiple wires for frequently replaced electrodes, or have a design that hinders continuous usage<cit.>. Additionally, certain high-risk populations, such as geriatric or paraplegic patients, require on-body alerts for sudden increases in cardiovascular parameters, especially systolic blood pressure <cit.>. However, the monitoring systems currently in use rely on external platforms for signal conditioning and communication to ensure accurate interpretation of bio-signals<cit.>.



The ongoing IoT advancements, low-power sensors, and the integration of machine learning into low-power processors are transforming data collection, processing, and interpretation<cit.>. This evolution is particularly significant for various applications, especially in the realm of wearables and medical devices, although it presents certain challenges <cit.>. One major breakthrough is the use of low-power sensors, which optimize energy consumption, making them ideal for battery-operated wearables. By pairing these sensors with energy-efficient processors, the operational lifetime of devices is significantly extended. In-sensor data analytics are a key feature here, eliminating the need for additional processing hardware, ensuring clean data for clinicians and decision support systems, and enhancing device efficiency. Nonetheless, miniaturized and thin form factor wearables have strict constraints<cit.>, including limited power<cit.>, accuracy, and device longevity<cit.>. Flexible hybrid sensors have been proposed as a solution but introduce potential reliability issues due to the integration of flexible and rigid electronics. Furthermore, the challenge of limited sensor lifespans due to battery dependence persists. Constant recharging and battery replacements are inconvenient and limit device capabilities. Therefore, addressing power challenges, whether through innovative power sources, energy harvesting, or more efficient processors, is crucial to fully unlock the potential of IoT in real-world applications, particularly in healthcare and beyond. In-sensor data analytics are crucial because they eliminate the need for additional hardware for processing and because they provide artifact-free information ready for interpretation by clinicians or medical decision support systems. Stringent constraints are set on the useable power, sensor accuracy, and lifetime of wearable devices due to the miniaturized and thin form factors. Flexible hybrid sensors with both flexible and rigid electronics have been proposed to circumvent such an issue, but these introduced reliability issues due to additional stresses in the electrical circuit<cit.>.In this way, the lifetime of autonomous smart sensors is limited in all real-world applications by the use of batteries.


This paper proposes an energy-efficient and effective approach to assist in prognosis by continuously monitoring multiple cardiovascular vital signs to aid in disease management of sepsis patients by using only digital biomarkers. In particular, the paper presents the design and the implementation in hardware and software of emphi-CardiAx, a wearable system for continuously monitoring cardiovascular parameters of cuffless blood pressure, heart rate, and respiratory rate, medical decision support can be provided at the point of care using energy efficient multi-sensor smart patch with on-board tiny machine learning based algorithms in a mesh enabled IoT setup. The system is designed to be power efficient, hence, ensuring approximately 2 weeks of monitoring with small batteries with a maximum capacity of 100 mAh. To evaluate the capability of using only ultra-low-power sensors, the vital signs are extracted from two low-power accelerometers which are placed on the chest in a comfortable form factor for unobtrusive monitoring over long durations. The algorithms for vital signs measurement are designed to be low in complexity and run on a low-power ARM Cortex-M33 microcontroller (MCU).  Autonomous, low-power, and mesh-enabled multi-sensor wearable systems based on the always-on smart sensing paradigm can continuously acquire, process, and report physiological data in real-time and are trained to autonomously detect the onset of neonatal sepsis symptoms. Thus eliminating delays in electronic medical records and reducing alarm fatigue.


 The main contributions of this paper are:

    
  * Design and development of an energy-efficient low-power sensing node based only on two accelerometers for cardiovascular parameter monitoring and onboard algorithm processing.
    
  * Dataset creation of acceleration data on the chest at two different anatomical sites and experimental evaluation of vital signs: cuffless blood pressure (BP), heart rate (HR), and respiratory rate(RR), from bi-nodal IMU system.
    
  * Development of deep learning model based on only cardiovascular vital sign extraction algorithms with low latency and memory requirements for implementation on the MCU.
    
  * Experimental evaluation of the performance of proposed hardware and software solution, in terms of power consumption, energy, accuracy, and lifetime. 

   





§ RELATED WORK

Research in predicting sepsis onset has leveraged data from electrical medical records using both statistical methods for patient survival analysis and deep learning models. Most statistical models assume a parametric function that subsumes the effect of input parameters to model a stochastic process. Such an assumption of a parametric relationship between the input and the hazard may not fully exemplify the heterogeneous nature of sepsis representation. Weilbull-Cox proportional hazards modeled 65 features extracted from the Emory University hospitals dataset and MIMIC-III dataset <cit.>. Similarly, a Cox proportionality hazard model used 54 features to calculate a risk score for severe sepsis and septic shock<cit.> with a prediction horizon of 28.2 hours before the onset of septic shock with a sensitivity of 0.85 and specificity of 0.67. Similarly many machine learning and deep learning algorithms on open-source electrical medical records from the ICUs have developed algorithms to predict sepsis onset. Temporal convolution networks (TCN) have been extensively applied to this task with one approach implementing a k-nn-based model with dynamic time warping that leveraged data sparsity by interpreting the missing data with a Gaussian process <cit.>. 44 irregularly sampled laboratory and vital parameters were used for predicting the onset of sepsis 0 to 7h preceding sepsis onset. Another work used a TCN for sepsis prediction by addressing the missingness of data by masking the missing data in a parallel branch of the neural network instead of performing forward or back imputation <cit.>. Gradient-boosted trees have also been used with the InSight model where data from multi-center, multi-ward settings was labeled with the SIRS criteria at the University of California, San Francisco from 2011 to 2016 <cit.>. This gave access to a large dataset with 90,353 patients for training on the model. This is orders of magnitude larger than any open-source EMR dataset and the result of this is seen in the high AUROC reported by the authors and reduced false positives <cit.>. 


However, most of these models use a large input vector dimension using almost all of the data in the electrical medical records. While the chemical bio-markers extracted at the laboratory are definitive in their prognosis, they are taken sporadically, are based on the analysis of the clinician, and require dedicated infrastructure and resources.  Also, the high input dimensions necessitate the use of dedicated computation infrastructure for providing medical decision support. All clinical environments may not be equipped with the required resources to provide such diagnosis, especially in low to middle-income countries (LMICs).


In contrast to many existing approaches that rely on extensive feature sets and resource-intensive models, our study takes a different direction. Our proposed approach has focused on creating a more efficient and accessible solution for predicting sepsis onset. Leveraging a carefully curated dataset and implementing a lightweight model, our approach seeks to address the limitations associated with high-dimensional input data derived from electronic medical records. While previous models have employed large input vectors that encompass a wide array of data from electronic medical records, we have prioritized a more streamlined approach. Our dataset selection process is meticulous, emphasizing data elements that are not only crucial for prognosis but are also available at a frequency that supports timely predictions. This approach aims to reduce the burden of dedicated infrastructure and resources required for sporadic chemical biomarker analysis, a common feature in many sepsis prediction models.


Recent advances in sensor technologies have led to many commercial monitoring wearables. However, most wearables focus on measuring the ECG and the HR from the chest. The Vivalink wearable ECG monitor provides multiple vitals such as HR and RR along with the ECG <cit.>. However, being an electrode-based system, it requires periodic electrode replacement and does not provide an estimate of the BP.

In such a scenario, low-power sensors such as accelerometers with high sensitivity are gaining popularity to provide solutions that allow long-term monitoring with low-complexity, lightweight algorithms that do not depend on the subject under test or the ambient conditions<cit.>. The feasibility of measuring and co-relating the PTT with the BP was demonstrated with SCG on the carotid artery<cit.>. However, the placement of vibrational transducers on the neck can be excessively corrupted by speech, swallowing, and motion artifacts.





Cuffless BP estimation has been extensively pursued in research with photoplethysmography (PPG) and ECG binodal systems that attempt to capture the pulse transit time (PTT) from the ECG at the chest to the pulse arrival measured by the PPG at the fingertip <cit.>. Such systems are inconvenient for prolonged use and suffer the limitations of optical sensor-based PTT approaches, namely ambient light condition, skin morphology and proximity to skin <cit.>. A promising approach is seismocardiography (SCG) using a 3-axis accelerometer which is independent of the individual under test and the ambient conditions to estimate the BP via vibrations on the chest<cit.>. The feasibility of measuring and co-relating the PTT with the BP was demonstrated with SCG on the carotid artery<cit.>. However, the placement of vibrational transducers on the neck can be excessively corrupted by artifacts due to swallowing, talking, and head movements and the neck is a conspicuous position that intrudes upon the privacy of the user. 




This paper distinguishes itself from existing research in the field of blood pressure estimation in several significant ways. While the state-of-the-art has primarily focused on photoplethysmography (PPG) and ECG binodal systems to estimate blood pressure by capturing pulse transit time (PTT), our approach offers a fresh perspective using only two novel sensitive low-power accelerometers. In fact, many previous approaches rely on complex PPG and ECG systems that require sensor placement on both the chest and fingertip, making them inconvenient for prolonged use. Those sensors are susceptible to limitations associated with optical sensor-based PTT methods, including sensitivity to ambient light conditions, skin morphology, and sensor proximity to the skin. In contrast, this work presents an innovative approach using seismocardiography (SCG) with a 3-axis accelerometer. This method is not only independent of the individual under test and ambient conditions but also offers a more reliable means of estimating blood pressure through chest vibrations. Furthermore, while some prior research explored the feasibility of measuring and correlating PTT with blood pressure using SCG on the carotid artery, our work takes a different direction. We recognize the limitations of vibrational transducers placed on the neck, which can be prone to artifacts from swallowing, talking, and head movements. Additionally, this placement is conspicuous and may infringe upon the user's privacy. 

This paper endeavors to bridge this gap in wearable cardiac monitoring by presenting a multi-vital sign system that not only measures blood pressure but also heart rate (HR) and respiratory rate (RR). Beyond the accuracy of vital sign extraction, we delve into the implementation of on-device data processing, proposing a fully quantized temporal convolutional neural network, which enhances the system's usability and efficiency. Furthermore, we estimate the operational longevity of our compact wearable system, addressing a critical aspect of practicality and user-friendliness. In summary, our approach offers a more versatile and user-friendly solution to the challenges associated with blood pressure estimation and wearable cardiac monitoring.
 


§ SYSTEM DESCRIPTION






 §.§ Hardware Overview




At the heart of the i-CardiAx is the BLE-enabled NRF5340 SoC which communicates and processes data. The SoC is clocked at 64 MHz and provides 512KB RAM, 1MB flash, and supports Bluetooth Low Energy (BLE). In addition, the system has the MAX77654 ultra-low power power-management IC that regulates the system power, an onboard flash, and 6-axis IMU (LSM6DSM). As seen in Figure <ref>, these components form the main processing core of the i-CardiAx and have been designed as a general-purpose controller for all subsystems. Using a high-density connector, this processing core can be attached to an additional circuit board featuring all sensors required for a given application and deployment. Attached on the sensing core, are both the optical emitters and sensors required for PPG sensing, a body temperature sensor, and two flexible interconnect-PCBs connecting to two side-wing PCBs, which in turn hold the batteries, micro-SD cards, and ECG-electrodes. The system is powered using 2 50mAh Li-ion batteries in parallel (total source capacity of 100 mAh). Finally, at the extreme ends of the device, are two high-performance always-on low-power LSM6DSV16BX inertial measurement units that can accurately measure the small vibrations related to blood pulse flow on the surface of the chest. The i-CardiAx only uses the 3-axis accelerometers for extracting cardiac features on the chest at a sampling rate of 120Hz and a range of ± 2g.



 §.§ Operational System Overview



 The system-level overview is shown in Figure<ref>. The main subsystems are:


    
  * A finite state machine (FSM) that controls the transition between different operating modes, such as idle, charging, and sensing. Transitions in the FSM are triggered by a predefined set of events that can be raised from all subsystems. Based on these events, the FSM will in turn control and configure all other subsystems using their public API It houses other components and peripherals that are used throughout the system, such as the 3-channel LED driver, IS31FL3194, and low-voltage translating 8-bit I2C-bus I/O ExpanderIO, PI4IOE5V6408.
    
  * The PWR subsystem contains all power and resource management logic. It periodically monitors the battery charge level, charger insertion and removal, and battery temperature.
    
  * The DATAMGR subsystem is responsible for collecting data produced by the onboard sensors and storing, transmitting, or retrieving it. All data storage in patchOS is built around protobufs<cit.>, which are compiled for this application using nanopb<cit.>. Every piece of data that needs to be logged or transmitted gets encapsulated and encoded as protobuf-defined data chunks allowing the DATAMGR subsystem to operate on variable-length blocks of binary data. This allows any changes to the data format to be made easily and without having to update any data management logic. For real-time transmission, these chunks are sent directly to the PC. For data logging, a collection of chunks is grouped into a so-called bundle, which is then stored in non-volatile storage.  Once the data is retrieved from local storage or streamed in real-time, the receiving side may easily decode the message using the protobuf definition and a protobuf library, making post-processing simple.
    
  * The COMM subsystem enables Bluetooth and USB communication. This enables data to be offloaded, data to be streamed, and the operation of the device controlled.
    
  * The SENSOR subsystem controls the management logic and interfaces to the different sensors available. Ultra-ow power, low noise (20 μg /√(Hz)) inertial measurement units, LSM6DSV16BX, provide a 6-axis accelerometer and gyroscope with charge variation (QVAR) capabilities along with a digital temperature sensor MAX30208.






 §.§ Low-Power Design


The i-CardiAx system was designed for low-power operation in order to ensure maximum operational longevity using low-power accelerometers (Active Power: 370 μW; Sleep Power: 12 μW ). The power-efficient system architecture is based on always-on low-power integrated circuits and energy-efficient power management. A real-time operating system (RTOS) handles the processing and control of the subsystems that allow the microcontroller unit (MCU) to stay in sleep mode as needed. To improve the energy efficiency the node has been designed to have a total sleep power of 0.155 mW. Experimental evaluation of the active power and sleep power will be illustrated in the following sections.  


Continuous operation mode is defined as the state where the sensors are always-on and the processor is woken only for two tasks: data collection from the FIFO and the processing of the vital sign algorithms. In continuous mode, the algorithms for HR and BP are executed every 2s, while the RR algorithm is executed every 30s. This frequency of tasks was heuristically chosen such that the accuracy of vital sign estimation is not compromised. At the end of every 30s, the average values of HR, BP, and RR are calculated. Hence, when the sensor is always on and vital signs are calculated in the order explained above, there are 30 measures (for the measured vitals of HR, RR, and BP) in an hour. Operational longevity can be further increased by reducing the number of measurements processed per hour and keeping the sensor node in sleep mode longer.

    




 §.§ Wearable Design

The electronic layout was optimized for unobtrusive wearable applications. The main design objective was to record the heartbeat, respiration pattern, and transiting blood pulses. For this purpose, two accelerometers were used with one placed at the xiphoid process on the chest to capture the heartbeat and respiration patterns. The second accelerometer was placed at a distance of 12 cm, towards the suprasternal notch, from the first accelerometer to record the transiting blood pulse along the aortic pathway. This distance was chosen with the aim to capture the blood pulse from the xiphoid process to the aortic arch <cit.>. Figure <ref> shows the wearable design with a compact form factor of 12 cm × 2.4 cm × 0.5 cm.




§ METHODS

Our primary objective is to demonstrate the feasibility of achieving sepsis alerts at the edge using only digital biomarkers, which can be extracted with two novel high-sensitive 3-axes accelerometers, while emphasizing the energy efficiency, and intelligence within i-CardiAx. In the existing literature, studies that employed a restricted set of biomarkers for model training consistently incorporated the following vital signs: heart rate (HR), respiratory rate (RR), body temperature, and systolic blood pressure. <cit.>. By leveraging these digital biomarkers, i-CardiAx offers a practical solution for sepsis detection without the need for invasive or additional sensors. Our system intelligently processes the data collected from these vital signs, enabling real-time analysis and timely alerts for sepsis onset. The careful selection and integration of low-power sensors ensure efficient power consumption, enabling prolonged device operation and minimizing the need for frequent battery replacements.



 §.§ Setup and Materials

The Polar H10<cit.> was used as the reference device for HR and RR while the GIMA Ambulatory Blood Pressure Monitoring system was used as the reference for the blood pressure. Data was collected from 10 healthy subjects ( 5 males and 5 females) with the following demographics:  age of 25 ± 4.78 years, weight in a range of 67 ± 7.97  Kg, and height in the range of 172.8 ± 8.24 cm.



 §.§ Experimental Protocol

For the evaluation of vital signs, data was recorded in a sitting position. Subjects were asked to sit 5 minutes before starting the recording to stabilize the blood pressure due to hydro-static pressure changes from standing to sitting position.  The experiment was carried out in 3 segments of continuous measurements with the i-CardiAx wearable for a period of 2 minutes each. All vitals of blood pressure, HR, and RR were measured before and after each segment. For the final comparison with ground truth, the mean of the estimates over a segment of 2 minutes was calculated to compare with the reference device measurement.




 §.§ Algorithms for Vital Sign Extraction



    
  * Heart Rate (HR): The algorithm for extracting the peaks of the heart activity is shown in Figure <ref>. The Euclidean norm of the lateral and vertical axis is calculated which is then band-pass filtered between 10 Hz and 40 Hz (differentiation on the MCU). Thereafter, the Shannon energy envelope of the signal is calculated. The heartbeat peaks are located at the positive zero crossings of the Hilbert transform of the envelope, which is used to calculate the heart rate in windows of 2s. To evaluate the estimation error, the calculated average HR is compared to the reference device to report the average error and the variation of the error across subjects.   

    
  * Cuffless Blood Pressure (BP): The same peak identification process, as described above for the HR estimation, is repeated for the two accelerometers. As seen in Figure <ref> (d), upon identification of peaks from both sensors, the pulse transit time is calculated as a difference of peaks located by ACC2 (12 cm away from the xiphoid process) and peaks located at ACC1 ( at the xiphoid process). The BP is evaluated for each subject from the mean PTT value over 2 minutes. This is due to the design of the experimental protocol, where the ground truth is taken from the cuff-based reference device before and after the 2-minute segment. For evaluation of the estimated blood pressure, a leave one out approach was used where n-1 subjects were used to extract the model of the blood pressure estimate using the PTT, and the error evaluation was done on the n^th subject. This process was repeated for all subjects to calculate the variation of the error for blood pressure estimation.

    
  * Respiratory Rate (RR): Only the vertical z-axis is used for respiratory rate calculation over a window of 30s. The signal is downsampled to 15 Hz (8Hz on the MCU) to reduce the data size for computation. A Fast Fourier Transform is performed on this 30s data window to identify the maximum power in the frequency spectrum of 0.05Hz and 0.78 Hz to calculate the respiratory rate. On the MCU the same algorithm for HR extraction is used on the downsampled accelerometer signal for memory constraint reasons.










 §.§ On-board Processing

The performance of the developed algorithm was evaluated on the ARM Cortex-M33 core on the i-CardiAx system. The power consumption and execution time calculation of the algorithms on the proposed device was tested with a Power Profiler Kit II from Nordic Semiconductors<cit.>. 


Figure <ref> shows the signal processing pipeline and the optimized CMSIS-DSP functions implemented on the ARM Cortex-M33 microcontroller on board of i-CardiAx. The differentiation is obtained with a subtraction operation, the Shannon energy with the multiplication of the power, and the logarithm of the power of the differentiated signal. Then a smoothing moving average is implemented with a convolutional filter before extracting the Hilbert Transform. The Hilbert Transform is obtained by convolving a windowing function to the smoothened Shannon energy, which is implemented by extracting the frequency components of the signal, windowing them, and taking the imaginary component of the inverse transform of the obtained signal. This last step is shown in Figure <ref>, where the zero crossings, corresponding to the peak positions, are highlighted in green.


Blood pressure is extracted by taking the pulse transit time between the identified peaks of the two accelerometers which capture the heartbeat at the two extremities. A model between the pulse transit time and the ground truth BP is deduced to estimate the BP values.  







 §.§  On-device Model Architecture for Sepsis Onset Detection


Building on top of <cit.>, we integrated a Temporal Convolutional Network (TCN) architecture for sepsis onset detection in our prototype.
Currently, sepsis detection relies on the comprehensive analysis of various parameters in electronic medical records. Nevertheless, the prospect of ubiquitous sepsis monitoring via wearable devices at the point of care hinges on the ability to identify sepsis onset using digital health biomarkers, particularly vital signs. To determine the feasibility of recognizing sepsis onset exclusively through vital signs, we constructed a retrospective model using solely digitally captured vital signs from electronic medical records. Subsequently, we developed and assessed a real-time sepsis onset detection model, aiming to identify sepsis occurrences at specific intervals.



  §.§.§ Dataset

To train and test a neural network to identify sepsis onset early, an open-source medical data set, HiRID was used <cit.>. The HiRID dataset comprises 34 thousand patient records from the Department of Intensive Care Medicine at Bern University Hospital in Switzerland (ICU). It includes anonymized demographic information, real-time measurements from bedside monitors, usage of medical devices like mechanical ventilation, observation notes by healthcare providers, laboratory-acquired biochemical markers, administered drugs, fluids, and nutrition .






  §.§.§ Inclusion Criteria and sepsis-onset labeling
 Patient selection adhered to specific inclusion criteria in line with prior research. These criteria encompassed a minimum hospital stay of 24 hours, a minimum age of 18 years, exclusion of patients who received antibiotics within the initial 7 hours of admission to the ICU, and exclusion of patients with a time to sepsis onset of less than 4 hours (as sufficient time is needed to identify patterns and thresholds by the machine learning algorithms). To ensure a balanced representation of positive and negative cases during training, the maximum duration of ICU stay was limited to 48 hours. Sepsis onset timing was determined in accordance with the sepsis-3 criteria <cit.>. A window for suspicion of infection was identified by pinpointing the time of antibiotic administration and creating a time window extending 48 hours before and 24 hours after this moment. For a patient, sepsis onset was determined when the hourly change in the Sequential Organ Failure Assessment (SOFA) score was greater than or equal to 2. It was assumed that sepsis-specific features would be present from the outset, so each 4-hour data window was labeled as class 1 for sepsis-positive cases and class 0 for all other patients. Following the application of these criteria, the dataset consisted of 1058 sepsis-positive patients and 7635 sepsis-negative patients, resulting in a class imbalance of 13.9%. Furthermore, to enable real-time prediction of risk leads to labelling each time window of 1 hour with a label of 1 for a positive class and a label of 0 for a control case. The data was split into training and test sets with an 80/20 ratio. Importantly, only the training set was balanced, ensuring the test set remained unaltered for a fair comparison under real-life conditions.

Figure <ref>(a) illustrates the distribution of sepsis-positive patients concerning the time to sepsis onset. The histogram reveals that over half of sepsis-positive patients develop sepsis within the first 8 hours of their ICU stay. This presents a challenge due to the limited data available for training machine learning models and exacerbates the class imbalance. Figure <ref>(b)  shows the distribution plots of the vital signs in the EMR that can be digitally acquired via wearable devices for both sepsis-positive and negative patients. Notably, some vital signs, such as blood pressure and heart rate, exhibit significant differences in distribution between the two groups.







  §.§.§ Temporal convolution network (TCN)

 A multi-head Temporal Convolutional Network (TCN) has been proposed and designed with a focus on implementation on low-power microcontrollers. TCNs can identify causality in time series data and are well-suited for real-time sepsis detection <cit.>. The TCN architecture, illustrated in Figure <ref>, differs from previous sepsis detection models utilizing TCN architectures <cit.> in that it leverages the causal relationships within each vital sign individually with a singular TCN at first which is followed by concatenating and flattening the outputs from all 'n' TCNs in the final fully connected layer. such a  design aims to learn multivariate relationships in the final dense layers, thus enabling the model to learn distinctive features from each input stream.


 As shown in Figure <ref>, the TCN uses only digitally acquired vital signs. Model optimization with respect to model hyper-parameters, such as the number of layers, convolution filters, learning rate, and schedule, a weight-informed Neural Architecture Search (NAS) was utilized <cit.>. The optimized architecture features 4 layers of dilated causal convolution, with a power-of-2 exponential increase in dilation size to capture longer-range dependencies in time-series data, along with 32 filters in each layer. The output of the convolution layer is flattened and passed into a dense layer comprising 32 neurons. Each convolution layer has a kernel dimension of 3, and batch normalization follows each convolution layer. A max pooling layer with a kernel and stride of 2 and a ReLU activation function is applied. The final layer employs a Sigmoid function, providing an interpretable output as a probability.



  §.§.§ Input data vector

The input of the model consists of 4 hours of data sampled at 2 samples/minute using the high-granularity HiRID dataset. Digital biomarkers from the EMR that were considered while developing the model are the heart rate, systolic blood pressure, respiratory rate, and core body temperature. The selection is motivated by the feasibility of acquiring these markers with onboard sensors, such as low-power commercial IMUs <cit.>, making the system non-dependant on external laboratories and/or manual data entries.






  §.§.§ Sepsis prediction with a consensus algorithm
 A consensus algorithm was applied to the output of the TCN over successive windows for real-time identification of sepsis onset, as depicted in Figure <ref>(d). When k successive windows are labeled as 1, the alarm is raised and the patient is labeled as sepsis positive. An analysis of possible values of k  and window stride is given in Section .







§ EXPERIMENTAL RESULTS






 §.§ Performance: Vital Sign Estimation Accuracy

The system performance in terms of correlation with the reference device and the variation of estimation error for the measured vital signs is shown in Figure <ref>. RR has an R^2= 0.82 with an error variation for RR estimation of -0.119 ± 0.7 breaths per minute (brpm). The error variation for HR is 0.829 ± 2.8 beats per minute (bpm) and a high correlation with R^2= 0.92. The BP measure for the systolic blood pressure shows a correlation with R^2= 0.67. the average error in BP estimation is -0.0849 mm of Hg and the variance of the error is ± 6.24 mm of Hg, thereby fulfilling the Association for the Advancement of Medical Instrumentation (AAMI) criteria where the tolerable error is less than 10mm of Hg <cit.>.







 §.§ On-Board Deep Learning Model Performance

Figure <ref> presents the model's specificity, sensitivity, and median time to sepsis predictions for various values of the aggregation parameter (k) and three different stride intervals of 10 minutes, 30 minutes, and 60 minutes. The solid line represents the mean, while the shaded area reflects the standard deviation over a 5-fold cross-validation performed on the test dataset.


An increased overlap between consecutive windows when combined with a shorter stride introduces redundancy which yields a low true positive rate as compared to longer window strides. On the other hand, longer windows may miss the causal dependencies between sepsis-informative features, resulting in reduced specificity and an elevated false positive rate. Increasing k makes the consensus algorithm more reliant on consecutive positively predicted windows, enhancing specificity. However, this increased reliance also necessitates more positive predictions to trigger an alarm, consequently elevating the false negative rate. The median time to sepsis was calculated by using stride of 30 minutes and an aggregation parameter of 8. This parameter selection resulted in a sensitivity of (0.56 ± 0.06) and specificity of (0.48 ± 0.047) and an estimate of median predicted time to sepsis of 8.2 hours (CI: [7 h, 9.4 h]).

The performance of the TCN is hampered by the confinement to the HiRID dataset, which lacks the diversity of a larger sample. However, this dataset was chosen due to its high granularity of vital sign measurements and being the only open-source dataset available.


In Table <ref> the results for the quantized model are reported. It can be seen that the performance difference between the floating point and the quantized model is below 10%, with a 4x memory usage improvement.




 §.§ On-board Implementation of vital sign algorithms

The onboard implementation on the ARM Cortex-M33 was clocked at 64 MHz.The HR and RR peak detection took 4.05 ms with a memory footprint of only 38 kB, including 2s of accelerometer data and algorithm parameters. The BP algorithm used the already identified peaks in the first task from one sensor (ACC1) and then executed the peak detection again on the second accelerometer (ACC2) to calculate the PTT, yielding the same metrics reported before. 1.4 ms were taken to calculate the average values of the HR and BP. The RR algorithm is executed every 30s and has the same metrics as the HR algorithm, but a lower filtering frequency to accommodate the slower-evolving respiratory signal.



 §.§ Power Profiling

 In Figure <ref> the power profiling for a standard workflow for i-CardiAx is reported. First, the node enables BLE advertisement packets, while simultaneously collecting data from the sensors. Once the FIFO buffer inside the sensors is full, every 2 seconds, the data is read by the MCU which is followed by vital sign extraction (digital signal processing (DSP) block as shown in Figure <ref>). The TCN inference is run after the TCN-input buffer of 4 hours is filled. Then for every time window stride, an inference is executed. The prediction of the TCN is saved for every inference and fed into the consensus algorithm, which when it encounters the conditions that satisfy the triggering of an alarm, transmits the alarm via BLE. 

In Table <ref> the current consumption and the latency of each state are reported, while in Table <ref> an overview of the active tasks is shown. The IDLE state of the system is defined as all subsystems in sleep mode with BLE advertising every 1s and the IMU collecting accelerometer data at 120Hz. Data is fetched from the sensors every 2 seconds with an average power consumption of 5.32 mW and takes 27 ms for the MCU to execute. The vital sign extraction is performed every 2s to extract the peaks from the accelerometer data. The total current consumption for this is 5.22 mW throughout 3.6 ms per accelerometer. 
The TCN inference is energy efficient and consumes 6.49 mW over 320 ms and the task is run every 30 minutes on a full NN-input buffer and according to the window stride. The BLE transmission of the inference accounts for 1 byte that is transferred in 0.5 ms at a power consumption of 1.39 mW. The operation of i-CardiAx is lightweight (approximately 100 kB of memory footprint) and always within mW-envelopes




 §.§ Battery Lifetime

Based on the power requirements of the sensors, onboard processing of vitals, BLE transmission, power supply quiescent draws, and other system management SoC tasks, a battery lifetime for different rates of on-board vital sign inference is estimated in Figure <ref> and summarised in Table <ref>. The estimated operational longevity is 337 hours with a neural network window stride of 2min and up to 455 with a neural network window stride of 60 minutes. For the case analyzed in this work (window stride of 30min), the estimated battery lifetime is 432h, around 18 days, with a battery of 100mAh. The simulation is done under the task period reported in Table <ref>


§ CONCLUSIONS

 This paper presented i-CardiAx, a long-lasting wearable system for continuous cardiovascular parameter monitoring using only two high-sensitive low-power accelerometers. The developed algorithms for HR, BP, and RR showed high accuracy in vital sign estimation when compared with reference devices and were compliant with international standards for medical instrumentation, such as AAMI for blood pressure monitoring. Additionally, the algorithms were fully embedded in a low-power ARM Cortex-M33  and transmitted via BLE. Due to the onboard processing and the low-power design, we demonstrated that  i-CardiAx can last for approximately two weeks with a single 100mAh battery when the sensor is always on and vitals of HR, RR, and BP are reported every 30s.  






ieeetr





 Paper Title: Augmenting Channel Simulator and Semi- Supervised Learning for Efficient Indoor Positioning.




















Augmenting Channel Simulator and Semi- Supervised Learning for Efficient Indoor Positioning




    Yupeng Li†, Xinyu Ning, Shijian Gao*, Yitong Liu, Zhi Sun$, Qixing Wang†, Jiangzhou Wang† 
 	
†China Mobile Research Institute, Beijing, China 

Beijing University of Posts and Telecommunications, Beijing, China

*The Hong Kong University of Science and Technology (Guangzhou), China

$Tsinghua University, Beijing, China 

		
  
  
    August, 2024
================================================================================================================================================================================================================================================================================================================================================



§ ABSTRACT 
This work aims to tackle the labor-intensive and resource-consuming task of indoor positioning by proposing an efficient approach. The proposed approach involves the introduction of a semi-supervised learning (SSL) with a biased teacher (SSLB) algorithm, which effectively utilizes both labeled and unlabeled channel data. To reduce measurement expenses, unlabeled data is generated using an updated channel simulator (UCHS), and then weighted by adaptive confidence values to simplify the tuning of hyperparameters. Simulation results demonstrate that the proposed strategy achieves superior performance while minimizing measurement overhead and training expense compared to existing benchmarks, offering a valuable and practical solution for indoor positioning.











Indoor positioning, semi-supervised learning, pseudo-label, deep learning, 6G




§ INTRODUCTION

The 3rd Generation Partnership Project (3GPP) has successfully conducted the fifth generation (5G) Advanced phase in its Release 18<cit.>. As one of the most significant potential advancements to increase automation and efficiency of the industrial domain, the indoor positioning has attracted widespread attention. However, the indoor factory scenario presents challenges, including a high volume of obstructions leading to numerous non-line-of-sight (NLOS) scenarios. These conditions often lead to significant performance degradation for the traditional positioning algorithms <cit.>. 






To address the issue of the limitations and difficulties of traditional positioning methods, a notable point is to integrate deep learning (DL) into indoor positioning systems, specifically fingerprint-based positioning. 

The basic idea of the fingerprint positioning is to create a mapping function between the channel measurements and the position coordinates, creating a main wireless radio environment among the transceivers <cit.>. Then, the user equipment (UE) can obtain its position by the measurements. The measurements of fingerprint positioning in the cellular systems are mainly channel impulse response (CIR), power delay profile (PDP), and so on <cit.>.



However, DL-based indoor positioning faces the challenge of insufficient data available for training the model <cit.>. Especially, in the flexible and dynamic communication systems, obtaining sufficient measurements and labeling them is a labor-intensive and resource-consuming task. 

















A powerful approach termed semi-supervised learning (SSL) was proposed, which involved using small number of labeled data combined with a large amount of unlabeled data to train the model <cit.>. 


Typical SSL <cit.> may lead to serious performance degradation due to errors in the pseudo-labels of the unlabeled data.

To address this issue, a smaller weighting coefficient was added to the unlabeled data during model training <cit.>.

The weighting coefficient is a hyperparameter, which requires meticulous manual tuning.

In <cit.>, the concept of confidence for the pseudo-label of the unlabeled data was proposed, which helps to select reliable unlabeled data in classification tasks. However, there is a lack of a method to define a label confidence in regression tasks like positioning. Furthermore, in an indoor scenario, obtaining the unlabeled data is also a challenge that requires extra measurement overhead.




To address the aforementioned challenges, a DL-based positioning algorithm with reduced overhead is proposed. To lower the measurement overhead, the unlabeled data is generated by an updated channel simulator according to the stochastic parameters of the labeled data. This approach reduces the measurement expenses of the unlabeled data. Additionally, to capture the features of both the labeled and unlabeled channel data, a semi-supervised learning with a biased teacher is proposed. The main contributions of this paper are summarized as follows:



  * An updated channel simulator is utilized to generate unlabeled data, thereby reducing the measurement expenses associated with acquiring such data. Initially, labeled data collected within an indoor factory environment is utilized, and the primary channel parameters are analyzed to extract the channel stochastic parameters of the measurement environment. Subsequently, the typical TR 38.901 <cit.> channel simulator is updated with these stochastic parameters, resulting in the creation of an updated channel simulator (UCHS). The UCHS is then employed to simulate the unlabeled channel data, which closely resembles the labeled data of the measurement environment.





  * To efficiently capture the mapping function between the CIR and the position coordinate while minimizing training expenses, a SSL with a biased teacher (SSLB) is introduced. The SSLB approach involves weighting the unlabeled data based on the confidence, which is derived from the distance bias predicted by the teacher model. This methodology assigns higher confidence to more reliable unlabeled data. Notably, this work represents the first instance of introducing confidence into regression tasks, such as indoor positioning, resulting in a significant reduction in training expenses.

  * To obtain the confidence of the unlabeled data, a novel SSL structure is designed, termed as SSLB. This structure incorporates an additional reference signal and a position bias, added separately to the input and the output of the traditional SSL model. As a result, SSLB can extract additional bias information from the labeled dataset, which can then be transformed into the confidence of the unlabeled data. 






























Notation: Vectors and matrices are denoted by boldface. ||·|| expresses the 2-norm of a vector. ||·||_F is the Frobenius norm of a matrix. For unlabeled data, we use a superscript ^*, while labeled data remains unmarked.



§ PROBLEM DESCRIPTION










 §.§ Adapting semi-supervised learning for indoor positioning


The dataset of traditional SSL consists of two parts: the labeled dataset and the unlabeled dataset. The input is the CIR and the output is the predicted position.

The labeled dataset is represented as 𝔇(H, l), where H is the labeled CIR and l is its corresponding position coordinate. The unlabeled dataset is represented as 𝔈(H^*), containing only the CIR.


The SSL model is divided into a teacher model Θ^t and a student model Θ^s. Firstly, the teacher model is trained with labeled data, and then pseudo-labels of unlabeled data are generated using the trained teacher model. Finally, the student model is trained with both the labeled data and the unlabeled data. In this paper, the loss function is defined as the Euclidean distance between the predicted position and the real position as shown in (<ref>). First, train the teacher model using this loss on the labeled dataset 𝔇(H, l).

    L = 1/N∑^N_i=1 ||l̂_i - l_i||.


Then, the label of dataset 𝔈(H^*) is generated with the trained teacher model

    l^* = f(H^*, Θ^t),

where l^* is the position coordinate of H^*. f(·) represents the mapping function between the channel measurements and position coordinates. In this way, the dataset 𝔈(H^*) is completed to 𝔈(H^*, l^*).

Finally, the student model is trained with the dataset of 𝔇(H, l) and 𝔈(H^*, l^*), applying weights to pseudo-labeled data. The loss function is defined as follows:

    L^w   =1/N+N^*(∑^N_i=1√(w||l̂_i-l_i||^2) +  ∑^N^*_j=1√(w^*||l̂^*_j-l^*_j||^2)),


where L^w is the loss of weighted semi-supervised algorithm, N and N^* are the numbers of labeled data and unlabeled data, and w and w^* are the corresponding weights of labeled data and unlabeled data. Detailed descriptions can be referred to <cit.><cit.>. it is worth mentioning that all models use the same DL model, illustrated in Fig. <ref>.








§ CHANNEL FEATURE AIDED DATA AUGMENTATION













 §.§ Wireless channel feature extracting method


In this section, some specific methods for extracting channel parameters will be described.



  §.§.§ Angle spread

The angle spread (AS) is used to characterize the dispersion features of the multipaths in the spatial domain. Initially, a two-dimensional Discrete Fourier transform (DFT) is performed over the channel matrix H(t) over the port domain, resulting in the angle domain channel matrix H_ang <cit.> as follows:

    H_ang = U^T H(t) V,

where U = [u_1^T; ...; u_N_t^T]^T is the transmit DFT matrix, and V = [v_1, ... ,v_N_r] is the receiver DFT matrix<cit.>.

Then, the receiving and transmitting AS are illustrated in the spatial domain. The angle spread value can be obtained as follows:

    σ_AS = √(∑^M_m=1θ_m^2P_m/∑^M_m=1P_m)
    ,

where P_m is the power of the mth multipath, θ_m is the angle of the mth multipath.



  §.§.§ Delay spread

The delay spread (DS) can reflect the richness of the channel multipaths. And the DS <cit.> can be obtained as follows:

    σ_DS = √(∑^M_m=1(τ_m - τ_mean)^2 P_m/∑^M_m=1P_m),

where τ_mean = ∑^M_m=1 (τ_mP_m)/∑^M_m=1 (P_m) is the mean delay.

Due to the space limit, only the above two unique channel statistical parameters are delineated here. Other channel statistics parameters can refer to <cit.>.



 §.§ Unlabeled data generation via UCHS


The TR 38.901 <cit.>, considered as the state-of-the-art 3GPP channel model, is crucial for generating channel simulation data and evaluating physical layer techniques. It illustrates the stochastic distribution characteristics of channel multipaths, serving as a benchmark in a general scenario. However, in a specific location, the channel parameters in TR 38.901 may not be precise. Regrettably, in an AI-enabled indoor factory, the training data needs to reflect the actual propagation environment as accurately as possible.

To address the aforementioned issue, we propose a data augmentation method based on the channel simulator, which can simulate the unlabeled channel data corresponding to the specific area. This approach significantly reduces the collection overhead of unlabeled data. 

The generation process of unlabeled data is shown as follows:


  * Step 1: the labeled channel data is collected by the UEs distributed in a specific indoor factory. Since labeled data is essential for training the teacher model, there is no extra overhead of data collection at this
stage.

  * Step 2: upload the labeled data from the UE to the BS.

  * Step 3: the primary statistical parameters of the channel data are extracted, as described in Section <ref>, effectively depicting the wireless propagation environment.

  * Step 4: these statistical parameters are used to update the parameters of
typical TR 38.901 channel model, resulting in the UCHS.

  * Step 5: the UCHS is used to simulate unlabeled channel data similar to the labeled data of the measurement environment.





§ SEMI-SUPERVISED PSEUDO-LABELING WITH A BIASED TEACHER







Compared to traditional SSLs, our proposed method assigns different loss weights to the pseudo-labels of the unlabeled data based on the confidence scores. By doing so, pseudo-labels with low confidence are assigned small weights during the training stage of the student model, thereby mitigating the impact of pseudo-labels with large offset on the performance of the student model.




The proposed method pipeline, as illustrated in Fig. <ref>, consists of three primary stages: the bias teacher training stage, the confidence-based pseudo-labeling stage, and the weighted loss student training stage:





  * Stage 1: an additional reference signal and a position bias are separately added to the input and output of the traditional SSL model. This addition helps extract additional information from the labeled dataset. Specifically, the position bias represents the distance between the input signal and its nearest signal, referred to as the reference signal. This allows the teacher model to not only determine the mapping function between the input signal and the position coordinates but also to assess the similarity between input signals using the distance bias.


  * Stage 2: the trained teacher model is utilized to generate pseudo-labels and the position bias of the unlabeled data. The position bias can then be used to define the confidence scores of the unlabeled data through a kernel density estimation (KDE) <cit.> function.

  * Stage 3: a weighted loss strategy is employed, wherein different loss weights are assigned to different pseudo-labels in the loss function to train the student model. This approach aims to mitigate the impact of pseudo-labels with larger errors on the performance of the student model.





 §.§ The biased teacher training



In the context of object detection, models not only output the bounding boxes of the detected objects but also provide confidence scores for these detections. Inspired by this, a biased teacher model with two prediction heads is proposed. One prediction head, called the 'predict head”, is responsible for predicting the position coordinates l̂ corresponding to the input CIR. The other prediction head, named as the ”biased head” is designed to predict the bias Δl̂ of the reference signal relative to the input CIR position.






The first step involves using the k-nearest neighbors (KNN)<cit.> algorithm to search the reference signal H^ref_i in labeled dataset 𝔇(H, l) that is closest to the input CIR H_i, thereby obtaining the reference signal data pair (H^ref_i, l^ref_i), i.e.,


    H^ref_i = H_j, min_j ∈ (1, 2, ..., N)  ||H_i-H_j||_F, i ≠ j ,



where || · ||_F is Frobenius norm, and i ≠ j excludes itself as the nearest neighbor. 

The position bias is defined as:

    Δl_i ≜l^ref_i - l_i.

The total loss function of the teacher model is defined as:

    L = L^p + L^b,




where L^p = 1/N∑_i = 1^N ||l̂_̂î-l_i|| and L^b = 1/N∑_i = 1^N ||Δl̂_̂î-Δl_i|| are the position prediction loss and the bias prediction loss, respectively.



















 §.§ Confidence-based pseudo-labeling



This process involves using the teacher model to generate estimated positions l^* and position biases Δl^* for the unlabeled data, which can then be used for subsequent steps in the SSL framework.




For the unlabeled data, the confidence is determined by calculating the distribution of the position bias. This involves computing the position bias as d=||Δl^*||. Subsequently, the distribution can be fitted using KDE as follows:

    f̂_h(d)=1/Nh∑^N_i=1K( d-d_i/h)
    ,


where f̂_h(d) is the KDE at point d. N is the number of samples, h is the bandwidth, d_i represents each data point in the sample, K(·) represents the kernel function, and in this work the Gaussian kernel is used. Fig. <ref>(a) illustrates the distribution of the unlabeled data in our experiment.


The confidence is defined as:

    w^*_i =f̂_h(d_i).


To ensure a fair comparison with the SSL, the obtained weights w^*_i need to divide the mean value of all the w^* to guarantee its mean value to be 1. This normalization process is important for comparison purposes. The distribution of w^* is shown in Fig. <ref>(b).
















 §.§ Weighted loss for student training


The student model Θ^s is trained with the mixture dataset, which is constructed by the unlabeled dataset 𝔈(H^*,l^*,w^*) 
combined with the labeled dataset 𝔇(H, l). 



For the student model, only the predicted coordinates l̂ are used to compute the loss, which is defined as

    L^w   =1/N+N^*( ∑^N_i=1 ||l̂_̂î-l_i|| + ∑^N^*_j=1√(w^*_i||l̂^*_j-l^*_j||^2)),


where w^*_i is the weight of the pseudo-label, and the weight of labeled data is set as 1.








§ EXPERIMENTS AND RESULTS



 §.§ Scenario deployment

The dense clutter high BS (DH) sub-scenario is a priority for evaluation in 3GPP for AI-enabled indoor factory (InF) <cit.>. In our considered scenario, there are 18 BSs uniformly distributed within the 60m*120m space with a spacing of 20m and height of 8m. The central frequency is 3.5 GHz and the bandwidth is 100 MHz in our experiment. The clutter density is 40% with the clutter height of 2m. The UEs are randomly generated within the space with a height of 1.5m.












 §.§ Experimental setup


In the experiment, a total of 17200 labeled channel samples are collected in the measurement scenario, where 10000 samples are randomly selected to construct the training dataset and the residual 7200 samples are allocated to the testing dataset. Besides, the unlabeled channel data are generated using UCHS. 


All the models are implemented using PyTorch and the Adam optimizer. The hyperparameters are learning rate 5e-2, an input batch size of 256, 150 epochs, and a cosine annealing rate scheduler. The criteria for InF positioning is the positioning accuracy at a cumulative distribution function (CDF) of 90% for users <cit.>.
























 §.§ Data structure

In the cellular positioning problem, UEs measure the time domain CIR H(t) from multiple BSs. For each position, the measured H(t) by each UE is in the form of N_BS*N_port*N_delay*2 dimensions, where N_BS, N_port, and N_delay represent the number of BS, the number of BS antenna respectively, and the number of the time delay, and 2 is from the complex-domain to real and imaginary parts. In most industrial scenario, the effective delay is mainly distributed in the first 64 delay taps. Under the current setup, this results in an extremely high-dimensional tensor for each position in the form of 18*4*64*2.




 §.§ Experimental results



  §.§.§ Main results


In this section, we carry out simulations to evaluate the performance of our proposed  SSL with position bias, which we call as “SSLB” scheme hereafter. We compare the proposed alrotihm with the following three benchmark schemes: 


  * a) SL: traditional supervised learning, whose reference signal H^ref is represented by the input signal H, with all the predicted position biases being zero. 

  * b) SLR: SL learning with reference signals. Its input signal contains the current CIR H and the reference signal H^ref, and the output is l̂ and Δl̂. 

  * c) SSLR: SSL pseudo-label learning without a biased teacher. 
The same DL model and training hyperparameters are used for the four schemes mentioned above. Our method aims to provide a convenient guideline for adaptively obtaining the weight of the unlabeled data using confidence. This is compatible with consistency-based pseudo-labeling SSL <cit.>, and combining them may lead to better results, but it is beyond the scope of this paper.

 














Fig. <ref> illustrates the positioning accuracy for the four schemes with different labeled samples. It is evident that the SSLB achieves the highest positioning precision. For instance, with less than 6300 samples, SSLB can achieve the precision of 0.897 m, while the SL requires 9900 samples to achieve a precision of 0.969 m. Therefore, with the help of UCHS, our proposed SSLB can reduce the data overhead by almost 40% compared to SL. However, it is worth noting that the performance of the SSL pseudo-label method degrades at 2700 samples, due to the introduction of enormous inaccurate pseudo-labels by the teacher model, which deteriorates the performance of the student model. In contrast, SSLB achieves relatively stable performance, as it can automatically assign small weights to the unlabeled data with large offset.








  §.§.§ Ablation study





  
The impact of weight on the test loss







In this section, we study the effects of weight magnitude α on the SSLR and our proposed SSLB. In the student model training stage, for both the SSLR and the SSLB, the unlabeled weights are scaled by the formula w^*'_i = α w^*_i. Fig. <ref> compares the performance of SSLR and SSLB under different scaling coefficients. As the scaling coefficient decreases, the performance gap between the two schemes reduces, whereas the SSLB outperforms the SSLR in all cases. Table <ref> demonstrates the position accuracy at CDF of 90% for users. With the decrease of scaling coefficients, the positioning precision of SSLR increases at first, and then decreases. However, the performance of KDE confidence based SSLB keeps relatively stable. Thus, our proposed scheme does not manually select the weights hyperparameters α of the unlabeled data and dynamically assigns different weights to the pseudo-labels. Therefore, the training expense is dramatically reduced.




  
Impact of confidence calibration

To verify the effectiveness of the KDE confidence based method for transforming the position bias to confidence, we compare it with a linear transformation method. The linear transformation method directly considers position bias as confidence, assuming that small bias corresponds to high confidence levels. Firstly, the Euclidean distance of the position bias is computed. Then,  this data is normalized within the range of 0 to 1, reversed by subtracting from 1, and finally divided by the mean. The comparison results between the linear method and KDE are presented in Table <ref>.







§ CONCLUSION

This paper has investigated the issue of indoor positioning with reduced expenses. On one hand, the the UCHS was used to generate unlabeled data and reduce measurement overheads. On the other hand, we proposed the SSLB, which adaptively weighted the unlabeled data based on confidence to differentiate the quality of the data. This approach effectively reduced the expenses associated with tuning the hyperparameters of the weights. Simulation results demonstrated that our proposed scheme exhibited superior performance, reducing the data measurement overhead by almost 40% compared to benchmark schemes. Furthermore, ablation study confirmed that our proposed scheme can automatically assign different weights to unlabeled data, significantly reducing training expenses.






























































































00

R18 3GPP TR 38.843, v1.1.0, ”Study on Artificial Intelligence (AI)/Machine Learning (ML) for NR air interface,” 2023.

geome1 L. Chen, X. Zhou, F. Chen, L.-L. Yang, and R. Chen, ”Carrier phase ranging for indoor positioning with 5G NR signals,” IEEE Internet of Things Journal, vol. 9, no. 13, pp. 10908-10919, Jul. 2022.





geome4 Y. Li, Y. Zhuang, X. Hu, Z. Gao, J. Hu, et al., ”Toward location-enabled iot (le-iot): Iot positioning techniques, error sources, and error mitigation,” IEEE Internet of Things Journal, vol. 8, no. 6, pp. 4035–4062, 2020.


fingerprint2 K. S. Wu, J. Xiao, Y. W. Yi, M. Gao, and L. M. Ni, ”FILA: Fine-grained indoor localization,” in Proc. IEEE INFOCOM, 2012, pp. 2210-2218.

TSG_RAN 3GPP TSG-RAN Metting #96, ”Revised SID: Study on Artificial Intelligence (AI)/Machine Learning (ML) for NR air interface,” 2022.

insufficient K. M. Chen and R. Y. Chang, ”Semi-Supervised Learning with GANs for Device-Free Fingerprinting Indoor Localization,” IEEE GLOBECOM, Taipei, Taiwan, pp. 1-6, 2020.

ssl Y. Ruan, L. Chen, X. Zhou, L. Zhaoliang, L. Xiaoyan, et al., ”iPos-5G: Indoor positioning via commercial 5G NR CSI,” IEEE Internet of Things Journal, vol. 10, no. 10, pp. 8718-8733, 2022.
ssl1 X. Yang, Z. Song, I. King and Z. Xu, ”A survey on deep semi-supervised learning,” IEEE Transactions on Knowledge and Data Engineering, vol. 35, no. 9, pp. 8934-8954, 2022.

ssl2 X. Lin, ”An Overview of 5G Advanced Evolution in 3GPP Release 18,” IEEE Communications Standards Magazine, vol. 6, no. 3, pp. 77-83, September 2022.
pseudo
D. H. Lee, ”Pseudo-label: The simple and efficient semi-supervised learning method for deep neural networks,” Workshop on Challenges in representation learning, vol. 3, no. 2, 2013.

semi_and_gan W. Njima, A. Bazzi and M. Chaffi, ”DNN-based indoor localization under limited dataset using GANs and semi-supervised learning,” IEEE Access, vol. 10, pp. 69896-69909, 2022.

fixmatch
K. Sohn, D. Berthelot, N. Carlini, Z. Zhang, et al., ”Fixmatch: Simplifying semi-supervised learning with consistency and confidence,” Advances in neural information processing systems, vol. 33, pp. 596-608, 2020.

901
3GPP TR 38.901, v17.0.0, ”Study on channel model for frequencies from 0.5 to 100 GH,” 2022.

ruisi
M. Yang, A. Bo, R. He, C. Liang, L. Xue, et al.,”A cluster-based three-dimensional channel model for vehicle-to-vehicle communications,” IEEE Transactions on Vehicular Technology, vol. 68, no. 6, pp. 5208-5220, June 2019.

extrpara
H. Miao, J. Zhang, P. Tang, L. Tian, ”Sub-6 GHz to mmWave for 5G-advanced and beyond: Channel measurements, characteristics and impact on system performance,” IEEE Journal on Selected Areas in Communications, vol. 41, no. 6, pp. 1945-1960, June 2023.

kde
R. A. Davis, K. S. Lii, and D. N. Politis, ”Remarks on some nonparametric estimates of a density function,” Annals of Mathematical Statistics, pp. 832-837, 2011.

knn
F. Gu, L. Shi, H. Guo, and J. Shang, ”Deep fingerprint metric learning for KNN-based indoor localization.” IEEE GLOBECOM, Kuala Lumpur, Malaysia, pp. 182-188, 2023.

mean_teacher A. Tarvainen, and H. Valpola, ”Mean teachers are better role models: Weight-averaged consistency targets improve semi-supervised deep learning results,” Neural Information Processing Systems, 2017.

flexmatch B. Zhang, Y. Wang, W. Hou, H. Wu, J. Wang, et al., ”FlexMatch: Boosting semi-supervised learning with curriculum pseudo labeling.” Neural Information Processing Systems, 2021.

usb Y. Wang, H. Chen, Y. Fan, W. Sun, R. Tao, et al. ”Usb: A unified semi-supervised learning benchmark for classification,” Advances in Neural Information Processing Systems, vol. 35, pp. 3938-3961, 2022.















