Paper Title: Post-Quantum Cryptography (PQC) Network Instrument: Measuring PQC Adoption Rates and Identifying Migration Pathways.










Post-Quantum Cryptography (PQC) Network Instrument:
 Measuring PQC Adoption Rates and 
 Identifying Migration Pathways
    Jakub Sowa 0009-0009-0115-4822^1, Bach Hoang 0000-0003-0827-2564^1, Advaith Yeluru^1, Steven Qie^2,  
 Anita Nikolich^2, Ravishankar Iyer^2, Phuong Cao*Corresponding author: Phuong Cao; Data: https://pmcao.github.io/pqchttps://pmcao.github.io/pqc 0000-0001-6028-0583^1,2,*




^1National Center for Supercomputing Applications,
^2University of Illinois at Urbana-Champaign


    August 5, 2024
==========================================================================================================================================================================================================================================================================================================================================================================================



§ ABSTRACT 
The problem of adopting quantum-resistant cryptographic network protocols or post-quantum cryptography (PQC) is critically important to democratizing quantum computing. The problem is urgent because practical quantum computers will break classical encryption in the next few decades. Past encrypted data has already been collected and can be decrypted in the near future. The main challenges of adopting post-quantum cryptography lie in algorithmic complexity and hardware/software/network implementation. The grand question of how existing cyberinfrastructure will support post-quantum cryptography remains unanswered.

This paper describes: i) the design of a novel Post-Quantum Cryptography (PQC) network instrument placed at the National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urbana-Champaign and a part of the FABRIC testbed; ii) the latest results on PQC adoption rate across a wide spectrum of network protocols (Secure Shell – SSH, Transport Layer Security – TLS, etc.); iii) the current state of PQC implementation in key scientific applications (e.g., OpenSSH or SciTokens); iv) the challenges of being quantum-resistant; and v) discussion of potential novel attacks. 

This is the first large-scale measurement of PQC adoption at national-scale supercomputing centers and FABRIC testbeds. Our results show that only OpenSSH and Google Chrome have successfully implemented PQC and achieved an initial adoption rate of 0.029% (6,044 out of 20,556,816) for OpenSSH connections at NCSA coming from major Internet Service Providers or Autonomous Systems (ASes) such as OARNET, GTT, Google Fiber Webpass (U.S.) and Uppsala Lans Landsting (Sweden), with an overall increasing adoption rate year-over-year for 2023-2024. Our analyses identify pathways to migrate current applications to be quantum-resistant.





§ INTRODUCTION


The problem of adopting quantum-resistant cryptographic network protocols or post-quantum cryptography (PQC) is critically important to democratizing quantum computing. The problem is urgent because practical quantum computers <cit.>, will break classical encryption in the next few decades. Major applications such as financial services, high-performance computing (HPC) or supercomputing, and health analytics must be migrated to be quantum-resistant. The main challenges of adopting post-quantum cryptography lie in both algorithmic complexity and hardware/software/network implementation. The grand question of how existing cyberinfrastructure will support post-quantum cryptography remains unanswered.

This paper describes: i) the design of a Post-Quantum Cryptography (PQC) network instrument placed in a national-scale supercomputing center, ii) the latest results on PQC adoption rate across a wide spectrum of popular network protocols (Secure Shell – SSH, Transport Layer Security – TLS, etc.), iii) the current state of PQC implementation in key scientific applications (e.g., OpenSSH, SciTokens), iv) the challenges of being quantum-resistant and v) discussion of potential novel attacks. Our result is critically important regarding adopting National Institute of Standards and Technology (NIST)'s draft algorithms, such as CRYSTALS-Kyber for encryption, FALCON and SPINCS+ for digital signature, and KEMTALS for key exchange <cit.> and protocol-specific PQC adaptation such as Hybrid Streamlined NTRU (Ring-Based Public Key Cryptosystem) Prime sntrup761 and x25519 with SHA-512 (sntrup761x25519-sha512) in OpenSSH <cit.>.




Motivation.
The cryptographic algorithms we use to secure and verify all this data today are far more complex than in centuries past, yet still heavily reliant on the "hardness" of certain mathematical problems. These "hard" problems, such as integer prime factorization and the discrete logarithm problem, form the basis of modern cryptosystems such as RSA and Ecliptic Curve Cryptography (ECC), respectively. As long as no computer can efficiently solve these problems, which are assumed to be hard, the security of the cryptosystem should be maintained. For example, it would take an exascale supercomputer such as Frontier (≈ 1.20 exaflops <cit.>) longer than the age of the universe, on average, for an attack to reveal the 256-bit private key in the curve25519 ECC cryptographic scheme.

With new developments in quantum computers and quantum algorithms on the rise, many of these problems previously assumed to be hard appear much easier. The 1994 Shor's algorithm for solving both discrete logs and prime factorization efficiently on quantum computers shows the need for this Post-Quantum Cryptography (PQC), which is made even more urgent when you consider an adversary who is storing classically encrypted traffic now to be decrypted with a quantum computer in the near future when they are powerful and reliable enough. The grand question of how existing cyberinfrastructure will support post-quantum cryptography remains unanswered and will be addressed in this paper.




Key contributions:

    
  * A novel network instrument to measure PQC adoption in real-time and provides historical trends of the adoption rates across seven layers of computer network protocols. 
    
    
    
    
  * The first statistical study of PQC's readiness across seven layers of computer network and HPC applications with a unique vantage point: a petascale supercomputing center with nation-scale visibility through the FABRIC TeraCore network.
    
    
  * Release of an open sample dataset containing metadata of cryptographic suites across major network protocols (SSH, TLS, RDP, etc.  <cit.>), available online at https://pmcao.github.io/pqc
    
    
  * Discussion of potential novel PQC attacks and characterization of major challenges blocking the adoption of PQC in HPC applications. 
    
    
    
    
    


Major results.
Our results highlight the difficulty of ensuring all systems are updated and using the most secure connection options available, which is important to reach widespread PQC adoptions as follows:



  * OpenSSH and Google Chrome have successfully implemented PQC and achieved an initial adoption rate of 0.029% (6,044 out of 20,556,816) for OpenSSH connections at NCSA.


  * The adoption rate for OpenSSH is increasing year-over-year for 2023-2024. 


  * Top U.S. large networks (Autonomous Systems – ASes) such as OARNET, GTT, Google Fiber Webpass, and Comcast; and Uppsala Lans Landsting (Sweden) are hosting clients that have already adopted PQC in OpenSSH. 


  * Over 83% of Server-side SSH protocols were from 2019 and earlier, 3 years before sntrup761x25519 was even introduced. Only about 65% of connections used TLS version 1.3, the most recent and most secure option for this use case The rest used TLS version 1.2, which still supports many of the weak cipher suites we found. 



Our analyses identify pathways to migrate current applications to be quantum-resistant. Through experimental deployment at different network vantage points, our approach may help identify empirical novel attacks against PQC implementations and give feedback to the National Institute of Standards and Technology (NIST) as well as developers to improve their drafts and successfully migrate to PQC <cit.>.



[width=, colback=white!95!black, boxrule=0.5pt, left=2pt,right=2pt,top=1pt,bottom=1pt]
remark
Remark remark:

Perspective w.r.t the state of the art.
To the best of our knowledge, this is the first network instrument that exists to monitor PQC adoption publicly. There are a few related work <cit.>, e.g., Cloudflare released a snapshot of TLS adoption in Feb 2024 through their content delivery network <cit.>. On the other hand, our network instrument is the first of such to monitor the adoption of both OpenSSH, TLS, and other protocols at large-scale, high-speed (gigabit to terabit) interconnected networks with a wide spectrum of not only traditional cloud but also scientific workloads. 








§ APPROACH OVERVIEW

This section describes the network topology, architecture of the network optical tap, placement of our network instrument, data collected, and statistical methods used.









Figure <ref> describes our network instrument. Using FABRIC testbed (A), we deployed our instrument at the NCSA site (B) to ingest Zeek connection metadata and parse session and application layer cipher suite information (C, D, E). An example of PQC key exchange and statistics that we parsed is shown for Secure Shell (SSH) protocol (F, G). Our instrument is embedded in a nation-scale network for monitoring a wide spectrum of scientific workloads as follows.

FABRIC testbed. FABRIC <cit.> is a nation-scale research infrastructure consisting of a TeraCore network between universities, national labs, and supercomputing facilities such as NSF Cloud testbeds CloudLab, NCSA, and Chameleon (Figure <ref>A). FABRIC powers exploratory networking research at scale in a variety of applications, including cybersecurity, distributed computing, high-speed storage, artificial intelligence, and HPC workloads. The unique approach to FABRIC is its Application Programming Interface (API), allowing experiments to programmatically compose extensible, high-speed interconnected optical link networks, compute, and storage. 

NCSA. The National Center for Supercomputing Applications (NCSA) offers cutting-edge cyberinfrastructure through the National Petascale Computing Facility (NPCF) that houses major NCSA infrastructure, including the Blue Waters supercomputer, to support diverse research needs. Major computing resources include: 1) Delta – NSF-funded GPU system ideal for GPU-accelerated applications (A100) and gateway-based workflows with nearline storage, Infiniband interconnect; 2) HAL Cluster supporting deep learning system with IBM POWER9 CPUs; and testbeds for ScienceDMZ, honeypot experiments and FPGA boards. All these resources are interconnected with long-term tape archives (Petabytes) and high-speed external connectivity (+400Gbps) to major research networks such as the Energy Sciences Network (ESnet) (Figure <ref>B).



Zeek network security monitor.
Zeek (formerly Bro <cit.>) is a flexible, open-source platform with decades of development history in network security monitoring. Defenders use Zeek's network analyzers to perform deep packet inspections and provide a broad view of many network protocols. Zeek enables high-performance, high-level, stateful semantic analysis at the application layer and is used operationally at various large sites such as Berkeley Lab and NCSA. The collection and analysis of these data provide a comprehensive view of PQC adoption, as described below.

Ethical considerations.
Our approach works on connection metadata produced by Zeek parsers. We did not use man-in-the-middle or traditional SSL termination equipment. Thus, we only see the encrypted data and not the original content. However, we do have access to the timing information of connection establishment to derive the trend and seasonality of the PQC adoption rate. The connection metadata generally includes standard information such as IP address, certificate, and negotiation of cipher suites. We do not see Personally Identifiable Information (PII).

 

[width=, colback=white!95!black, boxrule=0.5pt, left=2pt,right=2pt,top=1pt,bottom=1pt]
remark
Remark remark:

Our Zeek-integrated network instrument to measure PQC adoption has three main components: 

    
  * Parallel log parser to ingest large-volume of Zeek logs (metadata) from a +400Gbps network.
    
  * Historical analyses scripts that perform regression, trend, and statistics on the parsed metadata of cipher suites.
    
  * Real-time snapshot and comparison of our results vs. others (if any) to detect novel attacks and give feedback to the National Institute of Standards and Technology (NIST).
















§ DATASETS


This section presents the dataset that our network instrument has collected, which is built upon our prior work on the security testbed at NCSA <cit.>. We have collected a heterogeneous dataset containing metadata of cryptographic network protocols powering high-speed scientific workloads at NCSA between 2023-2024. In total, approximately 13TB of metadata has been generated and collected using the Zeek network observability framework from a 400Gbps network border link on the NCSA network and partner facilities, which supports a wide range of scientific applications <cit.> such as Globus, SciTokens, and Kubernetes. 

A summary of our dataset is specified in Table <ref>. Our dataset contains network connections in encrypted application-layer protocols (above layer 3 TCP/IP), such as Secure Shell (SSH), Remote Desktop Protocol (RDP), Hypertext Transfer Protocol Secure (HTTPS), etc. With metadata including network connections from various sources and destinations, we recorded a wide range of cryptographic algorithms (e.g., ECDH, RSA, SHA-256, etc.) and elliptic curves (e.g., x25519, secp256r1, etc.) that are used for encryptions tasks such as key exchange or key encapsulation mechanism.  Since this dataset is collected 24/7 for more than 16 months (January 2023 - present), it provides the orthogonal view of PQC adoption in scientific network traffic. 

[width=, colback=white!95!black, boxrule=0.5pt, left=2pt,right=2pt,top=1pt,bottom=1pt]
remark
Remark remark:

Our dataset is unique in its placement at a strategic network vantage point to observe metadata of a wide spectrum of applications across different protocols, validating our PQC measurement for both historical and real-time adoption rates.


























 



§ METHOD & RESULTS
 PQC NETWORK INSTRUMENT ON OPENSSH





This section presents both: i) longitudinal studies of PQC adoption on the Secure Shell (SSH) protocol over 16 months (Jan 2023 - Apr 2024) and ii) a detailed analysis of cipher suites used in OpenSSH on a sample working day. 

The first and most interesting protocol we studied, owing to its wide usage across most operating systems, was the Secure Shell (SSH) protocol which is well understood through our prior studies <cit.>. SSH metadata included information on four types of cryptographic algorithms used to secure the connections, the data for each is shown in Table 1. The most immediately apparent detail in the data is that the vast majority of the connections seem to be using some version of OpenSSH. Additionally, there are a few deprecated and insecure cryptographic algorithms in use, which would be a cause for more concern if not for their low usage numbers.

The data, stored as logs using the Zeek network analysis framework, is focused solely on connection metadata. The data was parsed and visualized in Python using its library, Matplotlib, and are shown in Figure 
<ref>,
<ref>,
<ref>. We first provide a background on PQC implementation in the latest version of OpenSSH (version ≥ 9.0), then describe our results based on SSH data.



 §.§ Background on PQC adoption in OpenSSH

This section presents details regarding the implementation
of PQ authentication into the OpenSSH library. The first version that OpenSSH supports PQC is 9.0 <cit.>. 
This developing standard PQC protocol has three main components:

    
  * sntrup761 is a key-encapsulation mechanism (KEM) in a family of ring-based public key cryptosystems called Streamlined NTRU Primes  <cit.> <cit.>, in which 761 is a prime number serving as a parameter. Cryptosystems in this family are parameterized by 3 positive integers (p, q, w), in which p, q are prime numbers, x^p - x - 1 is irreducible in the polynomial ring ℤ/q[x]. The parameters of sntrup761  KEM are a triplet of numbers p = 761, q = 4591, w = 286 <cit.>.
    
    
  * x25519 is the widely implemented Elliptic-Curve-Diffie-Hellman (ECDH) key exchange protocol, using Curve25519 as a underlying curve. Curve25519 <cit.> is an Montgomery elliptic curve in the form of y^2 = x^3 +  Ax^2 + x over the field ℤ/p, in which p = 2^255 - 19 is a prime number, and A = 486662 is an integer that A^2 - 4 is not a square modulo p.
    
    
  * SHA-512 is a secure cryptographic hash algorithm to ensure data integrity <cit.>. The input for SHA-512 is a message with a size up to 2^128 - 1 bits and the output is a word with a consistent length of 512 bits. 


[width=, colback=white!95!black, boxrule=0.5pt, left=2pt,right=2pt,top=1pt,bottom=1pt]
remark
Remark remark:

The sntrup761 KEM is a lattice-based post-quantum algorithm. Lattice-based algorithms are relatively new in the cryptography world and are thus not as strongly tested against even pre-quantum adversaries as some more traditional schemes. Many PQC implementations such as this one thus combine a more established cryptosystem with the intention of ensuring immediate security if just one of the involved algorithms are proven weak <cit.>.



To visualize this PQC protocol, we show key exchange with a client and server in Figure <ref>.



    
  * Ephemeral Key Generation:  Client and server generate temporary (ephemeral) key pairs using both sntrup761 and x25519 algorithms.

    
  * Public Key Exchange: Client and server exchange their public keys for both sntrup761 and x25519.

    
  * Shared Secret Calculation: Each side uses its private key and the other's public key to calculate a shared secret for both the sntrup761 and x25519 algorithms.

    
  * Combining and Hashing: The two shared secrets are combined and then hashed using the SHA-512 algorithm.

    
  * Final Key: The result of this process is the final, strong key used to encrypt and decrypt the SSH communication.


As SSH is widely adopted and frequently updated against vulnerabilities, it is more likely that SSH clients and servers will be the first to adopt PQC schemes. The described protocol is secure because it combines: 1) a hybrid approach, using both a post-quantum resistant algorithm (sntrup761) and a traditional algorithm with strong security (x25519) to provide fault tolerance; and 2) SHA-512, a hashing function securing the final key such that no information about the individual shared secrets can be easily derived.










 §.§ Results on PQC protocols adoption in SSH

First, we compare the adoption rate of SSH with the publicly available adoption rate from Cloudflare. Figure <ref> shows cross-protocol and cross-site comparison of adoption rate between SSH protocol at NCSA (our analysis) and TLS 1.3 protocol at Cloudflare  <cit.>. NCSA records 0.029% (6044 out of 20,556,816) monthly adoption rate for SSH, while Cloudflare recorded ≈ 1.78 percent adoption rate for TLS 1.3; more than 99% adoption came from Chrome <cit.>.

Figure <ref> shows a histogram of autonomous systems adopting PQC in SSH showing that the top 5 ASes (OARNE, GTT, Google Fiber, Comcast, Advanced Communications Technology, etc.)  from U.S. and Uppsala Lans Landsting (Sweden) accounted for the majority of PQC in the head of the distribution. A long list of ASes is shown in the long tail.

Figure <ref> shows the increasing adoption rate of PQC in SSH key exchange (sntrup761x25519-sha512@openssh.com), starting from January 2023 with only 37 exchanges, up to 1,585 exchanges in April 2024.






More interestingly, it is clear that there are no PQ algorithms in use except for one, the key exchange algorithm sntrup761x25519 using the SHA512 hash. This algorithm is an implementation of the post-quantum NTRU Prime cryptographic scheme, specifically a hybrid Streamlined NTRU Prime paired with the x25519 ECDH key exchange method to preserve security against classical adversaries in case a vulnerability in NTRU is found. It was introduced in the OpenSSH version 7.9 and added to the list of defaults in version 9.0 in 2022. It was developed by OpenSSH before the third round selections by NIST took place later that year, which actually ended up removing NTRU after the second round in the process. Unfortunately, we could only identify an insignificant amount of connections using this algorithm for key exchange, 0.08% for a sample day. The overall average adoption rate is  0.029% (6044 out of 20,556,816) for the period in which we compare SSH adoption rate with Cloudflare's TLS adoption rate (Figure <ref>).

[width=, colback=white!95!black, boxrule=0.5pt, left=2pt,right=2pt,top=1pt,bottom=1pt]
remark
Remark remark:

SSH is among the first protocols that implement PQC and actually have clients adopt their PQC protocol widely, as shown in our results.




 §.§ Results on classical protocols in SSH

Here, we provide further detailed traffic analysis for a regular working day in June 2023 for the OpenSSH protocol at NCSA. Network metadata was collected from a 400 Gbps network border link on the NCSA network. 

Importantly, we see the presence of various deprecated algorithms used by some connections as shown in Table  <ref>. This poses a security risk outside the scope of PQC and warrants another look at the versions of SSH protocols in use. 

Through further inspection of metadata logs, we determined that over 83% of Server-side SSH protocols were from 2019 and earlier, three years before sntrup761x25519 was even introduced. This highlights a common problem in the technology space—the difficulty of ensuring all systems are updated and using the most secure connection options available. This is important if we are to ever reach full PQC adoption.






§ METHOD AND RESULTS 
 PQC NETWORK INSTRUMENT ON OTHER PROTOCOLS

This section describes our analysis extended to other major network protocols such as Remote Desktop Protocol (RDP), Domain Name System (DNS), and Transport Layer Security (TLS). Note that this analysis is focused on a sample working day to illustrate the adoption. The objective is to measure the readiness level of applications using these protocols with regard to Post-Quantum Cryptography.



 §.§ RDP

The next network protocol we looked at was the Remote Desktop Protocol (RDP) for remote desktop visualization of Windows machines. Although the protocol today has no simple solution for using PQC, it is still worth analyzing the cryptography available given the protocol's prevalence in the IT space and its history as a tool for attackers <cit.>. RDP plays a crucial role for many organizations as a way to manage servers, troubleshooting, and remote work. With the rise in popularity of remote access solutions, ensuring RDP meets future quantum-safe standards is necessary to not only protect the integrity of data transmitted via RDP but also ensure that RDP sessions remain secure in the face of quantum computing advancements.  

Even though for RDP, there was a minimal data set of 26 connections, we were able to identify an interesting issue. Generally, RDP can be configured to use two types of cryptographic security, Enhanced Encryption and Network-Layer Authentication (NLA). Enhanced Encryption for RDP offers encryption over TLS of everything after the Connection Initiation stage and also allows authentication of the server from the client side. NLA requires the client to be authenticated before establishing a session with the server. However, out of all 26 connections, only 2 used the HYBRID-EX security protocol setting, which uses both Enhanced Encryption and NLA. As with SSH, this calls for more careful management of individual machines and making sure they are all configured to use both Enhanced Encryption and NLA for optimal security.

A final thing of note regarding RDP is that on connections where both server and client are using Windows 11 machines, it can be configured to run over TLS 1.3 to add additional levels of security.







 §.§ DNS


We then briefly examined the Domain Name System (DNS) protocol used for deriving IP addresses from domain names. Historically with DNS, packets are not encrypted - this is also true for the NCSA network. However, there are various ways of encrypting DNS traffic, namely through the HTTPS protocol, or over a TLS connection.

HTTPS can be used with DNS, called DoH, to encrypt DNS queries with HTTPS over port 443. This type of DNS can be enabled on some browsers such as Firefox and Chrome. As a side effect, DNS-over-HTTPS makes distinguishing it from general web traffic in logs difficult. 

Another form of secure DNS is DNS-over-TLS (DoT). DoT creates encrypted TLS channels for DNS queries with a DNS-over-TLS domain name resolver instead of the typical unencrypted DNS resolver.

With both these methods of securing DNS and many others not discussed here, it should be noted that any security properties of DNS are transferred from other network protocols, hence creating an opportune security dependence chain discussed further in later sections.










 §.§ TLS


The final network protocol investigated in our study was the Transport Layer Security (TLS) protocol. As with RDP and DNS, there has not been tremendous work finished in the community regarding PQC implementation in this protocol. Some companies and organizations have begun developing specifically tailored PQC implementations for their own use, but little has been developed for the common internet user. Microsoft, for example, remains in the process of collaborating with OpenSSL to integrate PQC into their open-source TLS libraries, but that is still in the works <cit.>. With that in mind, it remains to be said that no PQC was found in the TLS network traffic data at NCSA. As with RDP though, it is again worth looking at the security and PQC readiness of TLS here anyway.

Primarily, the security of TLS lies in its various cipher suites, or groupings of cryptographic algorithms, used for securing internet traffic. This data was collected in Table 2, where we show the top ten cipher suites appearing in the connections. While the most commonly found cipher suites in the data are today considered to be secure against classical adversaries, such as the top cipher suite TLS-AES-128-GCM-SHA256, a sizable portion of connections (over 8%) used weak cipher suites that have a concerning vulnerability.

Even though Chrome browser 116 and above offers TLS 1.3 hybridized Kyber support for PQC, the client does not turn it on by default. In addition, this problem is in part due to the other data point we examined, the TLS versions in use on the server end of the connections: only about 65% of connections used TLS version 1.3, the most recent and most secure option for this use case. The rest used TLS version 1.2, which still supports many of the weak cipher suites we found. Continuing this message of heightened security maintenance would greatly improve the resilience of network traffic to adversaries at NCSA if all machines were to adopt support for TLS v1.3. 

[width=, colback=white!95!black, boxrule=0.5pt, left=2pt,right=2pt,top=1pt,bottom=1pt]
remark
Remark remark:

This difficulty in adopting TLS v1.3, though, is still prevalent throughout the internet and not just here at NCSA, and it brings to light the unfortunate truth that if widely adopting the newest TLS has taken this many years, PQC adoption may show the same challenge.
























§ CURRENT PQC ADOPTION OF NETWORK PROTOCOLS AND THEIR APPLICATIONS AND LIBRARIES


This section describes the current adoption of critical network protocols regarding post-quantum cryptography (PQC). The problem of migrating applications to be quantum resistant is critical to ensure the security of key exchanges (KEX) and the integrity of digital signatures. As there is an increasing concern about future attacks made by quantum algorithms and computers, knowing the current adoption rate in critical protocols is crucial to ensure that these protocols are all secure against all these quantum computers. To the best of our knowledge, very few, if any, studies have done a systematic survey of PQC adoption across network protocol layers and the wide spectrum of workloads. 



[width=, colback=white!95!black, boxrule=0.5pt, left=2pt,right=2pt,top=1pt,bottom=1pt]
remark
Remark remark:

The key takeaway from our study is that only SSH and SSL have had PQC implementation and some real-world adoptions. The rest of the protocols have to roll their own crypto or encapsulate their data in TLS 1.3 (hybrid quantum key exchange). 



Table <ref> described the current PQC adoptions of some of the most important protocols in the world. Some protocols are one of many protocols lying in the applications layer of Internet Protocol Suite such as HTTP, DNS, DHCP, SSH, SSL, etc. In addition, there are widely used communications and authentication protocols such as Kerberos, Modbus, MySQL, SIP, and RDP. As described in Table <ref>, most critical protocols are not equipped with quantum-resistant algorithms, except for Secure Shell (SSH) and Secure Socket Layer (SSL). SSH protocol, OpenSSH library has implemented PQC for encryption in sntrup761x25519-sha512@openssh.com key exchange method after the release of its 9.0 version. SSL protocol, OpenSSL has another library called oqsprovider <cit.>, which is a provider to standard OpenSSL to implement quantum-safe cryptography for KEM key establishment in TLS1.3. Some quantum-safe algorithms that oqsprovider has implemented include KEM algorithms (BIKE, CRYSTALS-Kyber) and signature algorithms (Falcon, CRYSTALS-Dilithium).   




§ CASE STUDY: CHALLENGES OF MIGRATING 
 SCITOKENS TO PQC


SciTokens enables a federated ecosystem for authorization on distributed scientific computing infrastructures, enabling researchers to authenticate themselves as valid users to access scoped scientific computing resources<cit.>. Major supercomputing and scientific instruments (e.g., Laser Interferometer Gravitational-Wave Observatory (LIGO), Open Science Grid, Extreme Science and Engineering Discovery Environment (XSEDE)) rely on SciTokens. It is critical to safeguard SciTokens with quantum-resistant cryptography to avoid the forgery of digital signatures used in tokens (Figure <ref>).

Background.
The SciTokens protocol uses JSON web tokens (JWTs) between various parties, which will validate any access to resources. The process starts off by including information within the JWT. The information included most commonly consists of who issued the token, who the token is assigned to, when the token will expire, and what permissions and access the end user of such token will have. The tokens are then signed cryptographically to be authentic and certified when called upon to use. The benefit of this approach is that each organization or institution can issue its own tokens, allowing for easier and quicker access to resources without relying on one central authority to control the distribution of such tokens. 

Making SciTokens Quantum Resistant.
To be quantum-resistant, existing software must migrate encrypting in three main steps: signing, key exchange, and encryption. Key exchange and encryption are currently effective, but the looming threat of quantum computing could compromise these safeguards, potentially exposing sensitive data. For SciTokens, the main critical component that must be quantum secure is the signing step. When a digital signature is forged, the root of trust about the integrity of the signed token is broken. This undermines trust in historical transaction records, as the digital signatures securing them could be compromised by a quantum computer.




Existing work.
The NIST (National Institute of Standards and Technology) has determined four post-quantum resistant cryptographic algorithms. While there are four determined, one of the algorithms applies to just general encryption, while the other three refer to the digital signing aspect of quantum-resistant cryptography. The general cryptography algorithm is CRYSTALS-Kyber, which is beneficial due to its small encryption keys <cit.>. The three digital signing algorithms are CRYSTALS-Dilithium, FALCON, and SPHINCS+, which are distinct from each other due to the high efficiency that CRYSTALS-Dilithium and FALCON provide but also due to the variance in terms of mathematical approach SPHINCS+ has <cit.>.

Challenges of SciTokens migration to PQC.

1. As these four algorithms that NIST recommended are relatively new, the mitigation documents are scarce. There are protocol descriptions such as CRYSTALS-Dilithium and Falcon algorithms along with SPHINCS+ <cit.> <cit.> <cit.>, but implementation guidelines are very few if there are any.

2. Simply adapting SciTokens using existing cipher suites such as RSA-512 will create a new integration problem. For example, extending the length of the RSA digital signature from 256 bits to 512 bits will exceed the length of the HTTP header as defined in RFC 2616, breaking JWT when transmitted through intermediate proxies and interpreted by standard HTTP servers.

3. Emerging side-channel attacks targeting PQC implementations, such as Kyber and Dilithium, show that it is challenging to have a correct implementation, despite that the PQC protocols are correct <cit.>.


[width=, colback=white!95!black, boxrule=0.5pt, left=2pt,right=2pt,top=1pt,bottom=1pt]
remark
Remark remark:
To the developer community, it is unclear what is the process of determining which post-quantum secure algorithm to use and how to implement those correctly. In the future, we will work with the SciTokens community to correctly implement its quantum-resistant digital signature algorithm.






§ LESSONS LEARNED & 
 ACTION ITEMS TO PREEMPT NOVEL PQC ATTACKS

End-to-end PQC requires both client-server support.
To completely support PQC, we have found that both client and server have to support the same cipher suite, and widespread adoption hinges upon effective protocol negotiation when establishing the initial handshake, followed by key exchanges and digital signatures. For effective post-quantum security to be possible, it is thus necessary that as many machines as possible adopt post-quantum versions of network protocols.


Internet-wide scan for PQC implementations.
Another direction is to create a web spider that measures Post-Quantum Cryptography at NCSA and other organizations, constituting a "Network of PQC telescopes" that actively scans IPv4 and IPv6 space for PQC adoption across web services.
In addition, each organization can adopt its own internal PQC compliance checking tool as an HTTPS service so users can visit and perform self-assessments.

TLS v2.0 supporting PQC by default.
A more ambitious, yet significant direction for PQC adoption would be the introduction of PQC directly into the TLS standard as TLS v2.0. While some organizations are already working on integrating PQC as additions onto their TLS versions 1.2 and 1.3 <cit.>, distinguishing a fully post-quantum implementation with a new version number could prove to be essential in marketing the dire urgency in TLS adoption of PQC internet-wide.

This urgency is highlighted by the incredible potential of securing most network traffic, against both quantum and classical adversaries under TLS. We have shown in earlier sections that there are various methods to secure protocols such as DNS and RDP using just TLS as a wrapper. In addition to those methods, TLS Termination Proxies can also be used as wrappers around current legacy infrastructures to secure traffic seamlessly <cit.>. With many TLS-based solutions for cryptographically lacking protocols already out there, it should not be unrealistic to extend this security to the post-quantum world under a new version number, implementation, and standard.

Preempting PQC attacks.
Despite numerous attack attempts  <cit.> and analyses against current PQC drafts, none of the attacks have been publicly confirmed. We assert that if there were any successful attacks, they would highly likely leave traces in the network metadata, which would be measured by our network instrument or a honeypot <cit.>, given that we can achieve a widespread deployment on the Internet scale. Therefore, it is important to continue this line of network instruments for PQC measurement to preempt attacks <cit.> and failures <cit.>, particularly in HPC environments.

PQC downgrade attack. The security of a PQC scheme depends on both the protocol itself, the implementation, and the effective negotiation of cipher suites (e.g., hybrid) during the migration period. We hypothesize that future attacks may not attempt to exploit the PQC scheme directly but rather downgrade a PQC protocol to a classical protocol, which is less secure, similar to the SSL downgrade attack shown in  <cit.>.



[width=, colback=white!95!black, boxrule=0.5pt, left=2pt,right=2pt,top=1pt,bottom=1pt]
remark
Remark remark:
In the future, we will release an open dataset containing metadata of cryptographic suites across major network protocols (SSH, TLS, RDP, etc.) upon the paper's publication. With our network instrument in place, we will track the adoption of PQC algorithms, give feedback to NIST, expect to discover potential novel PQC attacks in the wild and help migrate HPC applications to be quantum-safe. Finally, we will expand our network instrument to a set of networked vantage points (PQC telescopes) to continuously measure PQC adoption at the Internet scale.








§ CONCLUSION AND FUTURE WORK

We have successfully implemented and deployed a novel network instrument to measure PQC adoption in real-time and provide historical trends of the adoption rates across seven layers of computer network protocols at the National Center for Supercomputing Applications at the University of Illinois at Urbana-Champaign. The instrument is placed at the network border router of nation-scale high-performance supercomputing centers and is a testbed for measurements. Using only metadata of network connections, without compromising user's privacy, we present the first statistical study of PQC's readiness across seven layers of computer network and HPC applications with a unique vantage point: a petascale supercomputing center with nation-scale visibility through the FABRIC TeraCore network.




§ ACKNOWLEDGEMENTS

This work was partly supported by the National Science Foundation under contract CCF #2319190. We acknowledge Dr. Jim Basney for the useful discussions about SciTokens, Dr. Santiago Nunez-Corrles, Dr. Edoardo Giusto, members of NCSA Quantum Task Force, Dependable Classical-Quantum Computing Systems Engineering Working Group, and the Illinois Quantum Information Science and Technology Center (IQUIST) faculty members for insightful feedback. We also want to recognize the following organizations/programs: NCSA Student Pushing Innovation Program, IBM-Illinois Discovery Accelerator Institute, FABRIC, PPoSS, and the National Science Foundation's CyberCorps Scholarship for Service (SFS) Program, NCSA Integrated Cyberinfrastructure/IRST team, and NIST HPC Security Working Group. Any opinions, findings, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of their employers or the
sponsors.






















IEEEtran





 Paper Title: REPS: Recycling Entropies for Packet Spraying to Adaptively Explore Paths and Mitigate Failures.








REPS]REPS: Recycling Entropies for Packet Spraying to Adaptively Explore Paths and Mitigate Failures


ETH Zürich
Microsoft


Microsoft


Microsoft


Microsoft


Microsoft


Sapienza University of Rome


ETH Zürich
Microsoft





















§ ABSTRACT 

Most existing datacenter transport protocols rely on in-order packet delivery, a design choice rooted in legacy systems and simplicity. However, advancements in technology, such as RDMA, have made it feasible to relax this requirement, allowing for more effective use of modern datacenter topologies like FatTree and Dragonfly. The rise of AI/ML workloads underscores the necessity for enhanced link utilization, a challenge for single-path load balancers due to issues like ECMP collisions.

In this paper, we introduce REPS, a novel per-packet traffic load-balancing algorithm that integrates seamlessly with existing congestion control mechanisms. REPS reroutes packets around congested hotspots and unreliable or failing links with remarkable simplicity and minimal state requirements.

Our evaluation demonstrates that REPS significantly outperforms traditional packet spraying and other state-of-the-art solutions in contemporary datacenter networks, offering substantial improvements in performance and link utilization.



[
    Torsten Hoefler
    August 5, 2024
===================






§ INTRODUCTION


With the recent explosion in AI/ML networks, the battle for better next-generation transport protocols has heated up with the recent release of Poseidon <cit.> by Google and the incoming release of Ultra Ethernet <cit.> by major tech players. Traditionally, AI/ML clusters have been using Infiniband and more recently Ethernet based solutions such as RoCEv2 <cit.> to reduce cost and offer an easier to deploy alternative.

However, existing solutions struggle to keep up with the increasingly demanding AI/ML workload where the expected load is much higher than what encountered in cloud workloads.  This increased load requirement showcases even more one of the main shortcomings of existing solutions: the routing algorithm. This is because, most of the times, the expectation is that each message will be routed using a single path to avoid out-of-order packets and reduce complexity of the load balancer. However, this results in ECMP collisions which reduce the overall performance of the whole transport layer due to queues building up and re-transmissions or PFC kicking in for lossless networks. 

Moreover, a number of recent industry papers have highlighted the drastic effect of failures in both training times and economic costs <cit.>. This highlights the importance of designing a transport layer with a load balancing scheme capable of adapting almost instantaneously to both network failures and asymmetries in the networks.

For this reasons, we propose REPS, a simple load balancing scheme based on adaptive packet spraying. We design REPS in order to work with any congestion control scheme and transport layer as long as it is able to handle out-of-order packets which is what next generation protocols are pushing for given the inherit performance advantage. 

REPS does not require any specific hardware support from switches beside ECMP which is commonly present in modern switches. If available, REPS can take advantage of packet trimming to further improve its accuracy but it is also capable of running without that.




 §.§ Background and Motivation

Modern datacenters tend to use Equal Cost Multi-Path (ECMP) <cit.> as their default routing scheme for their switches. ECMP uses the 5-tuple header of each data packet: the protocol number, the IP addresses and the TCP or UDP source and destination port numbers. It uses such information to determine the next link to take by using a hash function. Packets that belong to the same connection will normally be routed using the same exact path from source to destination, which is the case for RoCEv2 for instance. This helps keeping the design trivial since, ignoring failures, it is extremely unlikely to receive out-of-order packets at the receiver. On the other hand, if two different connections get hashed to the same link, there will be a ECMP collision resulting in queues building up and in more extreme cases re-transmissions. 

To mitigate such issues certain protocols have proposed splitting each flow into sub-flow and then routing each sub-flow individually, usually by artificially changing the source port field. This is the case of Flowlet Switching <cit.>, Flowcell <cit.> or PLB <cit.> that attempt to provide a trade-off between low amount of re-ordering and better performance. 

However, if we can design a new transport layer that can relax such out-of-orderliness requirement, then we can theoretically do such operation at the individual packet by choosing a random source port field for each packet. This is called packet spraying <cit.> and by its own, it is able to offer a significant performance improvement over single path algorithms or intermediate solutions such as Flowlet Switching.

Finally, we observe how we can take this one step further by adaptively spraying packets over paths. This, surprisingly, allows to achieve better results even with a perfectly symmetric network and an even bigger advantage when encountering failures. 



 §.§ Contribution






  REPS A powerful and lightweight adaptive load balancing scheme called Recycled Entropy Packet Spraying (REPS, Sec.<ref>). REPS achieves near-optimal performance at high bandwidth utilization and efficiently routes around path failures, congestion hotspots, and network asymmetries. Unlike more complex in-network load balancing solutions<cit.>, REPS does not require any switch support beyond ECMP. REPS provides a significant improvement over traditional packet spraying, both in perfectly symmetric networks and in the presence of asymmetries, whether due to differing topologies or potential failures. We demonstrate that REPS performs well regardless of the congestion control scheme used.









§ BACKGROUND



 §.§ Congestion Signals

In this section we are going over several congestion signals that REPS can take advantage of if they are implemented in the network. At minimum we require ECN to be supported by the switches.



  Delay End-to-end delay serves as a reliable indicator of congestion. Delay can be calculated by the sender through measuring the RTT, as seen in protocols like Swift <cit.> and TCP <cit.>. Alternatively, the receiver can annotate acknowledgment (ACK) packets with one-way delay information, a method used in protocols such as TIMELY <cit.> and DX <cit.>. In this context, we assume that precise delay measurements are obtained via NIC timestamping <cit.>.


  ECN-marking Explicit Congestion Notification (ECN) allows switches to indicate congestion by setting a bit in the traffic class field of the IP header when congestion is detected. This marked packet is then returned to the sender by the receiver, prompting the sender to adjust its load balancing or its transmission rate accordingly <cit.>. Unlike time-based signals that provide detailed queuing delay information, ECN uses a single-bit mark, offering less granularity. Switches can employ various strategies for marking packets. For instance, in Random Early Detection (RED)<cit.>, switches probabilistically mark packets based on queue size, with the marking probability increasing linearly between two thresholds (K_min and K_max). Although ECN was originally intended to mark packets upon enqueue<cit.>, marking at dequeue has been shown to enable quicker response times for congestion control (CC) algorithms. This dequeue marking method is easily implemented on most existing switches <cit.>. For this reason, the rest of the paper assumes the use of RED with dequeue marking for ECN packets. While ECN-based CC algorithms can respond faster than those based on delay, they may struggle with incast scenarios and can be difficult to fine-tune <cit.>.
Finally, while delay accumulates over multiple hops, ECN would be able to filter out such cases where there is not really a significant congestion happening but only a small amount of queueing over multiple hops. 



  Packet Losses Packet loss has long been a key indicator of severe congestion in networks <cit.>. However, using packet losses as the sole signal for congestion detection can result in delayed responses, as losses typically indicate a point of significant congestion. Additionally, packet loss detection often relies on timeouts, which can be challenging to calibrate and may lead to unnecessary retransmissions. Having said that, timeout can still be very useful in the case of network failures, especially when having out-of-order delivery support since we can not easily rely on a packet arriving out of order to detect losses.



  Packet Trimming Packet trimming is a technique where a switch removes portions of a packet, such as the payload, instead of discarding the entire packet. This method preserves crucial information, like headers, which enables the host to quickly identify and respond to congestion <cit.>. Researchers have shown that packet trimming can be implemented on switches such as Intel Tofino, Broadcom Trident 4, and NVidia Spectrum 2 by utilizing their capability to reroute packets to an alternate port when the primary egress queue is full <cit.>. 
This can be particularly helpful in detecting congestion events versus losses events, as the former would produce a trim even while the latter a timeout (excluding edge cases where even a trim would fail).



REPS utilizes ECN as its primary congestion signal to determine if a path is congested. Notably, ECN marking is performed at the egress to ensure that the most current information is captured. If packet trimming is supported, REPS leverages this feature to enhance its ability to distinguish between packets lost due to congestion and those lost due to network failures. In situations where packet trimming is not supported, REPS employs a straightforward timeout mechanism to detect packet loss within the network.







§ RECYCLED ENTROPY PACKET SPRAY (REPS)

REPS is a simple load balancer scheme that does not need any switch support besides ECMP <cit.>. With ECMP, each switch computes a hash function over some packet header fields to determine which path to forward the packet on. One of these fields is the entropy (e.g., the source port or the IPv6 Flow Label), which REPS changes on a per-packet basis.

The key idea behind REPS is very simple: we want to explore new possible paths when congestion is detected while re-using paths where there is little to no congestion. This has the immediate advantage of being both simple and lightway since we can potentially store only the last good entropy that we have seen and which is to be reused.
However, the slightly more advanced version of it uses a circular buffer of a fixed size where we save all the cached entropies. 

For the first 𝑏𝑑𝑝 of packets, REPS explores all possible entropies. This is necessary since initially we do not have any information about the status of the network. The receiver copies the entropies from the received packet to the ACK, thus forwarding them back to the sender. Whenever an ACK arrives, if the ACK is not ECN-marked, the entropy carried by the ACK is cached. Intuitively, because the corresponding data packet did not experience congestion, we can send other packets on the same path, and REPS will thus reuse it for subsequent packets. Otherwise, if the ACK was ECN-marked, REPS selects the next entropy. This can be further extended by defining additional policies (e.g., excluding bad entropies after some time or storing entropies in a priority queue based on their RTTs and/or ECN marking), but we do not explore this further in this initial version of this work. 
The circular buffer design guarantees that even when receiving back-to-back ACKs with good entropies, these can be correctly stored and used. On the sending side, if we have multiple elements in the buffer, we will always first select the oldest entropy to make sure to not get stuck on old information.

We evaluate REPS' advantages versus traditional packet spraying and ECMP in Sec. <ref>. Algorithm <ref> describes the REPS pseudocode, and Fig. <ref> shows an example of REPS' behavior.









  §.§.§ Interaction between CC and load balancing

We design REPS with the idea of it being capable of working with any transport layer that does not require in-order delivery of packets. Having said that, co-designing the congestion control protocol with the load balancer algorithm can bring several benefits. For these reasons, we optimize REPS to work with SMaRTT <cit.>.
REPS effectively mitigates the effects of fabric congestion, such as those caused by ECMP hashing collisions, without necessitating a reduction in the congestion window size. This capability is particularly important in permutation workloads on non-oversubscribed networks, where proper traffic balancing can theoretically prevent congestion. To enable REPS to respond to such congestion, SMaRTT is designed to ignore individual ECN-marked packets unless multiple marked packets are received. Specifically, SMaRTT maintains an exponentially weighted moving average of ECN-marked packets, updating the variable 𝑎𝑣𝑔_𝑤𝑡𝑑 using the formula 𝑎𝑣𝑔_𝑤𝑡𝑑 = α·𝑝.𝑒𝑐𝑛 + (1 - α) ·𝑎𝑣𝑔_𝑤𝑡𝑑. The congestion window remains unchanged as long as 𝑎𝑣𝑔_𝑤𝑡𝑑 < 0.25, meaning that the system does not react to congestion unless 25 percent or more of the acknowledgments indicate congestion—a threshold high enough to exclude transient load imbalances. This mechanism, referred to as Wait to Decrease (WTD), is illustrated in Fig. <ref>, which demonstrates how it significantly improves the flow completion time (FCT) for permutation workloads running on a 1,024-node non-oversubscribed fat tree network.



§ EVALUATION




  
Simulation Setup We ran simulations using and extending the htsim packet-level network simulator <cit.>. Our simulations consider different fat-tree topologies with 1024 nodes and different level of oversubsciption ranging from zero to 8:1. This is relevant due to the relevance of such topologies in production datacenters <cit.>. We configure the network with a 4 KiB MTU size, a bandwidth of 100 Gbps, and a switch traversal latency of 500 ns, reflecting the specifications of current-generation switches <cit.>. To simplify the analysis, we assume uniform link lengths and latencies, with each link exhibiting a latency of 500 ns.



  
State-of-the-art Comparison We pair REPS together with SMaRTT and compare it against packet spraying and ECMP. We use SMaRTT for all of them in order to have the same baseline regarding congestion control. We enable trimming for all the algorithms.



 §.§ Non-Oversubscribed Networks

In this section we evaluate the performance of REPS in a simple setting where there is no ovserubscription and no failures, meaning the network is perfectly symmetrical. Intuitively, this is the best situation for packet spraying since evenly splitting the packets across multiple links should result in the best performance. However, as we can see in Fig.<ref>, this is not the case as REPS still offers a 10% advantage over oblivious packet spraying. This is because of ECMP collisions that still happen with spraying. While over long period of time, each link will be evenly used, there will still be short term collisions happening that also increase the RTT of packets as seen in Fig.<ref>.





 §.§ Oversubscribed Networks

In Fig.<ref>, we evaluate the performance of SMaRTT with various load balancing algorithms: REPS, ECMP<cit.>, and oblivious packet spraying <cit.>. We simulate a 4 MiB permutation workload on a 8:1 oversubscribed fat tree network. The results show that REPS achieves again approximately a 10% improvement in completion time over oblivious packet spraying due to its superior fairness. In contrast, per-flow ECMP results in job completion times that are 1.5 times higher than those achieved with packet spraying and REPS, primarily because of increased flow collisions (which are not depicted in the plot). However, the individual RTTs for per-flow ECMP are relatively low since SMaRTT reduces the congestion window size upon detecting congestion caused by hash collisions. Additionally, an analysis of the packet RTT distribution in Fig. <ref> reveals that REPS maintains RTTs that are closer to the target with lower variability, indicating improved network utilization.





 §.§ Failures

We now evaluate what happens in a network where some links are failing, in particular we fail a fixed number of links between the aggregation switches and core switches of the fat tree. REPS now gains a 50% advantage (Fig. <ref>) over spraying because it is able to avoid more efficiently the failing links, while spraying would randomly select failing paths over time and hence increase the completion time. For ECMP, certain flows don't properly terminate as they are not able to adaptively avoid failing links.







§ CONCLUSION

We design REPS, a simple and resource-efficient load balancer that can be paired with different congestion control algorithms. REPS significantly improves the FCT, runtime and number of dropped packets as shown in our evaluation. REPS can drastically improve the load balancing performance compared to traditional per-flow ECMP or oblivious packet spraying at minimal complexity cost. We also show that REPS can effectively handle faults and network asymmetries. In a future version of this work we will expand REPS to be used alongside several congestion control algorithms and with different failure modes.












ACM-Reference-Format








 Paper Title: Priority and Stackelberg Game-Based Incentive Task Allocation for Device-Assisted MEC Networks.







Priority and Stackelberg Game-Based Incentive Task Allocation for Device-Assisted MEC Networks
    Yang Li2, Xing Zhang2 1, Bo Lei3, Zheyan Qu2, Wenbo Wang2 
2Key Laboratory of Universal Wireless Communications, Ministry of Education

2Beijing University of Posts and Telecommunications, Beijing 100876, China 

3Beijing Branch of China Telecom Co., Ltd., Beijing 100032, China 

1Email: zhangx@ieee.org

    August 5, 2024
=====================================================================================================================================================================================================================================================================================================================



§ ABSTRACT 
Mobile edge computing (MEC) is a promising computing paradigm that offers users proximity and instant computing services for various applications, and it has become an essential component of the Internet of Things (IoT). However, as compute-intensive services continue to emerge and the number of IoT devices explodes, MEC servers are confronted with resource limitations. In this work, we investigate a task-offloading framework for device-assisted edge computing, which allows MEC servers to assign certain tasks to auxiliary IoT devices (ADs) for processing. To facilitate efficient collaboration among task IoT devices (TDs), the MEC server, and ADs, we propose an incentive-driven pricing and task allocation scheme. Initially, the MEC server employs the Vickrey auction mechanism to recruit ADs. Subsequently, based on the Stackelberg game, we analyze the interactions between TDs and the MEC server. Finally, we establish the optimal service pricing and task allocation strategy, guided by the Stackelberg model and priority settings. Simulation results show that the proposed scheme dramatically improves the utility of the MEC server while safeguarding the interests of TDs and ADs, achieving a triple-win scenario.



Device-assisted edge computing, pricing and task allocation, Stackelberg game, Vickrey auction.




§ INTRODUCTION

With the rapid development of the Internet of Things (IoT) and 5G technology, there is an increasing need for computation-intensity and time-sensitive tasks. To address the need for real-time communication and computation in various emerging services, mobile edge computing (MEC) has emerged as a promising paradigm within IoT. It involves deploying computation, storage, and network services near IoT devices to support real-time processing of tasks. However, the increasing quantity of IoT devices and the demand for task processing could overwhelm MEC installations. Meanwhile, cost constraints limit the expansion of computation and storage capacities of edge servers (ESs). Additionally, IoT devices' continuously improving computation capabilities and storage capacities represent valuable resources that can enhance MEC <cit.>. Device-assisted MEC, as a subset of MEC, is garnering increasing attention from researchers, and much work has been carried out to investigate the issues in device-assisted MEC <cit.>.



However, none of the aforementioned works have addressed incentive design. Indeed, task IoT devices (TDs), auxiliary IoT devices (ADs), and ESs represent typical profit-driven entities with conflicting interests. Effective collaboration among the three parties may be impeded in the absence of a proper incentive mechanism. Therefore, some research has focused on motivating the diverse participants in MEC by developing incentive mechanisms. The authors in <cit.> explored the optimization of resource allocation via pricing mechanisms in MEC systems. The study in <cit.> devised a blockchain-based resource transaction framework, which utilizes the blockchain's features of decentralization, immutability, and smart contracts for secure resource transactions. To effectively incentivize user devices to act as computation providers (CPs) for computation requestors (CRs), Chen et al. <cit.> introduced an incentive mechanism based on contract theory to mitigate the information asymmetry issue in D2D computation offloading. However, the above studies exclusively focus on interactions between the ES and TDs or the ES and ADs, neglecting the scenario of interactions among all three parties. Consequently, none of the above incentive mechanisms can be effectively employed in device-assisted MEC scenarios.

In response to the limitations of current research, we have developed an incentive-driven pricing and task allocation algorithm tailored to device-assisted MEC scenarios. Specifically, we initially utilize the Vickrey auction mechanism to incentivize ADs. Subsequently, we utilize the Stackelberg game to analyze the interaction process between TDs and the ES. Lastly, building upon the Stackelberg model and priority settings, we introduce a service pricing and task allocation algorithm aimed at maximizing the utility of the ES. In addition, we conduct extensive simulations under different parameter settings to demonstrate the effectiveness of our incentive mechanism.




§ SYSTEM MODELS



This section presents the model of a device-assisted MEC network, as depicted in Fig. <ref>. The scenario involves the edge server (ES), task IoT devices (TDs), and auxiliary IoT devices (ADs), which are described as follows.

(1) Edge server: An ES is linked with a base station (BS) via a wired connection, offering edge computing services to users within the BS-managed cell. When the ES is overloaded, it can recruit some ADs to handle the overloaded tasks, such as TD 1, TD 4, and TD 5 in Fig. <ref>, but it should pay rewards to the ADs that handle the tasks.

(2) Task IoT device: TDs are devices that request task offloading from the ES. Assuming TDs act rationally, they offload some or all data to the ES based on its pricing to maximize their own utility.

(3) Auxiliary IoT devices: During recruitment initiated by the ES, ADs can provide information such as their location and available resources, as well as submit bids at the lowest acceptable price per CPU cycle. This allows them to benefit by assisting the ES in handling tasks.




The system is assumed to have N TDs and M ADs, denoted by the sets 𝒩 and ℳ, respectively. The task generated by TD i∈𝒩 is denoted as H_i ≜{L_i, C_i, t_i^max}, where L _i represents the size of the task H_i, C_i is the required number of CPU cycles to complete H_i, and t_i^max indicates the time constraint of H_i. Similar to <cit.>, we introduce a complexity factor ϕ_i to quantify the CPU cycles needed to process one bit of H_i. The offloaded portion of TD i's task is denoted as h_i≜{l_i, c_i, t_i^max}, where l_i signifies the size of h_i, c_i represents the required number of CPU cycles to complete h_i, and t_i^max indicates the time constraint of h_i. Based on the complexity factor ϕ_i, we can obtain c_i = ϕ_il_i, where l_i ∈ [0,L_i] and c_i ∈ [0,C_i].



 §.§ Communication Model


We use the same communication model as in <cit.>, and the transmission delay between the BS and ES is disregarded. The transmission rate from TD i to the ES is defined as R_i^B = W_B log_2(1+p_ig_i^B/N_0), where W_B is the channel bandwidth between TD i and the BS. p_i is the transmit power of TD i, and N_0 is the background noise power. g_i^B = μ_0(d_i^B)^τ is the channel gain between TD i and the BS, where μ_0 is the fading component, τ is the channel path loss exponent, and d_i^B is the distance between TD i and the BS.

Additionally, when the ES schedules the offloaded task h_i for processing on AD j, the transmission rate from the BS to AD j is defined as R_B^j = W_B log_2(1+p_Bg_j^B/N_0), where p_B is the transmit power of the BS. g_i^B = μ_0(d_j^B)^τ is the channel gain between the BS and AD j, and d_j^B is the distance between AD j and the BS.



 §.§ Computation Model

The ES determines whether each offloaded task is computed locally or scheduled for processing on an AD. For offloaded task h_i, we use x_i∈{0,1,2, ..., M} to indicate its processing location. x_i = 0 indicates that task h_i is processed on the ES, while x_i = m ∈{1,2,... ,M} indicates that task h_i is processed on AD m. Subsequently, we elaborate on each of these two scenarios.

(1) Processing on the ES: Based on the task model and communication model, the transmission time of task h_i from TD i to the ES can be calculated as t_i,B^tran=1_x_i=0l_i/R_i^B, where 1_{·} is the indicator function and equals 1 (resp., 0) if the condition is true (resp., false).

Given that the time constraint of task h_i is t_i^max, the processing time of h_i on the ES must not exceed t_i^max-t_i, B^tran. Therefore, the minimum computational resources allocated by the ES for TD i can be calculated as 

    f_i^B=1_x_i =0c_i/t_i^max-t_i, B^tran.


(2) Processing on AD j: Similarly, the transmission time of task h_i to AD j through the BS relay can be calculated as t_i,j^tran= l_i/R_i^B+l_i/R_B^j. Therefore, the minimum computational resources allocated by AD j for TD i can be calculated as 



    f_i^j=1_x_i=jc_i/t_i^max-t_i,j^tran.


In this study, we assume that the ES and ADs handle task h_i with the minimal computational resources necessary to meet its time constraint, motivated by their respective interests.



 §.§ Utility Model

(1) TD i’s model: For TD i, we employ a logarithmic utility function to represent the satisfaction derived from offloading computation tasks, which can properly reflect the relationship between TD i and satisfaction <cit.>. This function can be expressed as



    s_i = w_i ln(1+l_i),

where w_i denotes the satisfaction factor of TD i. Clearly, as TD i offloads more data, both its own battery life and hardware wear decrease, thereby leading to a greater value of the satisfaction function.

Naturally, when a TD offloads its computational task to the ES, it incurs a fee. Let's assume that for TD i, the ES imposes a fixed fee of d_i per CPU cycle required to process its task. Consequently, the payment from TD i to the ES can be calculated as



    o_i = d_iϕ_il_i.


Additionally, TD i incurs energy consumption for both local processing and offloading tasks, with its energy cost calculated as



    e_i = γ q_i ϕ_i(L_i-l_i)+γ p_il_i/R_i^B,

where q_i denotes the energy consumption per CPU cycle when TD i processes a task, and γ denotes the cost per unit of energy consumption.

Thus, the utility of TD i can be expressed as

    U_i = s_i+v_i-e_i-o_i,

where v_i represents the value to the user when the task is successfully processed.

(2) ES's model: For the offloaded task h_i, if the ES assigns the task to be processed locally, the energy cost is calculated as e_B^i = γ q_Bϕ_il_i, where q_B represents the energy consumption per CPU cycle when the ES processes a task[Since devices need to find a balance between power consumption and performance, they generally use processors with lower power consumption and relatively lower performance. In contrast, ESs, which frequently handle computationally intensive tasks and have access to a constant power supply, tend to utilize higher-performance processors. Therefore, q_B>q_i, ∀ i ∈𝒩.].

Thus, the utility of the ES for handling task h_i can be expressed as



    U_B,l^i = d_iϕ_il_i - γ q_Bϕ_il_i.


When the ES assigns task h_i to AD j for processing, its utility is calculated as the revenue generated minus the payment made to AD j and the energy cost incurred for forwarding that task. This can be expressed as

    U_B,j^i = d_iϕ_il_i - d_jϕ_il_i-γp_Bl_i/R_B^j,

where d_j represents the reward obtained per CPU cycle when AD j assists ES in processing a task.

In summary, the total utility of the ES can be expressed as

    U_B = ∑_i=1^N (1_x_i=0U_B,l^i+∑_j=1^M 1_x_i=j U_B,j^i).


(3) AD j's model: Assuming that AD j requires C CPU cycles to process tasks allocated by  the ES, and its bid is denoted as a_j, its utility can be expressed as U_j = (d_j-a_j)C.




§ PROBLEM FORMULATION AND ANALYSIS



 §.§ Pricing for ADs

The recruitment of ADs for the ES can be conceptualized as an auction game. Among ADs meeting the resource requirements of task h_i, the one which brings the highest utility to the ES wins the auction, serves as the processing device for the task, and earns a reward. Implementing the Vickrey auction can incentivize ADs to bid honestly <cit.>. In this auction format, the pricing strategy employed by the ES for AD j involves setting the price at the lowest bid among those exceeding AD j's offer. At this point, the AD's individual rationality is satisfied, i.e., its utility is non-negative.

The interactions between the ES and TDs can be modeled as a Stackelberg game. The subsequent subsection introduces the optimization problem based on the Stackelberg game and offers an analysis.





 §.§ Problem Formulation

Fig. <ref> illustrates the Stackelberg game involving the ES and TDs. In this game, the ES, acting as the leader, sets its pricing policy d_i for TD i, ∀ i ∈ N, while TD i, as a follower, decides the size of task h_i to offload. The pricing strategy and offloading strategy are denoted as d=(d_1,d_2,.... ,d_N) and l=(l_1,l_2,.... ,l_N), respectively. Since the pricing strategy is formulated by the ES, the goal of the pricing strategy is to maximize the utility of the ES. However, in the scenario we consider, the ES must not only determine the pricing strategy but also assign processing locations for all offloaded tasks. Therefore, the problem of maximizing the utility of the ES can be formulated as 



    𝒫_1:      max_d,x U_B 
    s.t.     (10a):d_i^min≤ d_i ≤ d_i^max, ∀ i 
       (10b):x_i ∈{0,1,...,M}, ∀ i 
       (10c):∑_i=1^N f_i^B1_x_i=0≤ F_B 
       (10d):∑_i=1^N f_i^j 1_x_i=j≤ F_j, ∀ j 
       (10e):1_x_i=0U_B,l^i + ∑_j=1^M 1_x_i=jU_B,j^i≥ 0, ∀ i,

where x={x_1,x_2,.... .x_N} denotes the task processing locations assigned by the ES for all TDs. F_B and F_j denote the overall idle resources of the ES and AD j, respectively. Constraint (10a) shows that the pricing strategy of the ES is within a specified range, where d_i^max and d_i^min represent the maximum and minimum pricing strategies for TD i, respectively. Constraint (10b) states that each offloaded task is processed at the ES or an AD. Constraints (10c) and (10d) specify that the total computational resources allocated by the ES and ADs cannot exceed their idle resources. Constraint (10e) specifies that the service pricing and task allocation must satisfy the individual rationality of the ES.

Subsequently, TD i determines the amount of offloaded data l_i to maximize its utility after being informed of the ES's pricing strategy. Thus, the optimization problem for TD i is expressed as



    𝒫_2:      max_l_i U_i 
    s.t.     (11a):0≤ l_i≤ L_i 
       (11b):U_i ≥ 0,

where (11a) constrains the amount of offloaded data l_i, while constraint (11b) states that TD i is a rational entity.



 §.§ Analysis of Game Problem

In this section, we initially assume that the ES possesses ample computational resources, meaning it doesn't delegate tasks to ADs due to overload. Subsequently, upon attaining the optimal solution based on this assumption, we account for the resource constraints of the ES and refine the solution accordingly.

Given the current assumption, we can utilize the backward induction method to analyze the proposed problem. In the first stage, the optimal offloading decision for each TD is determined. In the second stage, according to the optimal strategy of all TDs, we ascertain the optimal pricing strategy for the ES.

Initially, by referring to equations (<ref>), (<ref>), (<ref>), and (<ref>), we can rewrite the utility function of TD i as

    U_i =    w_i ln(1+l_i)+v_i 
       -(γ q_i ϕ_i(L_i-l_i)+γ p_il_i/R_i^B)-d_iϕ_il_i.

Then, the first-order derivative of U_i can be calculated as

    ∂ U_i/∂ l_i = w_i/1+l_i +γ q_i ϕ_i - γp_i/R_i^B-d_iϕ_i.

Next, the second-order derivative of U_i can be calculated as

    ∂^2U_i/∂ l_i^2 = -w_i/(1+l_i)^2.


It can be seen that the second-order derivative of U_i is negative. Consequently, the utility function of TD i exhibits strict concavity, indicating the existence of a unique optimal value of l_i that maximizes U_i.

By setting the first-order derivative of U_i to zero, we can derive the optimal offloading strategy for TD i as

    l_i^* = w_iR_i^B/γ p_i + d_iϕ_iR_i^B -γ q_i ϕ_iR_i^B -1.


Subsequently, we will derive the optimal pricing strategy for the ES. Given the assumption of sufficient computational resources for the ES, the expression for U_B undergoes transformation to U_B = ∑_i=1^N U_B,l^i = ∑_i=1^N (d_iϕ_il_i - γ q_Bϕ_il_i).


By substituting the optimal offloading strategy l^* for TDs obtained in the first stage into U_B, we can rewrite U_B as



    U_B  = ∑_i=1^N (d_iϕ_i-γ q_Bϕ_i)(wR_i^B/γ p_i + d_iϕ_iR_i^B -γ q_i ϕ_iR_i^B -1).


Then, the first-order derivative of U_B is formulated as

    ∂ U_B/∂ d_i =   ϕ_i(w_iR_i^B/γ p_i + d_iϕ_iR_i^B -γ q_i ϕ_iR_i^B -1)-
       (d_iϕ_i-γ q_Bϕ_i) w_iϕ_iR_i^B^2/(γ p_i + d_iϕ_iR_i^B -γ q_i ϕ_iR_i^B)^2.


Next, the second-order derivative of U_B is calculated as

    ∂^2 U_B/∂ d_i^2 = -2w_iϕ_i^2 R_i^B^2[γϕ_iR_i^B(q_B-q_i)+γ p_i]/(γ p_i + d_iR_i^B -γ q_i ϕ_iR_i^B)^3.


Since q_B-q_i>0, as stated in Section II-C, this implies that the second-order derivative of U_B is negative. Consequently, the utility function of the ES demonstrates strict concavity.

Similarly, the first-order derivative of U_B is set to 0, 

    ∂ U_B/∂ d_i = 0,

then we can derive the optimal pricing strategy of the ES for TD i. However, the equation is complex and nonlinear, making it challenging to find a closed-form optimal solution for d_i directly. As ∂^2U_B/∂ d_i^2<0, indicating ∂ U_B/∂ d_i is strictly monotonic, we can employ the bisection search method to obtain a near-optimal solution for d_i.

At this point, we have obtained the optimal pricing strategy of the ES for each TD when the ES has sufficient computing resources. Next, we need to consider the constraint of the ES's computational capacity. When the computational resources of the ES are insufficient, certain tasks can be delegated to ADs for processing, with corresponding fees remunerated to the ADs. Moreover, typically, higher asking prices result in less data offloaded by TDs. Hence, the ES can raise the asking price to incentivize TDs to offload less data, ensuring that the idle resources of the ES and ADs meet the time constraints for all offloaded tasks.

As mentioned above, when the computational resources of the ES are insufficient, we need to solve two problems: determining task allocation among ADs and deciding whether and by what margin to adjust the asking price of the ES for certain TDs. To address these two issues, we will give the specific service pricing and task allocation strategy in the next section.



§ PROPOSED SOLUTION

Recalling the problem analysis in the previous section, we first need to determine whether the computational resources of the ES are sufficient. Referring to Sections <ref> and <ref>, we can determine the  amount of computing resources that the ES needs to allocate to TD i is f_i^B(l_i^*) = ϕ_il_i^*/t_i^max-t_i, B^tran, assuming the ES has sufficient resources. Therefore, we can get the total computational resources that the ES needs to provide as f_B =∑_i=1^Nf_i^B(l_i^*). If f_B>F_B, the ES's computational resources are insufficient. At this point, we need to assign some tasks to ADs for processing and raise the asking price of the ES for certain TDs to incentivize them to offload less data.

A priority-based strategy is employed to screen tasks designated for processing on ADs and determine their processing locations. The priorities of offloaded tasks and ADs are defined as



    O_i =  d_iϕ_il_i^* - γ q_Bϕ_il_i^*/f_i^B(l_i^*),

and



    Q_j^i = U_B,j^i(l_i^*),

respectively.
Here, O_i represents the priority of task h_i for processing on the ES. A higher value indicates higher priority, which corresponds to higher utility per unit of computational resource used by the ES to process this task. Q_j^i signifies the priority of AD j for handling task h_i. A higher value implies higher priority, which corresponds to higher utility when the ES delegates task h_i to AD j for processing.

In terms of increasing the asking price, it is crucial to determine the maximum threshold of the asking price for each task. Exceeding this threshold would dissuade users from offloading any data. Thus, by substituting l_i^*=0 into equation (<ref>), we can calculate the maximum asking price for TD i as

    d_i^max = w_i/ϕ_i+γ(q_i-p_i/ϕ_iR_i^B).


Given the definitions and analysis provided above, Algorithm 1 is presented.

0.5em
 
0.5em 
Algorithm 1 allocates task processing positions based on task and AD priorities. It encourages TDs to offload less data by increasing the asking price gradually. Additionally, Algorithm 1 imposes a constraint on the maximum data each TD can offload to prevent irrational users from causing shortages in supply-side resources through excessive data offloading.

Algorithm Complexity Analysis: In Algorithm 1, we first solve N equations using the bisection method. Assuming the accuracy requirement is ϵ, the complexity is 𝒪(Nlog(1/ϵ)). Then, the TDs and ADs are sorted according to priorities with time complexities of 𝒪(NlogN) and 𝒪(MlogM), respectively. Additionally, the maximum number of price increments is L, which is a constant. Therefore, the overall time complexity of Algorithm 1 is 𝒪(NMlogNlogM).








§ NUMERICAL RESULTS

This section presents simulation results to assess the performance of our proposed algorithm. The network scenario depicted in Fig. 1 is simulated. Table I lists the primary simulation parameters, some of which have been utilized in <cit.>. All the above parameters will not change unless otherwise stated. Furthermore, we assume all TDs are rational entities, meaning they decide the quantity of offloaded data based on maximizing their utility. For comparison, we also simulate the following three intuitive strategies as benchmarks.



  * Uniform Pricing (UP): The ES has consistent pricing for all TDs.

  * Non-recruitment of ADs (NR): The ES does not recruit ADs to participate in task processing.

  * No Prioritization and Price Increment (NPPI): Prioritization and price increment are not factored into the determination of pricing and task allocation strategies.




Fig. 3 depicts the utility of the ES under different strategies. In Fig. 3(a), the utility of the ES under the UP strategy consistently lags behind the other three strategies due to insufficient incentives for TDs. In Fig. 3(b), the utility of the ES under the NR and UP strategies remains constant as the number of ADs increases due to the underutilization of ADs' resources. These two figures demonstrate that the proposed scheme shows more pronounced performance enhancement compared to the other schemes, especially with a larger number of TDs. And the proposed scheme can improve the utility of the ES by about 45% compared to the UP strategy.




Fig. 4 illustrates the average utility of each AD versus the number of TDs and ADs. As shown in Fig. 4(a), the average utility of each AD is greater than 0 only when the resource demand of TDs surpasses the computational capacity of the ES. And the average utility of each AD under the proposed scheme is greater than or equal to the other schemes. Additionally, consistent with Fig. 3, both the proposed scheme and the NPPI strategy exhibit an inflection point at N = 100. Fig. 4(b) reveals that with inadequate resources of the ES, the average utility of each AD increases as the number of TDs rises and the number of ADs decreases. This phenomenon occurs because a higher number of TDs results in more resource demand on ADs, while a lower number of ADs reduces competition among them.



Fig. 5 exhibits the average utility of each TD versus the number of ADs and TDs. From Fig. 6(a), it is evident that at N=60 and 90, the resources on the supply side are adequate, and increasing ADs does not impact the average utility of each TD. However, at N = 120 and 150, the supply-side resources become inadequate. This results in an improvement in the average utility of each TD as the number of ADs increases until the supply-side resources become sufficient again. After this point, the average utility of each TD stabilizes. Additionally, Fig. 6(b) illustrates that the UP strategy results in uneven incentives among TDs, thereby lowering the average utility of each TD. Tailored incentives for all TDs, employing the Stackelberg model, can effectively enhance their inclination towards offloading, thereby elevating the average utility of each TD by almost 50%.



§ CONCLUSION

This paper investigates a device-assisted MEC network. To optimize the utility of the ES while protecting the interests of ADs and TDs, we employ the Vickrey auction to price the resources of ADs and the Stackelberg game to model the ES-TDs interactions. To foster efficient collaboration among TDs, ADs, and the ES, we propose a service pricing and task allocation algorithm based on the Stackelberg model and prioritization. Simulation results demonstrate that the proposed scheme maximizes the utility of the ES and enhances that of TDs and ADs. Future work is in progress, which involves extending from a single cell to multiple cells and exploring collaboration among multiple cells.



§ ACKNOWLEDGMENT


This work is supported by the National Science Foundation of China under Grant 62071063 and by the BUPT Excellent Ph.D. Students Foundation under Grant CX20241066.


IEEEtran




 Paper Title: Discovery of 6G Services and Resources in Edge-Cloud-Continuum.







Discovery of 6G Services and Resources in Edge-Cloud-Continuum
    
    
        Mohammad Farhoudi1, Masoud Shokrnezhad1, Tarik Taleb2, 4, Richard Li3, and JaeSeung Song4

    
    
        1 Oulu University, Oulu, Finland;
        {mohammad.farhoudi, masoud.shokrnezhad}@oulu.fi 

        2 Ruhr University Bochum (RUB), Germany; tarik.taleb@rub.de
        

        3 Southeast University, Nanjing, China; richard.li@seu.edu.cn 

        4 Sejong University, Seoul, Korea; jssong@sejong.ac.kr
    

    August 5, 2024
=====================================================================================================================================================================================================================================================================================================================================================================================================================================================



§ ABSTRACT 
The advent of 6G networks will present a pivotal juncture in the evolution of telecommunications, marked by the proliferation of devices, dynamic service requests, and the integration of edge and cloud computing. In response to these transformative shifts, this paper proposes a service and resource discovery architecture as part of service provisioning for the future 6G edge-cloud-continuum. Through the architecture's orchestration and platform components, users will have access to services efficiently and on time. Blockchain underpins trust in this inherently trustless environment, while semantic networking dynamically extracts context from service requests, fostering efficient communication and service delivery. A key innovation lies in dynamic overlay zoning, which not only optimizes resource allocation but also endows our architecture with scalability, adaptability, and resilience. Notably, our architecture excels at predictive capabilities, harnessing learning algorithms to anticipate user and service instance behavior, thereby enhancing network responsiveness and preserving service continuity. This comprehensive architecture paves the way for unparalleled resource optimization, latency reduction, and seamless service delivery, positioning it as an instrumental pillar in the unfolding 6G landscape. Simulation results show that our architecture provides near-optimal timely responses that significantly improve the network's potential, offering scalable and efficient service and resource discovery.





Mobility-Aware Service Discovery Architecture, Predictive Discovery, Semantic Networking, Blockchain-aware Resource Discovery, Service Prediction, Service Requests, Service Advertisement, Service Selection, 6G Networks, and Edge-Cloud-Continuum












§ INTRODUCTION





In the wake of the 6G network revolution, ushering in unprecedented advancements in connectivity, a myriad of transformative services emerges,

reshaping people's lives. 

We are entering an era of ever-increasing device variety and number, driven by the convergence of Augmented and Virtual Reality (AR/VR) with next-generation wireless networks, robotics, and Artificial Intelligence (AI). Individuals will demand plenty of services, from localized specialized services <cit.> to immersive metaverse experiences which leads to extreme numbers of Service Requests (SRs). Having real-time devices with differing maximum data rates for machine-type communications could exacerbate the problem. Interest in multifaceted applications and services would lead to an ever-growing deluge of diverse data types as well. Take, for instance, a user in the metaverse equipped with a Head-Mounted Display (HMD), haptic sensors, and more, simultaneously accessing a diverse array of services – such as capturing breathtaking vistas with an HMD or obtaining information about their surroundings – with different types of requests. 




Device and SRs proliferation across various verticals introduces a wide range of service requirements <cit.>. Stringent Quality of Service and Quality of Experience (QoS/QoE) standards must be upheld to ensure the prompt delivery of cutting-edge real-time applications.

Anticipating the advent of 6G networks, we envision a landscape characterized by remarkable dynamism, marked by frequent changes in user requirements and locations, and the potential for resource or Service Instance (SI) shifts.

Consider our aforementioned user traversing the urban landscapes, dynamically connected to different drones to access their desired services. The perpetual motion of the user and Point of Attachments (PoAs) illustrates future dynamism as a challenge for accessing and using services in real-time. Conceivably, service provisioning is a problem-solution‑fit required to set up and make services available for users. It involves discovering, configuring, deploying, and managing services to ensure they are accessible, functional, and secure. Conventional service provisioning methods fall short when confronted with the impending surge in SRs accompanied by stringent requirements, as their static nature fails to adapt swiftly to the fluidity of evolving user needs inherent in the forthcoming 6G network. 








Within the service provisioning framework, Service and Resource Discovery (SaRD) assumes a pivotal role in ever-shifting user bases in a dynamic environment. It encompasses the process of systematically detecting, identifying, registering, and indexing offered network resources, facilitating access to services–tailored to meet specific needs with varied performance, functionality, and availability–in a timely and efficient manner. 

The expansive scope of users, devices, and network coverage demands scalable SaRD mechanisms. Considering automation and maintaining the continuity of discovered services are equally significant, requiring continual adaptation of SaRD processes in response to the dynamic nature. Consequently, traditional SaRD methods are less effective while lacking intelligence in discovering the desired SIs. 

In a future network environment where users transmit many requests in various ways, context-awareness is crucial. The absence of such awareness hampers the network's ability to determine the relevance of individual requests, contributing to the transmission of excessive information and impeding effective discovery based on real preferences. Hence, the network's ability to discern SR semantic from different models becomes essential. Moreover, 

providing access to legitimate services and resources in a trustless environment, in which SIs and user trust are unclear, will place an increasing emphasis. 

All in all, there is a pressing demand for a scalable, automated, trustworthy, and efficient foundation for SaRD. 



Addressing these vital issues requires the transformation of service provision, especially in SaRD with its upcoming architectures and frameworks, focusing on decentralized methods to enhance scalability. Balanji et al. <cit.> proposed a Service Discovery Management Scheme (SDMS) to enhance availability and reduce SR latencies by clustering services based on functional contexts that adapt to changing consumption patterns. Modern SaRD architectures also leverage semantic information to minimize service search latency.  

Deng et al. <cit.> developed a distributed SaRD using semantic information and graph attention networks to deal with the dynamic environment's service requirements, emphasizing robustness and reducing central cloud reliance. Additionally, some approaches take security-related concerns into account when handling SaRD. Chen et al. <cit.> proposed blockchain-based SaRD in the core network that records its function information on a blockchain, ensuring secure registration and discovery while avoiding trust issues.


These studies, while pioneering, fall short of accommodating future futuristic networks' complexities characterized by fluctuating user bases. While some research considered the issues mentioned above through interconnected networks, they often neglect the correlations between these networks and the need for dynamic adaptation, thereby not fully addressing the intertwined issues of scalability and determinism. 

Managing network changes like device additions or removals has been attempted, by locating service or resource directories. Still, future networks require predictive discovery for stringent service requirements, struggling to support dynamicity and maintain service continuity. Current architectures focus on semantics solely for SR matching, overlooking the broader potential of semantic networking, which includes semantic-aware data transmission rather than bit-oriented data and supporting determinism by extracting SR meanings across different modalities.

Finally, existing SaRD architectures do not fully address trustworthy services in large-scale environments, with blockchain's inherent challenges potentially bottlenecking service responses.



This paper bridges the gap by presenting a SaRD architecture for 6G that adeptly navigates the evolving and dynamic environment and paves the way for proactively meeting users' increasing demands. Our contribution consists of the categorization of a SaRD mechanism into distinct parts that lead to adaptive, scalable, intelligent, trustful, and contextually aware discovery. This breakdown fine-tunes service orchestration, showcases platform capabilities, optimizes network infrastructure, and considers application requirements. Our proposed architecture harnesses intelligence by incorporating predictive discovery, anticipating user requests in the next time frame to maintain continuity; semantic networking for precise request understanding–aimed at adapting to user preferences–and efficient transmissions; a robust zoning mechanism ensuring SaRD scalability; and blockchain for decentralized trust across all facets of the discovery process. This comprehensive approach pioneers a resilient 6G SaRD, redefining service provisioning.
























































The upcoming sections of this paper are arranged coherently as follows. The fundamental principles, inherent challenges, and potential solutions associated with SaRD are explored in Section <ref>, and

the proposed SaRD architecture, including its elements, components, and processes, are described in Section <ref>. In Section <ref>, future research directions are presented, and finally, Section <ref> offers concluding remarks that encapsulate the study's findings. 

































§ ‌BACKGROUND


 
In the future transformative landscape, and to ensure users' connectivity to enhanced services, 6G will emerge as a beacon of promise, poised to deliver ultra-fast data rates, minimal latency, and extensive connectivity. This vision of ubiquitous connectivity relies on the edge-cloud-continuum, revolutionizing service delivery and continuity by integrating cloud and edge computing. This synergy facilitates fluid data movement and processing, which empowers 6G networks to support bandwidth-intensive and real-time applications. Given the integrated infrastructure and the mentioned futuristic 6G edge-cloud-continuum issues,

addressing the multifaceted challenges related to SaRD 

is imperative, which lays a strong foundation for tackling other critical areas like efficiency.

















 §.§ Scalability





Considering the expected number of devices in the near future and their surging demand, SaRD mechanisms' scalability emerges as a key factor. Scalability denotes SaRD's ability to efficiently adapt to the escalating number of users, devices, and SIs within the dynamic edge-cloud-integrated 6G network. A particularly advantageous approach is Dynamic Overlay Zoning (DOZ), also known as sharding or splitting, which dynamically divides networks into distinct zones, each capable of independent operation while interoperating and exchanging information <cit.>. DOZ offers several compelling benefits, making it a desirable choice for dealing with the issue, yet its complexity should be handled carefully. With Machine Learning (ML) algorithms, clustering can be used to predict future demands and adjust the number of zones and their size to proactively enhance network performance and responsiveness, solving the SaRD methods' scalability issue.








 §.§ Determinism



Determinism, in this context, refers to reliably meeting SRs' needs. In a dynamic and evolving network with user mobility, diverse SRs, and fluctuating resources, the challenge lies in maintaining uninterrupted service flow for desired QoS/QoE. To ensure timely resource access within a futuristic network's dynamic fabric, it is instrumental to access the necessary SRs and plan ahead through Predictive discovery <cit.>. Anticipating SRs by forecasting them using historical data, traffic patterns, and network feedback elevates the concept of service continuity to an innovative and futuristic form. As our Metaverse user transitions between places, they must harness stored supercapacitor energy in smart bricks to meet their energy needs. Predictive discovery forecasts these needs, transmits the SRs along their path, and ensures a continuous, uninterrupted user experience upon arrival.








To achieve precise forecasting and thus the needed service requirements, the SaRD mechanism must interpret diverse contexts embedded in requests and manage distinct workloads across various timeframes. Increasingly, users communicate with their surroundings through various means like voice commands, text inputs, and sensory data, requiring an adaptive and automatic method to decipher the SRs' context across varied temporal scopes crucial for streamlined user engagement. Semantic networking and communication help extract SR meanings, eliminate superfluous information by exploiting semantic redundancy and ambiguity,

and contribute to determinism by providing a clear understanding of user needs and environmental context. Semantic networking leverages such semantic encoding and decoding techniques as semantic-based calculation and feature extraction to optimize data transmission <cit.>. Semantic Service Composition (SeCo) facilitates intelligent decision-making by filtering data and identifying suitable combinations of SIs to meet complex user needs. Specifically, anticipating our Metaverse user's concert request involves predicting SR surges at specific locations through other users' requests with varying modalities.

















 §.§ Trustworthiness







As the network environment evolves towards decentralized and autonomous structures, the challenge lies in upholding trust among various entities. In the context of 6G edge-cloud-continuum, where resource and SI trustworthiness loom as a substantial concern, blockchain provides decentralized trust, dynamic orchestration, and security enhancement <cit.>. Using blockchain technology fosters trustless collaboration among decentralized entities that automate cooperative processes, ensuring secure interactions with legitimate services <cit.>. Our Metaverse user envisions sending SR to a trusted smart brick while mistakenly directing the request to a fake brick could disrupt the experience, leading to disconnection from the concert, highlighting the crucial role of trust, particularly in scenarios like remote surgery.






















§ PROPOSED ARCHITECTURE








The proposed architecture contributes to service provisioning by providing an efficient SaRD foundation, harnessing 6G's future benefits to offer practical solutions to the issues at hand detailed in this section.

Besides, guidelines for ever-fluctuating SIs and users on how to register their offerings and access their desired services are provided.









 §.§ Elements

In the rich mosaic 6G SaRD architecture, the elements unveil a landscape of actors shaping discovery's functionality, including users, service providers, and infrastructure resource nodes (Fig. <ref>). 

The architecture actors could fluidly transition between different roles. In our Metaverse scenario, for example, a VR headset serves both as a user to request the concert video and as an instance of a service provider to provide real-time data analysis services.



  * Users refer to both tangible devices or entities seeking to make use of services. This includes familiar devices like smartphones and extends to immersive technologies like HMDs, neurotech interfaces, and smart contact lenses, all poised to redefine interactions in the Metaverse.



  * Service Providers are those elements responsible for providing services. It is up to them to define software components, data models, function chaining maps, and other key parameters based on service description standards. 
They offer their services in physical or virtual–using Virtual Network Functions (VNFs)–instances based on their predefined metrics like cost.


Services can also be composed to offer more complex solutions that aim at fulfilling user wishes that exceed the capabilities of individual atomic services.



  * Infrastructure Resource Nodes form the network's foundation, including cloud nodes, extending their reach as infrastructure pillars with networking, storage, computing, and computation capabilities, and edge nodes operate closely to users, enabling low-latency interactions. The continuum includes resource-rich devices like gNodeBs and Metaverse cloud hubs, as well as resource-poor nodes like drones and high-altitude platforms, each contributing distinct capabilities. Network core resources manage critical operations while PoAs connect users to the network. VNF-equipped infrastructure resource nodes aid in networking operations, deploying SIs, and streamlining service provisioning.





























 §.§ SaRD Components

The fundamental building blocks of the proposed SaRD architecture dictate how elements contribute to orchestrating services, enabling discovery, access, and delivery of requested services. 

Fig. <ref> categorizes components into two parts, each aligned with a specific facet of the SaRD process: Orchestration and Platform.
























  §.§.§ Orchestration
  

The Orchestration components are dedicated to decision-making operations. 

Using these components, informed decision processes can be navigated, with scalability, determinism, and trust issues being addressed, adapting to different network conditions, and gaining access to the required information provided by the Platform components.






SaRD processes hinge on how various elements interface with the network. Upon network entry, they must acquaint themselves with established standards and essential communication protocols such as learning how to describe services. 

The Bootstrapper component launches this process, which enables automatic request submission and interaction through the network, adapting to different device types and user needs. Within SaRD, two request types exist: Advertising Requests (AdRes) and SRs. AdRes are initiated by service providers who seek to promote their services and register their SIs - multiple SIs may offer the same services. Conversely, users issue SRs to access the services they require. To differentiate between these two, SRs are managed by the Requesting Manager, whereas AdRes are handled by the Advertising Manager. In addition, keeping track of both SRs' and SIs' historical records contributes greatly to discovery processes' intelligent decision-making. To this end, the Indexing Manager orchestrates SRs, offered services as well as SIs, service semantic Knowledge Bases (KBs), and resource indexing procedures, utilizing different metrics - such as accuracy, consistency, reliability - to optimize the process. Efficient SaRD operation requires a clear, autonomous interface and a well-functioning indexing manager to maintain real-time knowledge of the network's state.





When SRs are initiated, the process of selecting suitable SIs to fulfill these requests becomes pivotal. The Service Selection component, possibly with Service Composition, identifies appropriate SI(s) for requests on their path to reach their destinations. The decision requires three sources of information obtained from the Maintenance and Intelligence Hub components: the roster of available SIs, real-time SI data, and future predictions. Continuous monitoring by Maintenance is required to detect SI characteristics and changes such as load fluctuations, mobility, and disconnects, adapting to real-time network conditions. 

For future-oriented predictions, Intelligence Hub components predictive analysis and estimation of SIs' availability are leveraged. The decision on which SIs to select is guided by an array of metrics such as the locations of users and SIs, the proximity of SIs to users, the type of service required, current resource utilization levels, energy consumption patterns, and the specific QoS/QoE requirements of the SRs. 






In our SaRD architecture, SI selection aligns with users' requirements, ensuring chosen SIs not only meet but also uphold these standards throughout the service delivery process. In this regard, QoS/QoE Measurement component investigates the metrics that affect QoS/QoE through the network, providing a solid foundation for the selection process. User preferences for informed decisions provided by Maintenance components can impact SI selection. For instance, the Energy Management tracks nodes' power usage (battery and consumption rates) and energy harvesting methods. By allowing our Metaverse users to tap into a variety of energy supply sources based on their preferences, we enabled a knowledgeable and adaptive approach. Moreover, resources and SIs' trustworthiness must be prioritized. The Trust component considers the establishment of trust mechanisms using blockchain and defines such trust considerations as the protocols for different entities interacting with smart contracts, the management of these contracts, and the implications of split zoning on trust dynamics. By integrating these considerations, users are empowered to choose SIs that not only adapt to their priorities but are also reliable.
























Optimal SI selection and predictive discovery are continuous processes that unfold along the user's path to accessing the required services. The dispersion of users and SIs across the expansive edge-cloud-continuum makes it increasingly effective for segmenting the network into distinct zones to meet the scalability issue. This approach allows users to efficiently access their desired services and connect with appropriate SIs within a specific zone. DOZ supports network partitioning while streamlining proper service delivery by grouping requesters and targeting SIs within the same zones. Using network inputs - like real-time feedback, network load, and so forth. - and predictive insights like SI availability predictions, node locations, and SRs prediction, optimal zones are determined. Intelligence Hub estimates rezoning times based on user and SI mobility, adapting to dynamic network conditions. DOZ is especially pertinent in scenarios like our Metaverse user experience, where locating an SI globally is impractical. Alternatively, through DOZ, we can investigate the most relevant SIs within the user's zone that significantly contribute to the SaRD approach's scalability.








Several intelligent SaRD processes rely on predictions controlled by Intelligence Hub components.

Based on Graph Neural Networks (GNNs) and Deep Reinforcement Learning (DRL) algorithm, Movement Prediction anticipates user and SI location predictions, integrating node features such as location, velocity, acceleration, direction, and energy data. Our architectures employ strategies such as as effective sampling, dynamic updates, parallel processing, and hierarchical representations to ensure scalability and handle large-scale graphs in dynamic environments. Availability Prediction predicts SI availability in the upcoming timeframe to avoid dispatching SRs to unavailable or soon-to-be-recharging SIs. These components continually adapt to changing user behaviors and external factors through Continual Learning, addressing catastrophic forgetting. Further, dynamic zoning coupled with distributed users and SIs emphasizes the need for Federated Learning, resulting in shared learning models within these spatial-temporal contexts. All in all, the forward-thinking approach of Intelligence Hub components is crucial for maintaining service continuity under dynamic future environments.

































  §.§.§ Platform
  

Platform components collectively form the underlying foundation required to make informed decisions through the orchestration components. Each of its components provides a dedicated platform that collectively constitutes the bedrock upon which the entire architecture stands and gathers the necessary information for the execution of SaRD functions. 










To fully enable intelligent decision-making as well as enhance the contextual relevance and efficiency of data transmission in the context of SaRD, Semantic components must be incorporated into a holistic platform. The need for semantic awareness arises from the existence of different SRs with varied modalities and fixed meanings, enabling our architecture to handle varied modalities. Hence, merely preserving historical data without semantics proves insufficient in identifying relevant SRs. Semantic communication in our architecture involves the exchange of SR in a meaningful and context-aware manner, where a KB is considered a structured repository that stores semantic information. Semantic Transmitter and Semantic Receiver components work in tandem to extract knowledge from requests and perform the reverse operation, using semantic fusion methods <cit.>. The extracted information will be sent as a SR which leads to reduced data transmission as well. The Semantic components include detailed KBs, attributes, and semantic annotations, enriching SaRD with accurate, context-aware information. These KBs need to be maintained - within a dedicated KB Chain - to facilitate decoding and encoding requests which are continuously updated by KB Updater to align with evolving semantic features <cit.>. With up-to-date KBs, Thus, Semantic platform contributes significantly to exchanging information and enabling SaRD to its full potential.





As previously highlighted, to enable intelligent decisions in Intelligence Hub components, preserving historical SR data and exploring the trustworthiness of users and SIs for informed service selection are crucial. The Blockchain components in the architecture ensure trust and data integrity through a multifaceted platform and a hybrid structure combining on-chain and off-chain processing to maintain and manage records efficiently, which is depicted in Fig. <ref>. Each zone has its own Zone Chain that retains critical local data such as user states, SI information, attributes, and service semantic KBs that facilitate zone-level SaRD operations. Our architecture employs Ledger Merge and Ledger Split components to facilitate real-time information exchange and adjustments within Zone Chains, ensuring consistent and relevant data after any zone change. On a broader scale, the centralized Main Chain offers a high-level, network-wide perspective tasked with summarizing KB statistics; metadata; and filtered, pruned, and aggregated data regarding network states for informed decision-making in DOZ. Alongside, trust is established by deploying blockchain smart contracts, which authenticate each SI and assure users of their reliability, ensuring compliance with established network trust protocols during all interactions and transactions, while dynamically adapting to changes in trust requirements.

Utilizing Blockchain in SaRD introduces challenges such as transaction processing delays and performance bottlenecks due to frequent updates, coupled with inherent latency in transaction handling. To tackle these limitations, our architecture integrates a fast finality mechanism, which accelerates block confirmation; an optimistic roll-up for expedited transaction validation; and state channels while aggregating multiple updates into single transactions. State channels allow direct communication among participants while reducing consensus and blockchain interactions for each update, ensuring adaptability to varying transaction volumes and network conditions. 





























To establish connections between users and SIs while facilitating SRs and AdRes transmission, a solid networking platform becomes imperative. The incorporation of zoning strategies and semantics introduces novel dimensions to networking, departing from traditional approaches. Semantic-aware routing elevates network operations by introducing an intelligent layer that takes into account contextual information about services and resources when making routing decisions, adapting to dynamic network states. 



SaRD's Networking components are imperative in the 6G environment, which uses Network Function Virtualization (NFV) to host virtual SIs, ensuring the delivery of requests to its end with stringent requirements.

































    
    

    


    
    


    



















 §.§ Discovery Workflows


The SaRD workflow identifies systematic stages for user, service provider, and infrastructure node engagement in SaRD, outlining step-by-step procedures, communication, and interactions.










As depicted in Fig. <ref>, the SR process begins with users sending SRs through PoAs. The semantics-related features of the request are extracted by the Semantic Transmitter component, operating at the PoA. To determine if the requested service already exists in its cache, the PoA performs a cache check. If found within an acceptable aging window, it furnishes the response to the user. Otherwise, the Request Prediction component uses historical data to forecast the user's direction. Accordingly, the PoA propagates the user's intent–extracted features–as a multicast request to the user's targeted location zone. The Service Selection component chooses SIs based on the user's preferences, required QoS/QoE, and SI trustworthiness, with prioritized requests being resolved first. The SR, or a bundle of pending SRs, will be added to the SR history on the Zone and Main chains. Simultaneously, the response will be forwarded to the PoA, including the selected SI availability and its transition predictions, driven by the Availability Prediction and Movement Prediction components. The PoA's Semantic Receiver performs the inverse operation of the Semantic Transmitter. It promptly delivers responses to the user who initiated the SR, preserving semantic features and relevant data in its cache, subject to aging information. PoAs also communicate with other PoAs for service continuity, forwarding SRs on behalf of users to predicted PoAs along their paths. For SeCo, PoAs share information and send requests for required (atomic) services in advance, ensuring the timely existence of requested instances before users reach their future PoAs.











When a service provider wishes to register, modify, or deregister their instances, they initiate the AdRes process through PoAs using proper service descriptions (Fig. <ref>). Following the extraction of features, the PoA transmits service information, including instance details. Upon incorporating the offered SI or its modifications into the zone chain, the Maintenance components initiate the monitoring process to assess its availability and predictive behavior or cease observation if required. The PoA acknowledges the AdRe's success by confirming it with the service provider.









    










 





















§ PERFORMANCE EVALUATION
















The effectiveness of our SaRD architecture is quantitatively evaluated within a simulated edge-cloud-integrated 6G network considering real and diverse SRs.

The architecture accommodates various PoAs across different zones, mimicking the edge-cloud-continuum, and handles SRs with different modalities leveraging prediction algorithms. SaRD's performance is evaluated with varying numbers of nodes and SRs, where not only do individuals frequently change locations but also their data rates are varyingly adjusted to reflect the required dynamism. The evaluation assesses key metrics such as node, SI, and link costs, as well as SRs response time, providing a detailed understanding of our architecture's performance in a dynamic setting.
Simulation parameters, described in Table <ref>, offer flexibility to accommodate architectural elements.































The proposed architecture partitions the network based on real-time data and prediction info, ensuring demand fluctuations do not adversely affect the architecture. Blockchain safeguards SIs' trustworthiness with main and zone chains during the selection process. Additionally, semantic encoding/decoding is explored in the PoAs in which extracted features of requests are utilized as transmitted requests - beyond their modalities - for discovery using KB chain, enhancing prediction accuracy by tracking relevant SR history. Our implemented DRL algorithms continually predict each node's availability and each PoA's SRs, optimizing resource allocation to meet stringent QoS/QoE requirements and maintain service continuity in the dynamic environment.





Our architecture is benchmarked against alternative SaRD methods, including optimal SI and node assignments, random selection, the approach in <cit.> for Connected and Cooperative Autonomous Mobility (CCAM), and SDMS for IoT ad hoc networks <cit.>. Results in Fig. <ref> highlight response time and cost as performance metrics. While the optimal method excels at ideal SR assignment due to its omniscient knowledge of actual requests, its complexity–considering all combinations of SRs and resource allocations on different nodes–makes it impractical for timely solutions. Random selection is simple, yet results in high costs and a high rate of unsupported requests, leading to poor response times. The CCAM method performs better in terms of cost in networks with few nodes because of their inability to support all requests effectively, especially as the number of SRs and nodes increases. Unlike CCAM, which struggles with scalability and cannot serve single services on a single node, and SDMS, which focuses on service availability and overlooks allocation costs, our method, with its simplicity, scalability, and context-awareness, achieves significant efficiency gains by predicting future SRs, resulting in better allocation and enhanced response times. SR continuity is achieved by intelligently accommodating fluctuating SR volumes over time, providing the adaptability needed in a dynamic environment. Besides, our approach's zoning methodology demonstrates necessary scalability, maintaining high SR acceptance rates even as request numbers and data rates rise. This resilience is notable given the dynamic nature of SIs and users, including their mobility and subsequent zone transitions. 













§ FUTURE RESEARCH DIRECTIONS


The proposed research directions for the future of SaRD focus on three key domains: semantic-aware; scalable; and predictive discovery, each offering unique opportunities to shape the future landscape.


Future research should examine semantic-aware SaRD's efficiency and performance. Ensuring semantic data sharing privacy, optimizing semantic fusion, and investigating energy-efficient methods for semantic processing are vital. In the context of a dynamic semantic environment, synchronization, self-healing, and semantic reconciliation techniques become crucial to be explored. 

Further considerations could include energy-saving methods for semantic training and using resource-rich nodes for this step. 



To enhance the scalability and service continuity of SaRD, future research should delve into DOZ and address blockchain-related challenges. Exploring interoperable smart contracts and employing strategies for efficient ledger updates following zone changes, as discussed in <cit.>, could minimize zoning intervals, which would ultimately translate to improved service continuity. Also, dynamically adjusting chain block size could prevent a performance bottleneck. It would be a priority to optimize off-chain operations, considering overhead and utility. Additionally, a balance between maintaining historical chain data for accurate prediction and addressing storage concerns must be struck. 

Lastly, in a trustless environment, preventing unauthorized access and privilege escalation during bootstrapping becomes paramount for ensuring the integrity of the architecture.

For the goal of more suitable predictive discovery, the challenges posed by federated learning, such as heterogeneity of data, domain shifts, model stability, and the interplay between continual and federated learning must be addressed. In federated learning, privacy-preserving methods should be of paramount importance, safeguarding exchange models during data exchange. The incorporation of continual and federated learning with GNNs introduces complexities related to model stability, optimization, and complexity that require investigation to enhance prediction accuracy. Meta-learning techniques should also be explored to improve SaRD learning capabilities. 




































































§ CONCLUSION


In a world on the brink of the 6G revolution, the service and resource discovery architecture proposed herein emerges as a linchpin for future service provisioning in a highly dynamic environment. As a result of the architecture's orchestration, blockchain-backed trust, semantic networking, and pioneering predictive capabilities, powered by graph neural networks and deep reinforcement learning, it is positioned at the forefront of innovation in the discovery domain. The blockchain infrastructure, encompassing different zone chains and a main chain, delivers an immutable record that aids the learning algorithm in performing intelligent predictive discovery while accommodating dynamic zone changes. Crucially, the incorporation of dynamic overlay zoning imparts not only resource optimization and latency reduction but also scalability, adaptability, and resilience—essentials for a future defined by evolving service demands. Our learning algorithms predict user and service instance movement patterns and anticipate service requests using continual learning and privacy-preserving federated learning across different zones to cope with futuristic dynamicity. Beyond its immediate applications, it lays the groundwork for a more responsive, efficient, and reliable 6G edge-cloud-continuum that preserves service continuity in the future. 









§ ACKNOWLEDGMENT

Prof. Tarik Taleb and JaeSeung Song are co-corresponding
authors of this work. This work was partially conducted at ICTFICIAL Oy. It is partially supported by the European Union’s HE research and innovation program HORIZON-JUSNS-2023 under the 6G-Path project (Grant No.
101139172). It is also partially supported by the Business Finland 6Bridge 6Core project under Grant No. 8410/31/2022. The paper reflects only the authors’ views, and the European Commission bears no responsibility for any utilization of the information contained herein.












unsrt


Mohammad Farhoudi (Student Member, IEEE) received his Master's degree in Computer Networks from the Amirkabir University of Technology, Tehran, Iran, in 2017. He is currently pursuing a Ph.D. at the Centre for Wireless Communications, University of Oulu, Finland. He has served as a Senior Research Assistant at the Amirkabir University of Technology, working on a national project focused on utilizing SDN and NFV in large-scale data and transport networks. Additionally, he has over eight years of professional experience with Kilid Gostar Bina, Saba System Sadra, and Fava Pars Company, specializing in computational networking and computer security. His research interests include semantic-aware service orchestration, edge-cloud networks, and AI/ML mechanisms for service discovery and resource allocation.


Masoud Shokrnezhad
received his Ph.D. degree (recognized as a bright talent) in computer networks from Amirkabir University of Technology (Tehran Polytechnic), Tehran, Iran, in 2019. He is currently a postdoctoral researcher with the Center of Wireless Communications at the University of Oulu, Finland. Between June 2021 and December 2021, he was a postdoctoral researcher at the School of Electrical Engineering, Aalto University, Espoo, Finland. Prior to that, he worked as a senior system designer and engineer with FavaPars and Pouya Cloud Technology in Tehran, Iran, since 2013. Throughout his career, Dr. Shokrnezhad has been involved in numerous national, international, and European projects focused on designing and developing computing and networking frameworks. He has also co-managed a startup that develops SDWAN solutions for B2B use cases.


Tarik Taleb
is currently a Full Professor at Ruhr University Bochum, Germany. He was a Professor with the Center of Wireless Communications (CWC), University of Oulu, Oulu, Finland. He is the founder of ICTFICIAL Oy, and the founder and the Director of the MOSA!C Lab, Espoo, Finland. From October 2014 to December 2021, he was an Associate Professor with the School of Electrical Engineering, Aalto University, Espoo, Finland. Prior to that, he was working as a Senior Researcher and a 3GPP Standards Expert with NEC Europe Ltd., Heidelberg, Germany. Before joining NEC and till March 2009, he worked as an Assistant Professor with the Graduate School of Information Sciences, Tohoku University, in a lab fully funded by KDDI. From 2005 to 2006, he was a Research Fellow with the Intelligent Cosmos Research Institute, Sendai. He received the B.E. degree (with distinction) in information engineering and the M.Sc. and Ph.D. degrees in information sciences from Tohoku University, Sendai, Japan, in 2001, 2003, and 2005, respectively.


Richard Li
is Chair Professor of Network Technologies, School of Computer Science and Engineering, Southeast University, Nanjing, China. Prior to joining Southeast University in April 2024, Richard worked with Futurewei Technologies, aka Huawei R&D USA, from 2007 to 2024 as Chief Scientist, SVP, and Head of Network Technologies Lab in the San Francisco Bay Area, USA. Before that, Richard worked with Cisco and Ericsson on their networking products, technologies, solutions and network operating systems. Richard also served as the Chairman of the ITU-T FG Network 2030 from 2018 to 2020, the Vice Chairman of the Europe ETSI ISG Next-Generation Protocols from 2016 to 2019, and Chairs of steering committees and technical program committees of some academic and industrial conferences. Currently, he serves on the advisory board for IEEE IoT Journal, as a technical editor for IEEE Network Magazine, and as guest editors for special issues of some journals and magazines.


Jaesung Song
is a full professor in the Department of Computer & Information Security Sejong University. He holds the position of Technical Plenary Vice Chair of the oneM2M global IoT standards initiative. Prior to his current position, he worked for NEC Europe Ltd. and LG Electronics in various positions. He received a Ph.D. at Imperial College London in the Department of Computing, United Kingdom. He holds B.S. and M.S. degrees in computer science from Sogang University. His research interests span the areas of beyond 5G and 6G, AI/ML enabled network systems, software engineering, networked systems and security, with focus on the design and engineering of reliable and intelligent IoT/M2M platforms.





 Paper Title: Mobility-Aware Federated Self-supervised Learning in Vehicular Network.



Article Title]Mobility-Aware Federated Self-supervised Learning in Vehicular Network










1]Xueying Guxueyinggu@stu.jiangnan.edu.cn


[1,2]Qiong Wuqiongwu@jiangnan.edu.cn


3]Pingyi Fanfpy@tsinghua.edu.cn


4]Qiang Fanqf9898@gmail.com




[1]School of Internet of Things Engineering, Jiangnan University, Wuxi, 214122,  China

[2]State Key Laboratory of Integrated Services Networks, Xidian University, Xi'an , 710071, China


[3]Department of Electronic Engineering, Beijing
		National Research Center for Information Science and Technology, Tsinghua
		University, Beijing, 100084, China

[4]Qualcomm, San Jose CA, 95110, USA




Federated Learning (FL) is an advanced distributed machine learning approach, that protects the privacy of each vehicle by allowing the model to be trained on multiple devices simultaneously without the need to upload all data to a road side unit (RSU). This enables FL to handle scenarios with sensitive or widely distributed data. However, in these fields, it is well known that the labeling costs can be a significant expense, and models relying on labels are not suitable for these rapidly evolving fields especially in vehicular networks, or mobile internet of things (MIoT), where new data emerges constantly. To handle this issue, the self-supervised learning paves the way for training without labels. Additionally, for vehicles with high velocity, owing to blurred images, simple aggregation not only impacts the accuracy of the aggregated model but also reduces the convergence speed of FL. This paper proposes a FL algorithm based on image blur level to aggregation, called FLSimCo, which does not require labels and serves as a pre-training stage for self-supervised learning in the vehicular environment. Simulation results demonstrate that the proposed algorithm exhibits fast and stable convergence.



















[
    [
    Accepted for publication in Astrophysics & Space Sciences (ApSS)
====================================================================





§ INTRODUCTION


The development of Internet of Things (IoT) makes many practical applications available to people, such as automatic navigation, weather forecast and self-driving systems, which not only reduces the probability of accidents, but also improves the happiness of life <cit.>. Training a good model can make the practical application more robust, which requires a lot of data. Moving vehicles can constantly collect new data by their devices, most of which can be captured in the form of images via camera mounted on the roof of the vehicle. Compared with the traditional method of collecting image data with onboard radar, this way greatly reduces the cost and technical requirements, making the shift of image collection paradigm to vision-based measurement (VBM) has broad promising <cit.>.

After the acquisition is completed, the vehicle process the image data to realize image recognition and classification. It provides the necessary information for self-driving and driver assistance systems <cit.>, and helps drivers perceive and understand their surroundings. Especially in nighttime or bad weather conditions, image recognition and classification models can be critical valuable in identifying road signs, other vehicles, and obstacles, providing specific object classification, improving driver visibility, and improving driving safety. Therefore, image recognition and classification in self-driving plays a key role in reducing the risk of traffic accidents and improving road safety.

As mentioned earlier, training a robust model requires a lot of data. However, the data stored locally on each vehicle may be private and sensitive, and drivers are reluctant to share it with each other in order to protect their privacy. If the vehicles do not share information, they can only train the local model using their own data <cit.>. Due to the single driving environment, the image categories will be skewed, so that the trained model does not have good transference. Compared to single vehicle training, FL can solve this issue by aggregating different vehicles' local trained models with local data. This approach can take a large data set into account in the training process without the need for vehicles to share local data, resulting in superior performance and enhanced generalization <cit.>.

However, this approach still faces the following two issues: low image quality and missing or incorrect labels. The first one is due to the motion blur caused by the movement of vehicles, and the other is because of the high cost and incorrectness of labeling.

In a vehicular network, a high velocity usually leads to insufficient exposure time for the camera sensor, causing motion blur. In this paper, we consider the motion blur caused by different vehicle velocities in the training process, and adjust corresponding weights of the model parameters for FL model aggregation. 

Self-supervised learning <cit.> can abandon labels for pre-training, and thus mitigates the impact of incorrect labels on the model and removes the cost of the labeling process, which makes it suitable for IoT, where new data is constantly appearing.

To the best of our knowledge, few research works have taken into account both the privacy protection of vehicles and the blurring of images in real scenes, as well as the cost of labels during the training of models.

In this paper, we build a framework, called FLSimco[The source code has been released at: https://github.com/qiongwu86/FLSimCo], to train the vehicle side local model by self-supervised learning at dual temperatures. Only upload the local models of the vehicles, after the completion of local training, and then  the RSU through the FL aggregated the received local models, both to protect the privacy of the vehicle, and training a generation model.

The remaining of this paper is organized as follows. In Section <ref>, we will review related works. The Section <ref> details the system model we designed. Section <ref> will showcase and discuss the experimental results obtained in the simulation environment. Finally, we will summarize the research findings and present conclusions in the Section <ref>.




§ REVIEW OF RELATED WORKS


In recent years, the emergence of deep neural networks, particularly convolutional neural network (CNN), has facilitated significant advances in computer vision benchmarks. Self-supervised learning is a special form of unsupervised learning, and its main feature is that it uses information inherent in the data itself to help machine learning models get a better representation, thereby improving performance across a variety of tasks<cit.>.

Until now, research on unsupervised learning has focused on mining shared features between pre-trained tasks and downstream tasks <cit.>. Wu et al. <cit.> introduced noise contrast estimation (NCE) loss as an objective function to distinguish between different instances. Each image was treated as a positive sample and the others as negative samples, effectively treating each image as a category, constituting an instance-level classification task. SimCLR <cit.> has maximized the sample characteristics of similarity to study representations. It abandoned the memory library <cit.> and generated more negative samples by increasing the batch size. The core idea of these methods is to encourage models to place similar data representations(positive sample pairs) together in the embedding space, while separating dissimilar representations(negative sample pairs) to learn feature extraction.
Since these methods do not consider the fact that the distance between samples of the same category should be smaller than that of different categories' samples, but treat all pairs equally. In this way, a large number of negative samples are required. It will pose significant challenges to storage and computing power in the vehicle environment, increasing the requirements of hardware configuration levels.

MoCo <cit.>, as a self-supervised learning method for CNN <cit.>, has made an important contribution by introducing queue and momentum updating techniques to create a large and consistent dictionary conducive to contrast learning. It replaced the original memory library with a queue as an additional data structure to store these negative samples. Momentum encoder k is used to replace the original loss constraint term. The key advantage of the MoCo family is the use of the queue as negative samples and the momentum update of the queue, which greatly reduces the need for vehicle storage capacity and computing power in local training. <cit.> used MoCo for the pre-training model weights on ImageNet and achieves excellent results in downstream fine-tuning tasks. <cit.> employed MoCo for unsupervised learning with large amounts of unlabeled data on remote servers. The local vehicles downloaded the trained model and used it as the initialization model.


These methods are characterized by the presence of a large default datasets that can be used for unsupervised learning, and the recognition accuracy increases with the expansion of the dictionary size<cit.>. Simultaneously, as the dictionary size increases, the recognition accuracy gradually improves. However, it is important to note that, larger dictionaries also requires larger memory size, increasing the storage cost and the computing capability of hardness. SimCo <cit.> attributed the need for a large dictionary in MoCo to hardness awareness between anchors, and the author believed that the consistency between positive and negative keys was more crucial than that between negative keys. Therefore, it utilized dual temperatures to differentiate inter-anchor samples and intra-anchor samples, as shown in Fig. <ref>, which eliminates the need for a dictionary of negative samples.

Due to the imbalance of data categories, the data distribution faces the challenge of non-independent identically distributed (non-IID) characteristics <cit.>, requiring more training rounds and interaction frequency to improve accuracy. However, increased interaction means require more computing and network resources <cit.>. To protect the privacy of local data and its distribution, FedCo proposed to employ FL to aggregate local models. Each vehicle passed its own k value to the RSU for forming a bigger new queue. However, a new queue consisting of the k values of different vehicles violates MoCo's inherent requirement for consistency in negative key pairs. When the vehicle uploaded the trained model and the corresponding k value, it can, to some extent, reconstruct the original input. As such, it does not ensure privacy protection and defeats the original purpose of using FL. <cit.> used federated self-supervised learning for a single-event classification, adding a binary classifier for each new event by adhering to a one-to-many paradigm. However, these FL algorithms do not take into account the effect of image blurring. As a result, they cannot effectively simulate real-world scenarios.

In this paper, we will train the local model on the vehicles side by self-supervised learning with dual temperatures. After the training is completed, only the local models of vehicles are uploaded, and then the received local model is aggregated in RSU by  FL, which not only protects the privacy of vehicles, but also produces a model with better transform through distributed training. At the same time, considering that the images collected in the actual vehicle environment may be blur, the blur level is used as the weight during aggregation, which makes the aggregated model more reliable and stable.







§ SYSTEM MODEL




 §.§ System design



As shown in Fig. <ref>, let's consider a scenario where an RSU is deployed at the intersection and there are several vehicles driving in coverage of the RSU. Each vehicle driving straightly when cross the intersection. We firstly build a truncated Gaussian distribution model as the mobility model for each vehicle. At the same time, each vehicle is equipped with a large mount of images, called local data, captured by camera mounted in the roof before entering the coverage of the RSU. It is worth noting that vehicles coming from different directions may capture different categories of images from each other.

In each FL round,  each vehicle downloads the global model from the RSU as it enters the coverage of the RSU. Then each vehicle adopts the global model for the pre-training of the self-supervised learning to classify each category of captured local image data. During this process, the vehicle randomly selects a specified number of images from its local data <cit.>. When the local training is complete, the vehicle uploads its local model to the RSU. Then, the RSU  aggregates the local models from different vehicles to obtain a new global model. After that, FL moves on to the next round until the maximum round R^max is reached. Next, we will describe the models for each round in the system.



 §.§ Mobility model

Similar with <cit.>, we adopt the following mobility model to reflect the real vehicle mobility. We consider the velocities of different vehicles are independent identically distributed (IID). The velocity of each vehicle follows the truncated Gaussian distribution. Let N_r be the number of vehicles traveling within the coverage area of RSU in the round r, v_n_r be the velocity of vehicle n_r∈[1, N_r], and v_min and v_max be the minimum and maximum velocity of vehicles, respectively, namely velocity v_n_r∈[v_min,  v_max], and the probability density function of v_n_r is expressed as  <cit.>

    f(v_n_r) = {e^ - 1/2σ ^2(v_n_r - μ )^2/√(2πσ ^2) [erf(v_max - μ/σ√(2)) - erf(v_min - μ/σ√(2))]
    v_min≤v_n_r≤v_max,
    
    		0                    otherwise.
    	.
where erf(μ, σ^2) is the Gaussian error function of velocity v_n_r with mean μ and variance σ^2. 




 §.§ FLSimCo algorithm

Self-supervised learning requires a large amount of unlabeled data. In the vehicle scene, it is easy for the vehicles to collect enough image data during driving. However, due to the limitations of the environment, storage and camera perspective, the categories of image data of each vehicle are limited, which may not meet the IID <cit.> requirement. To be specific, when the vehicle uses the image data stored by itself for local training, a model shewed to its own image categories is obtained, namely a model with poor generation. After the training is completed, the vehicle uploads the parameters of local model, and the RSU aggregates the received local model parameters from different vehicles, making the aggregated model more general and effectively solving the data non-IID problem.

We establish a novel algorithm called FLSimCo, which means simplified MoCo with no momentum in FL. Typically, the specific FLSimCo process is as follows:

Step 1, initialization: The RSU initializes a global model θ^0 for an encoder f^0 with random initial parameters.

Step 2, local training: For round r, N_r vehicles take part in the FL, where each vehicle n_r downloads the model θ^r of encoder f^r from the RSU. 

The blur level L_n_r of local image data in the vehicle n_r can be represented as <cit.>:

    L_n_r=Hs/Qv_n_r.

In this context, v_n_r is the velocity of vehicle n_r. Hs/Q is the parameter of the camera, in which H represents the exposure time interval, s is the focal length, and Q represents pixel units.

For each image x_i∈{x_1, x_2, … x_i,… x_M}, after applying two different data augmentation methods π_1(·) and π_2(·),  and passing through the encoder f^r, we obtain anchor sample q_n_r^i, positive sample k_n_r^i and negative samples k_n_r^j:

    q_n_r^i=f^r[π_1(x_i)]=ResNet[π_1(x_i)]|_r, i∈[1,M],


    k_n_r^i=f^r[π_2(x_i)]=ResNet[π_2(x_i)]|_r, i∈[1,M],


    k_n_r^j=f^r(x_j)=ResNet(x_j)|_r, j∈[1,M] and j≠ i.


According to <cit.>, the loss can be transformed into


    ℒ_q_n_r^i^DT=   -sg[(W_β)_n_r^i/(W_α)_n_r^i]×
       logexp(q_n_r^i· k_n_r^i/τ_α)/exp(q_n_r^i· k_n_r^i/τ_α)+∑_j=1^Kexp(q_n_r^i· k_n_r^j/τ_α),

and

    (W_β)_n_r^i=1-exp(q_n_r^i· k_n_r^i/τ_β)/exp(q_n_r^i· k_n_r^i/τ_β)+∑_j=1^Kexp(q_n_r^i· k_n_r^j/τ_β),


    (W_α)_n_r^i=1-exp(q_n_r^i· k_n_r^i/τ_α)/exp(q_n_r^i· k_n_r^i/τ_α)+∑_j=1^Kexp(q_n_r^i· k_n_r^j/τ_α),
where ℒ_q_n_r^i^DT is the dual-temperature (DT) loss of the i-th image of the vehicle's anchor sample in the round r. K represents the queue length and sg[·] indicates the stop gradient. The denominator in the above equations consists of one positive sample and K negative samples. It is noting that '·' in Eq. (<ref>) - Eq.  (<ref>) means dot product. τ_α and τ_β are different temperature hyper-parameters <cit.>, and controls the shape of the samples distribution.  Based on the different requirements for dictionary size by inter-anchor and intra-anchor, and the temperature's ability to control the feature distribution, different temperatures will be used to control the distance between different samples, thus eliminating MoCo's dependency on a large dictionary. This will remove the inter-anchor's reliance on a large dictionary. 

The objective function can be defined as minimizing the loss function <cit.>

    θ̂_n_r=θ_n_r argmin1/M∑_i=1^Mℒ_q_n_r^i^DT(θ_n_r,  q_n_r^i, k_n_r^i, k_n_r^j),
where θ_n_r represents the parameter of local model of vehicle n_r in round r. Each vehicle performs the local training to approach θ̂_n_r according to the following algorithm:

    θ_n_r θ_n_r-η^r∇ℒ^DT(θ_n_r, q_n_r^i, k_n_r^i,  k_n_r^j),
where η^r represents the learning rate for the  round r.

Step 3, Upload model: N_r vehicles upload the parameter {θ_1, θ_2…θ_n_r…θ_N_r} of local models after their local training finished, along with the velocity {v_1,v_2,… v_n_r… v_N_r} of each vehicle.

Step 4, Aggregation and Update: After receiving the trained models from N_r vehicles, the RSU employs a weighted federated algorithm to aggregate the parameters of N_r models based on the blur level L_n_r. The expression for the aggregated model is


    θ^r+1=∑_n_r=1^N_r[(∑_n_r=1^N_rL_n_r-L_n_r)θ_n_r/∑_n_r=1^N_r(∑_n_r=1^N_rL_n_r-L_n_r)].








Repeat the above step 2 to step 4 until reaching max round R^max.



§ RESULTS


In this section, we will introduce the setup of experiments, show the results, and give a brief explanation.



 §.§ Experimental setup


We adopt an improved ResNet-18 with a fixed dimension of 128-D as the backbone model and employ SGD (Stochastic Gradient Descent) as the optimizer. In addition, inspired by the concept of cosine annealing, we gradually reduce the learning rate at different stages of training to improve the training efficiency of the model <cit.>.

Datasets: This paper uses the datasets CIFAR-10 <cit.> with 50,000 unlabeled images as the training datasets, which is distributed in 10 different categories, i.e., each category consists of 5,000 images.




(1) IID: Samples that follow IID are independent of each other and share the same distribution. The appearance of each sample is not affected by the other samples. We uniformly assigned 10 categories from 50,000 images in CIFAR-10 to 95 vehicles, ensuring that each vehicle have at least 520 images available for training. As can be seen from the Fig. <ref>, the categories on each vehicle are evenly distributed.

(2) Non-IID: IID provides theoretical convenience, but in a practical scenario, there is very little data that meets the IID requirement. Therefore, it is importance to use data under non-IID conditions to transfer the model to the real scenario. As shown in Fig. <ref> and <ref>, the Dirichlet distribution parameter α is 0.1 and 1, respectively <cit.>. Clearly, the smaller the α, the larger the gap between data categories. We set the Dirichlet distribution parameter α to 0.1 to simulate the Non-IID data in the vehicular scene, in order to simulate the uneven distribution of the  image categories collected by each vehicle due to the limited viewing perspective and environmental constraints. In order to ensure that there is enough data for local training, for CIFAR-10 we ensure that there are at least 520 images per vehicle.




 §.§ Results


According to FedCo <cit.>, in the round r of the training process, we set each vehicle uploads all stored k-values (with a batch size set to 512 in the experiment) to the RSU (global queue set to 4096) to update the global queue. 

As shown in Fig. <ref>, we consider the FedCo simulation result as baseline1. In addition, our proposed method FLSimCo is represented by a red line in the same diagram. It can be observed that FLSimCo outperforms FedCo given the same number of rounds. From FedCo's perspective, the update queues with k values from different vehicles has compromised the Negative-Negative consistency requirement in MoCo, resulting in a less accurate approach. Meanwhile, FedCo enables the vehicle's own k value to be uploaded to the RSU, which also goes against FL's purpose of protecting user privacy. In addition, we also conducted experiments on Non-IID data sets, and the results show that the training performance of Non-IID data sets is slightly lower than that of IID data sets, but still better than FedCo algorithm.

 

For Fig. <ref>, we assume that some images will be blurred when the vehicle velocity exceeds 100 km/h and conduct three comparative experiments.To better demonstrate its performance, we also introduce two baseline algorithm.  FedAvg <cit.> is employed as baseline1, which averaging the model parameters accordingly. Baseline2 indicates that RSU will discard the model trained by the vehicle velocity exceeding 100 km/h, that is, discard the local model trained with the blurred images, and then use the FedAvg aggregation model parameters.

From the results of baseline1 and baseline2, it is obvious that it is not beneficial to drop the model trained with motion blurred images(baseline2). Because it drops some models during the aggregation process, baseline2 has the lowest number of models used in the global training. This suggests that the number of models involved in aggregation is more critical than the quality of the models. Finally, our proposed aggregation method assigns smaller weights to models trained by faster vehicles. It can be seen that the proposed algorithm can effectively reduce the fluctuation of the loss function, while facilitating the loss function and converge to a smaller value faster. This demonstrated that proposed approach can reduce the effect of image blur and also increase the global model training speed.




§ CONCLUSION


In this paper, we propose a FLSimCo algorithm. First, on the basis of self-supervised learning, the advantages of FL are used. The main goal is to solve the problem of data category skew and to train a  model with well generation without sharing private data. By integrating self-supervised learning and FL, the algorithm can improve the model's adaptability to data heterogeneity to achieve a more robust pre-training model. More importantly, considering the data blur caused by vehicle movement, the blur level is used as the weight of federated aggregation, which makes the algorithm more suitable for real scenarios.

In summary, the FLSimCo algorithm ensures the protection of vehicle privacy, and requires no labeled image data, while addressing the aggregation challenges in vehicular environments characterized by significant data shew and image blurring due to vehicle movement. 



§ DECLARATIONS
 

Competing interests: The author declare that there is no competing interest.

Open Access: This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article?s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licen ses/by/4.0/.

00

wsy01
Wu Q, Wang S, Ge H, Fan P, Fan Q, Letaief K (2023) Delay-sensitive Task Offloading in Vehicular Fog Computing-Assisted Platoons. IEEE Transactions on Network and Service Management.https://doi.org/10.1109/TNSM.2023.3322881. 

ss01
Wu Q, Shi S, Wan Z, Fan Q, Fan P, Zhang C (2023) Towards V2I Age-aware Fairness Access: A DQN Based Intelligent Vehicular Node Training and Test Method. Chinese Journal of Electronics. pp. 1230-1244. 

ldx01
Long D, Wu Q, Fan Q, Fan P, Li Z, Fan J (2023) A Power Allocation Scheme for MIMO-NOMA and D2D Vehicular Edge Computing Based on Decentralized DRL. Sensors.



cmi2021
Mi C, Huang Y, Fu C, Zhang Z, Postolache O (2021) Vision-Based Measurement: Actualities and Developing Trends in Automated Container Terminals. IEEE Instrumentation and Measurement Magazine. pp. 65-76, https://doi.org/10.1109/MIM.2021.9448257.


zzhang2023
Zhang Z, Ma S, Yang Z, Xiong Z, Kang J, Wu Y, Zhang K, Niyato D (2023) Robust Semi-supervised Federated Learning for Images Automatic Recognition in Internet of Drones. IEEE internet of things journal. pp. 5733-5745.


dye2020
Ye D, Yu R, Pan M, Han Z (2020) Federated learning in vehicular edge computing: A selective model aggregation approach. IEEE Access. pp. 23920-23935, https://doi.org/10.1109/ACCESS.2020.2968399.


canagnostopoulos2023
Anagnostopoulos C, Gkillas A, Piperigkos N, Lalos S (2023)  Federated Deep Feature Extraction-based SLAM for Autonomous Vehicles. 2023 24th International Conference on Digital Signal Processing (DSP). https://doi.org/10.1109/DSP58604.2023.10167897.


yzhao2023
Wu Q, Zhao Y, Fan Q, Fan P, Wang J, and Zhang C (2023) Mobility-Aware Cooperative Caching in Vehicular Edge Computing Based on Asynchronous Federated and Deep Reinforcement Learning, IEEE Journal of Selected Topics in Signal Processing, vol. 17, no. 1, pp. 66-81.

sshi2023
Wu Q, Shi S, Wan Z, Fan Q, Fan P, and Zhang C (2023) Towards V2I Age-aware Fairness Access: A DQN Based Intelligent Vehicular Node Training and Test Method, Chinese Journal of Electronics, vol. 32, no. 6, pp. 1230-1244.






wxb01
Wu Q, Wang X, Fan Q, Fan P, Zhang C, Li Z (2023) High Stable and Accurate Vehicle Selection Scheme based on Federated Edge Learning in Vehicular Networks. China Communications. pp. 1-17. 

xsy01
Wu Q, Xia S, Fan Q, Li Z (2019) Performance Analysis of IEEE 802.11p for Continuous Backoff Freezing in IoV. Electronics. 


ryan2023
Yan R, Qu L, Wei Q, Huang S, Shen L, Rubin D, Xing L,  Zhou Y (2023) Label-Efficient Self-Supervised Federated Learning for Tackling Data Heterogeneity in Medical Imaging. IEEE Transactions on Medical Imaging. https://doi.org/10.1109/TMI.2022.3233574.

kwang2019
Wang K, Yu F, Wang L, Li J, Zhao N, Guan Q, Li B, and Wu Q (2019) Interference Alignment with Adaptive Power Allocation in Full-Duplex-Enabled Small Cell Networks, IEEE Transactions on Vehicular Technology, vol. 68, No. 3.

sxia2018
Wu Q, Xia S, Fan P, Fan Q, and Li Z (2018) Velocity-Adaptive V2I Fair Access Scheme Based on IEEE 802.11 DCF for Platooning Vehicles, Sensors, vol. 18, no. 12, Art no. 4198.

jfan2010
Fan J, Yin S, Wu Q, and Gao F (2010) Study on Refined Deployment of Wireless Mesh Sensor Network, in Proc. of IEEE International Conference on Wireless Communications, Networking and Mobile Computing (WICOM?10), Chengdu, China, pp. 370-375.



czhang2022dual temp
Zhang C, Zhang K, Pham T, Niu A, Qiao Z, Yoo C, Kweon I (2022) Dual temperature helps contrastive learning without many negative samples: towards understanding and simplifying MoCo. 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR).  https://doi.org/10.1109/CVPR52688.2022.01404.


swang2024
Wu Q, Wang S, Ge H, Fan P, Fan Q, and Letaief K. B. (2024) Delay-sensitive Task Offloading in Vehicular Fog Computing-Assisted Platoons, IEEE Transactions on Network and Service Management, vol. 21, no. 2, pp. 2012-2026. 

hliu2019
Wu Q, Liu H, Zhang C, Fan Q, Li Z, and Wang K (2019) Trajectory Protection Schemes Based on a Gravity Mobility Model in IoT, Electronics, vol. 8, no. 148.

snie2018
Wu Q, Nie S, Fan P, Liu H, Fan Q, and Li Z (2018) A Swarming Approach to Optimize the One-Hop Delay in Smart Driving Inter-Platoon Communications, Sensors, vol. 18, no. 10, Art. no. 3307.


jzheng2015
Wu Q, and Zheng J (2015) Performance Modeling and Analysis of the ADHOC MAC Protocol for VANETs, in Proc. of IEEE International Conference on Communication (ICC?15), London, UK, pp. 3646-3652.

jfan2010
Fan J, Wu Q, and Hao J (2010) Optimal Deployment of Wireless Mesh Sensor Networks based on Delaunay Triangulations, in Proc. of IEEE International Conference on Information, Networking and Automation (ICINA?10), Kunming, China, pp. 1-5. 

xwang2023
Wu Q, Wang X, Fan Q, Fan P, Zhang C, and Li Z (2023) High Stable and Accurate Vehicle Selection Scheme based on Federated Edge Learning in Vehicular Networks, China Communications, vol. 20, no. 3, pp. 1-17.






cdoersch2016
Doersch C, Gupta A, Efros A (2016) Unsupervised visual representation learning by context prediction. 2015 IEEE International Conference on Computer Vision (ICCV). https://doi.org/10.1109/ICCV.2015.167.

zwu
Wu Z, Xiong Y, Yu S, Lin D. Unsupervised Feature Learning via Non-Parametric Instance Discrimination. https://arxiv.org/pdf/1805.01978.pdf.


tchen2020SimCLR
Chen T, Kornblith S, Norouzi M, Hinton G (2020) A Simple Framework for Contrastive Learning of Visual Representations. https://arxiv.org/abs/2002.05709. 


skong2023
Kong S (2023) Self-supervised Image Classification Using Convolutional Neural Network. 2023 IEEE 3rd International Conference on Power, Electronics and Computer Applications (ICPECA). https://doi.org/10.1109/ICPECA56706.2023.10075949.


rhjelm
Hjelm R, Fedorov A, Laavoie-Marchildon S, Grewal K, Bachman P, Trischler A, Bengio Y. Learning deep representations by mutual information estimation and maximization. https://arxiv.org/pdf/1808.06670.pdf.


khe2020
He K, Fan H, Wu Y, Xie S, Girshick R (2020) Momentum contrast for unsupervised visual representation learning. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). https://doi.org/10.1109/CVPR42600.2020.00975.


xchen2020MoCoV2
Chen X, Fan H, Girshick R, He K (2020) Improved Baselines with Momentum Contrastive Learning. https://arxiv.org/pdf/2003.04297.pdf.

jzhao01
Zhao J, Xiong X, Zhang Q, Wang D (2023) Extended Multi-Component Gated Recurrent Graph Convolutional Network for Traffic Flow Prediction. IEEE Transactions on Intelligent Transportation Systems. https://doi.org/10.1109/TITS.2023.3322745.




tcai2022
Cai T, Gan H, Peng B, Huang Q, Zou Z (2022) Real-Time Classification of Disaster Images from Social Media with a Self-Supervised Learning Framework. IGARSS 2022 - 2022 IEEE International Geoscience and Remote Sensing Symposium. https://doi.org/10.1109/IGARSS46834.2022.9883129.


jzhao2021Hotfed
Zhao J, Li R, Wang H, Xu Z (2021) HotFed: Hot Start through Self-Supervised Learning in Federated Learning. 2021 IEEE 23rd Int Conf on High Performance Computing and Communications; 7th Int Conf on Data Science and Systems; 19th Int Conf on Smart City; 7th Int Conf on Dependability in Sensor, Cloud and Big Data Systems and Application. https://doi.org/10.1109/HPCC-DSS-SMARTCITY-DEPENDSYS53884.2021.00046.

jzheng2016
Wu Q, and Zheng J (2016) Performance Modeling and Analysis of the ADHOC MAC Protocol for Vehicular Networks, Wireless Networks, vol. 22, no. 3, pp. 799-812. 

jzheng2014
Wu Q, and Zheng J (2014) Performance Modeling of the IEEE 802.11p EDCA Mechanism for VANET, in Proc. of IEEE Global Communications Conference (Globecom?14), Austin, USA, pp.57-63.

dlong2023
Long D, Wu Q, Fan Q, Fan P, Li Z, and Fan J (2023) A Power Allocation Scheme for MIMO-NOMA and D2D Vehicular Edge Computing Based on Decentralized DRL, Sensors, vol. 23, no. 7, Art. no. 3449.


jzhao00
Zhao J, Li Q, Ma X, Yu FR. Computation Offloading for Edge Intelligence in Two-Tier Heterogeneous Networks. IEEE Transactions on Network Science and Engineering. https://doi.org/10.1109/TNSE.2023.3332949.



jzheng20141
Wu Q, and Zheng J (2014) Performance Modeling of IEEE 802.11 DCF Based Fair Channel Access for Vehicular-to-Roadside Communication in a Non-Saturated State, in Proc. of IEEE International Conference on Communication (ICC?14), Syndey, Australia, pp. 2575-2580.

sxia2019
Wu Q, Xia S, Fan Q, and Li Z (2019) Performance Analysis of IEEE 802.11p for Continuous Backoff Freezing in IoV, Electronics, vol. 8, no. 12, Art. no. 1404. 

wwang2024
Wu Q, Wang W, Fan P, Fan Q, Wang J, and Letaief K. B. (2024) URLLC-Awared Resource Allocation for Heterogeneous Vehicular Edge Computing, IEEE Transactions on Vehicular Technology, doi: 10.1109/TVT.2024.3370196.




jzhao02
Zhao J, Quan H, Xia M, Wang D (2023) Adaptive Resource Allocation for Mobile Edge Computing in Internet of Vehicles: A Deep Reinforcement Learning Approach.IEEE Transactions on Vehicular Technology. https://doi.org/10.1109/TVT.2023.3335663.


qwu2024
Wu Q, Wang W, Fan P, Fan Q, Wang J, Letaief K (2024) URLLC-Awared Resource Allocation for Heterogeneous Vehicular Edge Computing. IEEE Trans. Veh. Technol. https://doi.org/10.1109/TVT.2024.3370196.


mfeng2022
Feng M, Kao C, Tang Q, Sun M, Rozgic V, Matsoukas S, Wang C (2022) Federated self-supervised learning for acoustic event classification. ICASSP 2022 - 2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP). https://doi.org/10.1109/ICASSP43922.2022.9747472.

 wwang20241
Wu Q, Wang W, Fan P, Fan Q, Zhu H, and Letaief K. B. (2024) Cooperative Edge Caching Based on Elastic Federated and Multi-Agent Deep Reinforcement Learning in Next-Generation Networks, IEEE Transactions on Network and Service Management, doi: 10.1109/TNSM.2024.3403842.

ssong2024
Song S, Zhang Z, Wu Q, Fan P, and Fan Q (2024) Joint Optimization of Age of Information and Energy Consumption in NR-V2X System Based on Deep Reinforcement Learning, Sensors, vol. 24, no. 13, Art. no. 3448.

swan2021
Wan S, Lu J, Fan P, Shao Y, Peng C, and Letaief K. B. (2021) Convergence Analysis and System Design for Federated Learning Over Wireless Networks, IEEE Journal on Selected Areas in Communications, vol. 39, issue. 12. pp. 3622-3639, doi: 10.1109/JSAC.2021.3118351.







qwu2023

Wu Q, Zhao Y, Fan Q, Fan P, Wang J, Zhang C (2023) Mobility-Aware Cooperative Caching in Vehicular Edge Computing Based on Asynchronous Federated and Deep Reinforcement Learning. IEEE journal of selected topics in signal processing. pp. 66-81.






zyu2021
Yu Z, Hu J, Min G, Zhao Z, Miao W, Hossain MS (2021) Mobility-Aware Proactive Edge Caching for Connected Values using Federated Learning. IEEE Trans. Intell. Transp. Syst. pp. 5341-5351. https://doi.org/10.1109/TITS.2020.3017474.


khsieh2020
Hsieh K, Phanishayee A, Mutlu O, Gibbons P (2020) The Non-IID Data Quagmire of Decentralized Machine Learning. https://arxiv.org/abs/1910.00189.


yzhao2022
Zhao Y, Li M, Lai L, Suda N, Civin D, Chandra V (2022) Federated Learning with Non-IID Data. https://arxiv.org/abs/1806.00582.


sshirmohammadi
Shirmohammadi S, Ferrero A. Camera as the instrument: the rising trend of vision based measurement. IEEE Instrumentation & Measurement Magazine. https://doi.org/10.1109/MIM.2014.6825388.


jacorts-Osorio2018
Corts-Osorio JA, Gmez-Mendoza JB, Riao-Rojas JC (2018) Velocity estimation from a single linear motion blurred image using discrete cosine transform. IEEE Transactions on Instrumentation and Measurement. pp. 4038-4050.


khou2020
Hou K, Lv X, Zhang W (2020) An adaptive fusion panoramic image mosaic algorithm based on circular LBP feature and HSV color system. 2020 IEEE International Conference on Information Technology,Big Data and Artificial Intelligence (ICIBA). https://doi.org/10.1109/ICIBA50161.2020.9277348.

kxiong2014
Xiong K, Fan P, Xu Z, Yang H, and Letaief K. B. (2014) Optimal Cooperative Beamforming Design for MIMO Decode-and-Forward Relay Channels, IEEE Transactions on Signal Processing, vol. 62, issue. 6, pp. 1476 ? 1489, doi: 10.1109/TSP.2014.2298380. 

xchen2017
Chen X, Lu J, Fan P, and Letaief K. B. (2017) Massive MIMO Beamforming With Transmit Diversity for High Mobility Wireless Communications, IEEE Access, vol. 5, pp. 23032 ? 23045, doi: 10.1109/ACCESS.2017.2766157. 

jliu2020
Liu J, Xiong K, Ng D, Fan P, Zhong Z, and Letaief K. B. (2020) Max-Min Energy Balance in Wireless-Powered Hierarchical Fog-Cloud Computing Networks, IEEE Transactions on Wireless Communications, vol. 19, issue.11, pp. 7064 ? 7080, doi: 10.1109/TWC.2020.3007805. 

pfan2002
Fan P, Feng C, Wang Y, and Ge N (2002) Investigation of the time-offset-based QoS support with optical burst switching in WDM networks, 2002 IEEE International Conference on Communications. Conference Proceedings. ICC 2002, doi: 10.1109/ICC.2002.997330.

rjiang2019
Jiang R, Xiong K, Fan P, Zhang Y, and Zhong Z (2019) Power Minimization in SWIPT Networks With Coexisting Power-Splitting and Time-Switching Users Under Nonlinear EH Model, IEEE Internet of Things Journal, vol. 6, issue. 5, pp. 8853-8869, doi: 10.1109/JIOT.2019.2923977.

yguo2021
Guo Y, Xiong K, Lu Y, Wang D, Fan P, and Letaief, K. B. (2021) Achievable Information Rate in Hybrid VLC-RF Networks With Lighting Energy Harvesting, IEEE Transactions on Communications, vol. 69, issue. 10, pp. 6852-6864, doi: 10.1109/TCOMM.2021.3098030.





CIFAR-10
Krizhevsky A (2009) Learning Multiple Layers of Features from Tiny Images. https://www.cs.toronto.edu/ kriz/learning-features-2009-TR.pdf


hzheng2021
Zheng H, Xiong K, Fan P, Zhong Z, and Letaief K. B. (2021) Age of Information-Based Wireless Powered Communication Networks With Selfish Charging Nodes, IEEE Journal on Selected Areas in Communications, vol. 39, issue. 5, pp. 1393 ? 1411, doi: 10.1109/JSAC.2021.3065038.



sweiFedCo
Wei S, Cao G, Dai C, Dai S, Guo B (2023) FedCo: self-supervised Learning in Federated Learning with Momentum Contrast. 2022 IEEE 24th Int Conf on High Performance Computing & Communications; 8th Int Conf on Data Science & Systems; 20th Int Conf on Smart City; 8th Int Conf on Dependability in Sensor, Cloud & Big Data Systems & Application (HPCC/DSS/SmartCity/DependSys). https://doi.org/10.1109/HPCC-DSS-SmartCity-DependSys57074.2022.00192.

hmcmahan2017
Mcmahan H, Moore E, Ramage D, Hampson S, Arcas B (2017)  Communication-Efficient Learning of Deep Networks from Decentralized Data. Proceedings of the 20 th International Conference on Artificial Intelligence and Statistics (AISTATS). https://arxiv.org/abs/1602.05629.





 Paper Title: Age of Information Analysis for Multi-Priority Queue and NOMA Enabled C-V2X in IoV.






 Age of Information Analysis for Multi-Priority Queue and NOMA Enabled C-V2X in IoV

	This work was supported in part by the National Natural Science Foundation of China under Grant No. 61701197, in part by the National Key Research and Development Program of China under Grant No.2021YFA1000500(4), in part by the 111 Project under Grant No. B12018.

    1st Zheng Zhang
	School of Internet of Things Engineering 

		Jiangnan University

		Wuxi, China 

		zhengzhang@stu.jiangnan.edu.cn
	 

	

	
	3rd Pingyi Fan
	Department of Electronic Engineering 

		Tsinghua University

		Beijing, China 

		fpy@tsinghua.edu.cn
	
	
	2nd Qiong Wu*
	School of Internet of Things Engineering 

		Jiangnan University

		Wuxi, China 

		qiongwu@jiangnan.edu.cn
	Corresponding Author
	 

	

	
	4th Ke Xiong
	School of Computer and Information Technology 

		Beijing Jiaotong University

		Beijing, China 

		kxiong@bjtu.edu.cn			

    
=================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================================




§ ABSTRACT 
	As development Internet-of-Vehicles (IoV) technology and demand for Intelligent Transportation Systems (ITS) increase, there is a growing need for real-time data and communication by vehicle users. Traditional request-based methods face challenges such as latency and bandwidth limitations. Mode 4 in Connected Vehicle-to-Everything (C-V2X) addresses latency and overhead issues through autonomous resource selection. However, Semi-Persistent Scheduling (SPS) based on distributed sensing may lead to increased collision. Non-Orthogonal Multiple Access (NOMA) can alleviate the problem of reduced packet reception probability due to collisions. Moreover, the concept of Age of Information (AoI) is introduced as a comprehensive metric reflecting reliability and latency performance, analyzing the impact of NOMA on C-V2X communication system. AoI indicates the time a message spends in both local waiting and transmission processes. In C-V2X, waiting process can be extended to queuing process, influenced by packet generation rate and Resource Reservation Interval (RRI). The transmission process is mainly affected by transmission delay and success rate. In C-V2X, a smaller selection window (SW) limits the number of available resources for vehicles, resulting in higher collision rates with increased number of vehicles. SW is generally equal to RRI, which not only affects AoI in  queuing process but also AoI in the transmission process. Therefore, this paper proposes an AoI estimation method based on multi-priority data type queues and considers the influence of NOMA on the AoI generated in both processes in C-V2X  system under different RRI conditions. This work aims to gain a better performance of C-V2X system comparing with some known algorithms..



C-V2X, NOMA, RRI, AoI




§ INTRODUCTION

With the innovation and advancement of IoV technology, the development of ITS has catered to a range of in-vehicle applications such as automated navigation, collision warning, and multimedia entertainment <cit.>. Due to the high-speed mobility of vehicles, ensuring timely access for the required data and content through user requests is crucial <cit.>. The traditional request-based approach involves users communicating with the base station, which then accesses the data center to retrieve the requested data<cit.>. However, this method suffers from end-to-end delays and limited backhaul bandwidth <cit.>. To address these issues, C-V2X proposes an interface called PC5 for communication between autonomous vehicles. PC5 offers two resource allocation methods: Mode 3, where User Equipment (UE) requests time and frequency-domain transmission resources from the eNodeB, and Mode 4, where UE autonomously selects resources without involving the cellular infrastructure. Mode 4 not only eliminates the limited coverage drawback but also minimizes interaction between base stations and vehicles, thereby resolving excessive delays and overhead <cit.>.

In Mode 4, vehicles autonomously select communication resources using the SPS protocol based on distributed sensing. SPS allows vehicles to choose from several standardized message intervals (e.g., 10 Hz, 20 Hz, or 50 Hz). However, this increases the likelihood of collisions when multiple vehicles occupy the same resources with the same message transmission interval, leading to higher BLER <cit.>. NOMA is a potential solution for C-V2X communication, promising to enhance spectrum efficiency and handle large-scale vehicle communications, mitigating latency and packet reception probability degradation caused by high vehicle density <cit.>. Serial Interference Cancellation (SIC) is a well-known Multi-User Detection (MUD) technique used to extract overlapping signals, decoding different power levels of received signals occupying the same resource. High-power signals no longer interfere with other low-power signals after decoding, improving the Signal-to-Interference-plus-Noise Ratio (SINR) of low-power signals, reducing transmission failures due to collisions, and lowering BLER <cit.>.

Some studies have examined the effectiveness of NOMA applied in C-V2X. In <cit.>, a NOMA receiver based on Continuous Interference Cancellation (SIC) and Joint Decoding (JD) was proposed, demonstrating its implementation in the current C-V2X communication and its ability to reduce BLER compared to traditional Orthogonal Multiple Access (OMA) methods. In <cit.>, TAKESHI et al. introduced SPS-NOMA based on UpLink Non-Orthogonal Multiple Access (ULNOMA), improving SIC's Signal-to-Noise Ratio (SNR) under broadcast scenarios and alleviating channel congestion. In <cit.>, Utpal et al. proposed a model with a large-scale MIMO Jacobi detection algorithm for PHY layer of C-V2X, enhancing reliability by reducing bit error rate compared to existing PHY layer frameworks. These works demonstrated improvement of NOMA on reliability and transmission delay in the C-V2X system, where reliability and transmission delay were frequently used to assess vehicle communication performance in existing engineering and 3GPP standards. 

It is worth noting that these two metrics are often in trade-off, meaning an increase in reliability performance may come at the cost of increased delay. Therefore, a new metric is necessary to comprehensively reflect both reliability and latency performance, such as AoI, where lower average AoI indicates either lower latency with the same reliability or higher reliability with the same delay <cit.>. In <cit.>, Peng et al. adopted AoI to evaluate the MAC layer performance of C-V2X sidelink, proposing a Piggyback-based cooperative method for vehicles to inform each other of potential resource occupation, reducing collisions and exhibiting good AoI performance. In <cit.>, Zoubeir et al. presented a resource allocation problem based on NOMA, optimizing resource allocation to provide minimum AoI and high reliability for vehicle safety information. 

However, the aforementioned studies did not consider the AoI aspect of packet in the queue in C-V2X system. The packet received by the receiver describe the information at the time of packet generation by the transmitter, necessitating consideration of the queuing process. 
In <cit.>, Akar et al. investigated the freshness of information in IoT-based state update systems using the AoI performance metric. They studied discrete-time servers in multi-source IoT systems, assuming Bernoulli arrivals of information packets and universally distributed discrete phase-type service times across all sources. Their analysis of AoI under various queuing disciplines was formulated in matrix-geometric terms.
In <cit.>, Zhang et al. considered a dual-server short-block wireless communication system to ensure real-time delivery of newly generated information at a relatively high update rate to its destination. Information is generated at a relatively high update rate, encoded into two short-block queues, and delivered in real-time through two parallel paths. The study based on the Markov-chain process investigated the AoI performance of the dual-queue system in the presence of block delivery errors.

The AoI of packet generated in the queue is influenced by packet generation rates and service rates, where the service rate in C-V2X depends on the RRI <cit.>. However, for the same RRI, while the AoI of transmitter undergoes a short queuing time, the collision probability increases, leading to a potentially large AoI at the receiver <cit.>. This motivates us to consider this work. In this paper, we propose an AoI calculation approach based on a multi-priority data type queue in C-V2X. We first design a queue model with four types of messages, each with different priorities, allowing the AoI of receiver to better reflect the ability to observe the  status of transmitter in a timely manner. We then consider the impact of NOMA on AoI in both processes, calculating AoI for different RRIs, and analyzing  effect of muti-priorities queues and NOMA on the C-V2X communication system.
[This paper has been submitted to WCSP. The source code has been released at: : https://github.com/qiongwu86/Analysis-of-the-Impact-of-Multi-Priority-Queue-and-NOMA-on-Age-of-Information-in-C-V2X]. The remaining parts of this article are as follows, Section 2 provides a brief introduction to the system model. Section 3 presents a description of the proposed modeling of approach in detail. In Section 4, we present some simulation results, followed by the conclusion in Section 5.




§ SYSTEM MODEL



We consider the system model as shown in Fig. <ref>. In this model,  a cellular-based V2X communication system is with N half-duplex vehicles. In C-V2X Mode 4, all communication resources are in a resource pool, with the smallest unit in the frequency dimension being an RB (Resource Block) with a size of 180 kHz. A sub-channel typically consists of 10 RBs. The time axis is set as a discrete value, with each time slot lasting 1 ms, representing the size of one sub-frame in the resource pool. In C-V2X Mode 4, there are four priority levels of message generating types. When the multi-priority queue is not empty, vehicles will reserve communication resources from the resource pool. The time interval between reserved resources is defined as the RRI , commonly taking values of 20 ms, 50 ms, or 100 ms. The initial value of reserved resources (RC) is defined as 500/RRI + rand(1000/RRI). Moreover, due to half-duplex communication, vehicles cannot receive signals while using communication resources.

We use AoI as the performance metric for this system and calculate the AoI generated in the multi-priority queue and the transmission process. Concerning the AoI in the queue, using communication resources will change the AoI and position of messages in the queue, while not using communication resources will keep them unchanged. Regarding the AoI in the transmission process, upon receiving a signal, the receiver updates its AoI for the transmitter to the AoI of the received signal; otherwise, it increments the AoI by one time slot. For simplicity, the receiver determines whether it receives the signal by comparing the SINR of the received signal with the SINR threshold as the criteria of successful transmission. When C-V2X employs OMA, it will compute SINR, where interferences mainly come from the collisions of other signals. For NOMA mode, SIC (see Eqn.(<ref>) for detail) is used to improve the SINR of low-power signals in collisions, with SINR calculated for received signals in descending order of their power, where considering only signals with lower power than the current signal as interference. Additionally, in C-V2X, for the same RRI, the AoI in the queue may be very small while the receiver's AoI can be large. To better reflect the improvement of collisions by NOMA, it is necessary to separately calculate the AoI generated in the two processes and analyze the effect of NOMA  on the C-V2X communication system.





§ MATHEMATICAL MODEL
 

In this section, we first establish a computational model to measure the AoI for a C-V2X system with a multi-priority message queue. Subsequently, we propose the use of NOMA based on SIC to improve the AoI in the C-V2X system.



 §.§ AoI in C-V2X


1) C-V2X Queue Model

C-V2X Model 4 consists of four message types with the following priorities: HPDDENMCAMMHD. CAM-type messages are generated periodically, while the other three types are triggered. The generation probability of a new CAM packet is 1/T_c, where T_c is the fixed packet generation period. The new packet generation probabilities for HPD, DENM, and MHD are represented by a Poisson distribution:


    P(arr_i, n^t=1)=λ_n^1/1 ! e^-λ_n

where n ∈{HPD, DENM, MHD}, and λ_n represents the number of packet arrivals for each type of message in a certain time period. For DENM and MHD, the new packets need to be retransmitted multiple times to ensure successful transmission, and this retransmission process is periodic. Thus, four corresponding First-In-First-Out (FIFO) queues are established based on different signal types. Their queue capacity is L and the queue length is q. When the queues meet q L, corresponding new packets can be added and the same transmission opportunity that leaves the queue can be shared.

The transmission opportunity refers to vehicles can use their own reserved resources, defined as t. At this point, the vehicle transmits messages from different queues based on their priority. When the q of the high-priority message queue is non-zero, the corresponding transmission action s is set to 1, and for other lower-priority queues, s is set to 0. Only when the length q of the high-priority queue is zero, the s of next lower-priority queue can be set to 1. A value of s=1 indicates that the messages in that queue can be transmitted; otherwise, they cannot be transmitted. Therefore, the expression for the transmission action s of the multi-priority queue is:


    {[ s_i, H^t=1, s_i, C^t=0             q_i, H^t=1; s_i, H^t=0, s_i, C^t=1 q_i, C^t(1-q_i, H^t)=1;    s_i, H^t=s_i, C^t=0              otherwise ].


Due to the sequential prioritization, two priority queues can be used to represent the relationship between the s of different queues. Let s_i, C^t represent the transmission opportunity of the HPD queue at the time t, and s_i, H^t represent the transmission opportunity of the CAM queue. When the q of all queues are zero, it indicates that no packets can be transmitted, and at this moment, all s  are set to 0.

2) Multi-priority queue AoI Model

The cumulative AoI of messages in the queue is a result of the queuing process, defined as the time from packet generation to transmission. Thus, the processing rate affecting the queue length will also influence AoI. The AoI expression for each message in each queue is given by:


    φ_i, n^t+1, b={[ φ_i, n^t, b+1+1      s_i, n^t=1;   φ_i, n^t, b+1      s_i, n^t=0 ].

where n includes the all four queues, b ∈ [1, q-1] represents the position of the message in the queue at time t for vehicle i, and φ_n,i^t,b denotes the AoI of the message in queue n at position b for vehicle i at time t. When s_n,i^t=1, the position of all messages in the queue will move except for the first message; When s_n,i^t=0, all message positions in the queue remain unchanged. Here, s=1 represents the processing rate, and it depends on the 1/RRI in C-V2X. Thus, the RRI will influence the size of AoI.

The cumulative AoI during the communication process can be considered as the time spent to complete the transmission between the receiver and the transmitter, which is influenced by both the transmission delay and the transmission process. Additionally, since the received message reflects the situation at the time of packet generation by the transmitter, the receiver needs to inherit its size to represent the degree of understanding about the transmitter's situation. The AoI expression for receiver j regarding transmitter i is given by:


    Φ_i → j^t+1={[  φ_i^t, 1+1 u_i → j^t=1; Φ_i → j^t+1 u_i → j^t=0 ].

where Φ_j,i^t represents the AoI of vehicle j regarding vehicle i at time t, and u_j,i^t indicates whether the message is successfully transmitted to j by i. If u_j,i^t=1, Φ_j,i^t is equal to the φ_n,i^t,1 of the highest-priority message in the queue transmitted by vehicle i, plus the transmission delay. If u_j,i^t=0, Φ_j,i^t+1 is equal to Φ_j,i^t plus the subframe size. According to Eq.(<ref>) and (<ref>), it can be observed that transmission failure will cause an greater increase in Φ_j,i^t+1. In C-V2X model 4, the unique way of reserving resources by vehicles means that every transmission failure requires waiting for an RRI interval. The main reason for transmission failure is a low SINR caused by collisions, which can be addressed by NOMA.



 §.§ NOMA in C-V2X
 

1) Collisions in C-V2X

In C-V2X model 4, vehicles autonomously allocate resources and transmit data in a broadcast manner. When multiple vehicles reserve resources in the neighboring time slots, they may select the same resources. Moreover, due to the broadcast nature, multiple vehicles can simultaneously communicate with the same receiver. This situation leads to collisions when multiple vehicles use the same resources to communicate with the same receiver. According to <cit.>, the non collision probability in C-V2X model 4 can be expressed as:


    P_ncol   ≈
       [1-[1-∏_i=0^Γ-1(1-π/1-π i)] 1-P_rk/CSR-N_v+1]^N_v-1

where π represents the probability that a vehicle is in the moment of preparing to select a resource, refers to the probability of simultaneously satisfying the three following conditions: the vehicle queue is not empty, the RC is zero, and a new resource is being rescheduled. P_rk represents the probability of a vehicle selecting a new resource when the reselection counter resets to zero. CSR represents the total number of resources in the selection window. N_v represents the total number of vehicles. Γ is the size of SW, so CSR follows Γ Increase and increase.
 However, the variation range of Γ is small, typically being 20, 50, or 100. Thus, when a specific value of Γ is chosen, [1-∏_i=0^Γ-1(1-π/1-π i)] 1-P_r k/C S R-N_v+1 is a value that changes with the number of vehicles N_v, and as N_v increases, P_col also increases.Furthermore, changing the value of Γ as the time interval for vehicles to transmit messages may reduce the collision probability, but it can also increase the waiting time of messages in the queue, leading to a larger AoI. Therefore, considering the impact of collisions, we explore the use of NOMA to mitigate this issue while changing Γ.

2) SINR Calculation

In C-V2X model 4, the size of transmission resources is not fixed. Vehicles calculate the required bandwidth B and the transmission rate threshold Rth for successful transmission within one subframe based on the size of the message Q, as expressed by:


    R_th^t=B_ilog _2(1+SINR_th)

where R_th = Q because the time taken to complete data transmission for a message must be less than the length of one subframe, i.e., Q/R ≤ 1. Next, the SINR threshold SINR_th is calculated based on the bandwidth and the transmission rate threshold, as shown in the following expression:


    SINR_th =2^Q_i / B_i-1



Due to the varying distances between vehicles at different time instants, the communication experiences different channel impairments, leading to varying communication rates at the receiver. Additionally, since multiple vehicles may use different communication resources within the same subframe, the receiver also receives signals at different rates within different communication resources. If the receiver receives only one signal in a resource, then the SINR between vehicle i and vehicle j at time t is given by:


    SINR_i → j^t, n=p_i → j^t|h_i → j^t, n|^2/σ^2

where P_i is the transmission power of vehicle i, |h_i → j^t, n|^2 is the channel gain between vehicles i and j for resource n, and N_0 is the noise energy. However, due to the half-duplex resource selection scheme in C-V2X, vehicles may select the same resource block when choosing resources, leading to collisions when they use the same resource for transmission. In this case, the wireless information transmission rate is defined as:


    SINR_i → r^t, n=p_i → j^t|h_i → j^t, n|^2/∑_m ∈ N_m p_m → j^t|h_m → j^t, n|^2+σ^2

where m represents interfering vehicles in the C-V2X scenario, and their transmission energy is denoted as P_m. It can be observed that when multiple vehicles use the same resource, the transmission rate decreases, and if the transmission rate is too low, it may result in transmission failure.

Therefore, to address this issue, we introduce NOMA based on SIC. The receiver sorts the received signals in descending order of received power, considering the maximum received power signal as the target signal and the rest as interference signals. After decoding and removing the target signal, this process is repeated for all signals to compute their SINR. Let N_k = {k∈ N∖ i | p_i → j^t|h_i → j^t, n|^2 > p_k → j^t|h_k → j^t, n|^2} represent the set of vehicles whose received power is weaker than vehicle i. Then, the SINR for vehicle i is given by:


    SINR_i → r^t, n=p_i → j^t|h_i → j^t, n|^2/∑_k ∈ N_k p_k → j^t|h_k → j^t, n|^2+σ^2


It can be seen that using NOMA in the event of a collision can reduce the interference on the target signal and increase SINR, thereby reducing the possibility of SINR being lower than the threshold and leading to transmission failure. It is generally believed that |h_i → j^t, n|^2= c_ijd_ij, c_ij represents the coefficient between vehicle i and vehicle j, d_ij denotes the distance between vehicle i and vehicle j, and η is the path loss exponent. The value of d_ij depends on the positions of vehicles i and j at time t.

3) Vehicle Movement
Consider a two-way road with a length of D and U/2 lanes in each direction. We establish a coordinate system where the position of vehicle i at time t is defined as (x_i^t, y_i^u). The origin of the coordinate system is set at the leftmost position of the road, with the x-axis representing the direction of vehicle movement, and the y-axis representing different lanes. Assuming that the vehicle updates its position every very short time, the speed V of vehicle i can be considered constant:


    x_i^t+1=x_i^t+δ V τ, x_i^t∈[0, D]

where δ represents the direction of vehicle, and set x_i^0∈[0, D] be the initial position of the vehicle. And y_i^u depends on the lane index u ∈ [1, …, U] of vehicle i, calculated as:


    y_i^u=u d_y-y_0

where d_y is the width of one lane, and y_0 represents the distance of vehicle i to the edge of the lane. Typically, y_0 is taken as 1/2 d_y, which is half of the lane width.





§ NUMERICAL RESULTS AND DISCUSSION


Simulation Tool is MATLAB R2021b. The simulation is conducted based on the existing work with additional modifications. In this section, we present the simulation results for the comparison of AoI in C-V2X under NOMA and OMA, validating the impact of NOMA on AoI in C-V2X.



 §.§ Simulation Settings

According to the C-V2X standard, we use a 10 MHz bandwidth with a total of 50 RBs. The message size is set to 500 Bytes, and QPSK modulation is applied for propagation. The T_C can be an integer multiple of 100 ms between 100 ms and 1 s, typically set to 100 ms. The arrival rates λ_H, λ_D, and λ_M for different message types are set to 0.0001. The T_H and T_D are set to 100 ms and 500 ms, respectively. The K_H and K_D are set to 8 and 5, respectively. The transmission power of all vehicles is uniformly set to 23 dBm, and the speed is set to 120 km/h.



 §.§ Performance Evaluation


Take the mean of φ_i, n^t, b to represent the total AoI of packets in the queue of each vehicle on average, reflecting the queuing situation.


    φ=1/N_v1/N1/b∑_i=1^N_v∑_n ∈ N∑_b^Lφ_i, n^t, b

where Nv represents the total number of vehicles in the scene, and N is 4, indicating four types of messages. The Δ^t is defined as the average of Φ_i → j^t between each pair of vehicles.


    Δ^t=1/N_v1/N_v-1∑_j=1^N_v∑_i=1^N_v-1Φ_i → j^t

where, Φ_i → j^t represents the AoI between vehicles i and j at time t, which have already computed the AoI between vehicles acting as transmitters and receivers at each moment, taking into consideration the impact of half-duplex communication. Thus, the summation involves calculating the mean AoI between different vehicle pairs i and j. In general, a higher Δ^t indicates a higher number of transmission failures.




Table <ref> presents the communication success rate for different vehicle counts under different values of Γ, which represents the proportion of successfully received messages compared to all received messages. The vehicle density is set at 60 and 100 vehicleskm, and the length of load is set at 500 m. So it can be observed that the success rate is generally higher for 30 vehicles compared to 50 vehicles. This is because an increase in the number of vehicles enhances the probability of reserving resources at the same time, leading to a higher likelihood of resource contention.
Furthermore, the success rate at Γ = 20 ms is consistently lowest. This is because a smaller CSR will resulting in a higher probability of each resource being reserved by multiple vehicles in scenarios with a greater number of vehicles. Moreover, it is worth noting that, for a given number of vehicles, regardless of the value of Γ, employing NOMA technology consistently yields better transmission success rates compared to C-V2X with OMA.



Fig. <ref> shows the age variation trend of various messages when the number of vehicles is set to 50 and Γ is set to 100 ms. The upper part of the graph shows the age change of information when a vehicle uses a single queue, while the lower part shows the situation when a vehicle uses a multi priority queue. At this point, the messages in the queue will pile up, leading to an increasing AoI. 
It can be seen that when there is only one queue, multiple messages queue together, so their overall trend of AoI change is similar.
In a multi priority queue, although the probability of generating MHD messages is the lowest, its AoI is the highest due to its lowest priority. So in multi priority queues, the AoI of high priority messages decreases to a certain extent, ensuring the timeliness of high priority messages.


Fig. <ref> presents the φ with different values of Γ when the number of vehicles is 30 and 50, respectively. When Γ is 20 ms, the comprehensive production cycle of various types of packets is close to 100 ms (much larger than Γ), so the queue remains empty for most of the time, the φ_i, n^t, b hardly increases. Similarly, when Γ is 50 ms, the φ is also relatively small. However, when Γ is 100 ms, which is greater than the production cycle, packets experience more time while waiting in the queue. In addition, it can be observed that when there are more vehicles, the value of φ increases to a certain extent compared to when there are fewer vehicles.


Fig. <ref> shows the Δ^t of 30 vehicles at different Γ. It can be observed that for different Γ, NOMA consistently decreases Δ^t. As shown in Table <ref>, when Γ is 20ms, the collision probability is relatively high, resulting in the message spending the shorter time in the queue, but the information age of the receiving end is not being smaller. By contrast, when Γ is 100 ms, the collision probability is lower. However, due to the accumulation of φ_ i, n^t, b, it continues to rise and instead becomes the oldest information age among the three situations. When Γ is 50ms, the collision rate is the lowest among the three considered cases, and φ_i, n^t, b does not accumulate, resulting in the lowest Δ^t throughout the entire observation period.




Fig. <ref> depicts the Δ^t  for 100 vehicles under different values of Γ. A comparison between Fig. <ref> and Fig. <ref> reveals that an increase in the number of vehicles leads to a general upward trend for all cases, which is attributed to the rising collision probability with an increasing number of vehicles. Additionally, the trends in these two cases are similar to those shown in Fig. <ref>, with the highest information age at Γ=100ms and the lowest information age at Γ=50ms. NOMA has positive effects on information age for different Γ.



§ CONCLUSION

This paper considers the mobility of vehicles and analyzes the impact of NOMA on the AoI in different scenarios. Firstly, we propose a novel multi-priority queue AoI calculation model for C-V2X communication. Then, we investigate the improvement in transmission success rate using NOMA based on SIC to observe changes in AoI. The following conclusions are drawn from our analysis:

	
  * Under the same Γ, the collision probability varies with different vehicle counts. Adjusting Γ can reduce collision occurrences, but it must be carefully selected to avoid increased AoI. 
	
  * NOMA enhances SINR and reduces the effects of collisions, leading to decreased AoI in various scenarios in C-V2X system.






100
	
	FJ2010
	Jing Fan, Shitang Yin, Qiong Wu and Fei Gao, “Study on Refined Deployment of Wireless Mesh Sensor Network,” in Proc. of IEEE International Conference on Wireless Communications, Networking and Mobile Computing (WICOM’10), Chengdu, China, Jul. 2010, pp. 370-375.
		
	QWHXL2019
	Qiong Wu, Hanxu Liu, Cui Zhang, Qiang Fan, Zhengquan Li and Kan Wang, “Trajectory Protection Schemes Based on a Gravity Mobility Model in IoT,” Electronics, Vol. 8, No. 148, Feb. 2019.
	
	QWJZ2014
	Qiong Wu and Jun Zheng, “Performance Modeling of the IEEE 802.11p EDCA Mechanism for VANET,” in Proc. of IEEE Global Communications Conference (Globecom’14), Austin, USA, Dec. 2014, pp.57-63.
	
	QWJZ2014C
	Qiong Wu and Jun Zheng, “Performance Modeling of IEEE 802.11 DCF Based Fair Channel Access for Vehicular-to-Roadside Communication in a Non-Saturated State,” in Proc. of IEEE International Conference on Communication (ICC’14), Syndey, Austrilia, Jun. 2014, pp. 2575-2580.
	
	P2022ITS
	P. Arthurs, L. Gillam, P. Krause, N. Wang, K. Halder and A. Mouzakitis, "A Taxonomy and Survey of Edge Cloud Computing for Intelligent Transportation Systems and Connected Vehicles," in IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 7, pp. 6206-6221, July 2022, doi: 10.1109/TITS.2021.3084396.
	
	wu2023ss
	Qiong Wu, Shuai Shi, Ziyang Wan, Qiang Fan, Pingyi Fan* and Cui Zhang, “Towards V2I Age-aware Fairness Access: A DQN Based Intelligent Vehicular Node Training and Test Method”, Chinese Journal of Electronics, vol. 32, no. 6, 2023, pp. 1230-1244.
	
	QWSN2018
	Qiong Wu, Shuzhen Nie, Pingyi Fan, Hanxu Liu, Qiang Fan and Zhengquan Li, “A Swarming Approach to Optimize the One-Hop Delay in Smart Driving Inter-Platoon Communications,” Sensors, Vol. 18, No. 10, Art. no. 3307, Oct. 2018.
	
	QWJZ2016
	Qiong Wu and Jun Zheng, “Performance Modeling and Analysis of the ADHOC MAC Protocol for Vehicular Networks,” Wireless Networks, Vol. 22, No. 3, Apr. 2016, pp. 799-812. 
	
	QWJZ2015P
	Qiong Wu and Jun Zheng, “Performance Modeling and Analysis of IEEE 802.11 DCF Based Fair Channel Access for Vehicle-to-Roadside Communication in a Non-Saturated State,” Wireless Networks, Vol. 21, No.1, Jan. 2015, pp.1-11.
	
	Liuchen2021
	L. Liu, C. Chen, Q. Pei, S. Maharjan and Y. Zhang,  “Vehicular Edge Computing and Networking: A Survey,” Mobile networks and applications, vol. 26, no.3, pp. 1145-1168, 2021.
	
	wu2024urllc
	Qiong Wu, Wenhua Wang, Pingyi Fan, Qiang Fan, Jiangzhou Wang and Khaled B. Letaief, “URLLC-Awared Resource Allocation for Heterogeneous Vehicular Edge Computing,” IEEE Transactions on Vehicular Technology, 2024, doi: 10.1109/TVT.2024.3370196.	
	
		wu2023YZ
	Qiong Wu, Yu Zhao, Qiang Fan, Pingyi Fan, Jiangzhou Wang and Cui Zhang, “Mobility-Aware Cooperative Caching in Vehicular Edge Computing Based on Asynchronous Federated and Deep Reinforcement Learning”, IEEE Journal of Selected Topics in Signal Processing, Vol. 17, No. 1, Jan. 2023, pp. 66-81.
	
		wu2023XB
	Qiong Wu, Xiaobo Wang, Qiang Fan, Pingyi Fan, Cui Zhang and Zhengquan Li, “High Stable and Accurate Vehicle Selection Scheme based on Federated Edge Learning in Vehicular Networks”, China Communications, Vol. 20, No. 3, Mar. 2023, pp. 1-17.
	
		wu2023DX
	Dunxing Long, Qiong Wu, Qiang Fan, Pingyi Fan, Zhengquan Li and Jing Fan, “A Power Allocation Scheme for MIMO-NOMA and D2D Vehicular Edge Computing Based on Decentralized DRL”, Sensors, Vol. 23, No. 7, 2023, Art. no. 3449.
	
		wu2019SY
	Qiong Wu, Siyang Xia, Qiang Fan and Zhengquan Li, “Performance Analysis of IEEE 802.11p for Continuous Backoff Freezing in IoV,” Electronics, Vol. 8, No. 12, Art. no. 1404, Dec. 2019.
	
		SW2021
	S. Wan, J. Lu, P. Fan, Y. Shao, C. Peng and K. B. Letaief, "Convergence Analysis and System Design for Federated Learning Over Wireless Networks," in IEEE Journal on Selected Areas in Communications, vol. 39, no. 12, pp. 3622-3639, Dec. 2021, doi: 10.1109/JSAC.2021.3118351. 
	
	wu2024wwh
	Qiong Wu, Wenhua Wang, Pingyi Fan*, Qiang Fan, Huiling Zhu and Khaled B. Letaief, “Cooperative Edge Caching Based on Elastic Federated and Multi-Agent Deep Reinforcement Learning in Next-Generation Networks,”, IEEE Transactions on Network and Service Management, 2024, doi: 10.1109/TNSM.2024.3403842. 
	 
	Dai2019
	Y. Dai et al., "Artificial Intelligence Empowered Edge Computing and Caching for Internet of Vehicles, " IEEE Wireless Commun., vol. 26, no. 3, June 2019, pp. 12–18. 
	
		KW2019
	Kan Wang, F. Richard Yu, Lei Wang, Junhuai Li, Nan Zhao, Quansheng Guan, Bing Li and Qiong Wu, “Interference Alignment with Adaptive Power Allocation in Full-Duplex-Enabled Small Cell Networks,” IEEE Transactions on Vehicular Technology, Vol. 68, No. 3, Mar. 2019.
	
		QW2018SYX
	Qiong Wu, Siyang Xia, Pingyi Fan, Qiang Fan and Zhengquan Li, “Velocity-Adaptive V2I Fair Access Scheme Based on IEEE 802.11 DCF for Platooning Vehicles,” Sensors, Vol. 18, No. 12, Art no. 4198, Dec. 2018
	
		QW2024WSY
	Qiong Wu, Siyuang Wang, Hongmei Ge, Pingyi Fan, Qiang Fan and Khaled B. Letaief, “Delay-sensitive Task Offloading in Vehicular Fog Computing-Assisted Platoons”, IEEE Transactions on Network and Service Management, Vol. 21, No. 2, Apr. 2024, pp. 2012-2026. 
	
		QW2015ICC
	Qiong Wu and Jun Zheng, “Performance Modeling and Analysis of the ADHOC MAC Protocol for VANETs,” in Proc. of IEEE International Conference on Communication (ICC’15), London, UK, Jun. 2015, pp. 3646-3652.
	
		JFQW2010
	Jing Fan, Qiong Wu and Junfeng Hao, “Optimal Deployment of Wireless Mesh Sensor Networks based on Delaunay Triangulations,” in Proc. of IEEE International Conference on Information, Networking and Automation (ICINA’10), Kunming, China, Oct. 2010, pp. 1-5. 
	
		PYF2002
	Pingyi Fan, Chongxi Feng, Yichao Wang and Ning Ge, "Investigation of the time-offset-based QoS support with optical burst switching in WDM networks," 2002 IEEE International Conference on Communications. Conference Proceedings. ICC 2002 (Cat. No.02CH37333), New York, NY, USA, 2002, pp. 2682-2686 vol.5, doi: 10.1109/ICC.2002.997330.
	
	Molina2017
	R. Molina-Masegosa and J. Gozalvez, “LTE-V for sidelink 5G V2X vehicular communications: A new 5G technology for short-range vehicle to-everything communications,” IEEE Vehicular Technology Magazine, vol. 12, no. 4, pp. 30–39, December 2017.
	
	3GPP2017
	3G Partnership Project. Evolved Universal Terrestrial Radio Access (E-UTRA); Medium Access Control (MAC) Protocol Specification (v14.3.0, Release 14); Technical Report. 36.321; 3G Partnership Project (3GPP): Sophia-Antipolis, France, 2017.
	
	Di2017
	B. Di, L. Song, Y. Li, and G. Y. Li, "Non-orthogonal multiple access for high-reliable and low-latency V2X communications in 5G systems, " IEEE Journal on Selected Areas in Communications, vol. 35, no. 10, pp. 2383–2397, 2017.
	
		KX2014
	K. Xiong, P. Fan, Z. Xu, H. -C. Yang and K. B. Letaief, "Optimal Cooperative Beamforming Design for MIMO Decode-and-Forward Relay Channels," in IEEE Transactions on Signal Processing, vol. 62, no. 6, pp. 1476-1489, March15, 2014, doi: 10.1109/TSP.2014.2298380. 
	
		XC2017
	X. Chen, J. Lu, P. Fan and K. B. Letaief, "Massive MIMO Beamforming With Transmit Diversity for High Mobility Wireless Communications," in IEEE Access, vol. 5, pp. 23032-23045, 2017, doi: 10.1109/ACCESS.2017.2766157.
	
	Zhang2017
	Y. Zhang, K. Peng, Z. Chen, and J. Song, "SIC vs. JD: Uplink NOMA techniques for M2M random access, "Communications (ICC), 2017 IEEE International Conference on, pp. 1–6, 2017.
	
		JL
	J.Liu, K. Xiong, D. W. K. Ng, P. Fan, Z. Zhong and K. B. Letaief, "Max-Min Energy Balance in Wireless-Powered Hierarchical Fog-Cloud Computing Networks," in IEEE Transactions on Wireless Communications, vol. 19, no. 11, pp. 7064-7080, Nov. 2020, doi: 10.1109/TWC.2020.3007805. 
	
		RJ
	R.Jiang, K. Xiong, P. Fan, Y. Zhang and Z. Zhong, "Power Minimization in SWIPT Networks With Coexisting Power-Splitting and Time-Switching Users Under Nonlinear EH Model," in IEEE Internet of Things Journal, vol. 6, no. 5, pp. 8853-8869, Oct. 2019, doi: 10.1109/JIOT.2019.2923977.
	
		YG
	Y.Guo, K. Xiong, Y. Lu, D. Wang, P. Fan and K. B. Letaief, "Achievable Information Rate in Hybrid VLC-RF Networks With Lighting Energy Harvesting," in IEEE Transactions on Communications, vol. 69, no. 10, pp. 6852-6864, Oct. 2021, doi: 10.1109/TCOMM.2021.3098030.
	
	Situ2020
	Z. Situ, I. W. -H. Ho, Y. Hou and P. Li, "The Feasibility of NOMA in C-V2X," IEEE INFOCOM 2020 - IEEE Conference on Computer Communications Workshops (INFOCOM WKSHPS), Toronto, ON, Canada, 2020, pp. 562-567, doi: 10.1109/INFOCOMWKSHPS50562.2020.9163009.
	
	Hirai2020
	T. Hirai and T. Murase, "Performance Evaluation of NOMA for Sidelink Cellular-V2X Mode 4 in Driver Assistance System With Crash Warning," in IEEE Access, vol. 8, pp. 168321-168332, 2020, doi: 10.1109/ACCESS.2020.3023721.
	
	Dey2022
	U. K. Dey, R. Akl and R. Chataut, "Performance Improvement in Cellular V2X (C-V2X) by Using Massive MIMO Jacobi Detector," 2022 IEEE 19th International Conference on Smart Communities: Improving Quality of Life Using ICT, IoT and AI (HONET), Marietta, GA, USA, 2022, pp. 122-127, doi: 10.1109/HONET56683.2022.10019065.
	
	Kaul2012
	S. Kaul, R. Yates, and M. Gruteser, "Real-time status: How often should one update?" in Proc. IEEE INFOCOM, Mar. 2012, pp. 2731–2735.

		
	ke2024
	Y. Ge, K. Xiong, Q. Wang, Q. Ni, P. Fan and K. B. Letaief, "AoI-minimal Power Adjustment in RF-EH-powered Industrial IoT Networks: A Soft Actor-Critic-Based Method," in IEEE Transactions on Mobile Computing, doi: 10.1109/TMC.2024.3356229.
	
	HZ	
	H. Zheng, K. Xiong, P. Fan, Z. Zhong and K. B. Letaief, "Age of Information-Based Wireless Powered Communication Networks With Selfish Charging Nodes," in IEEE Journal on Selected Areas in Communications, vol. 39, no. 5, pp. 1393-1411, May 2021, doi: 10.1109/JSAC.2021.3065038.
	
	Peng2020
	F. Peng, Z. Jiang, S. Zhang and S. Xu, "Age of Information Optimized MAC in V2X Sidelink via Piggyback-Based Collaboration," in IEEE Transactions on Wireless Communications, vol. 20, no. 1, pp. 607-622, Jan. 2021, doi: 10.1109/TWC.2020.3027353.
	
	Mlika2022
	Z. Mlika and S. Cherkaoui, "Deep Deterministic Policy Gradient to Minimize the Age of Information in Cellular V2X Communications," in IEEE Transactions on Intelligent Transportation Systems, vol. 23, no. 12, pp. 23597-23612, Dec. 2022, doi: 10.1109/TITS.2022.3190799.
	
	Akar2021
	N. Akar and O. Dogan, "Discrete-Time Queueing Model of Age of Information With Multiple Information Sources," in IEEE Internet of Things Journal, vol. 8, no. 19, pp. 14531-14542, 1 Oct.1, 2021, doi: 10.1109/JIOT.2021.3053768. 
	
	Zzx2021
	Z. Zhang, X. Zhu, Y. Jiang, J. Cao and Y. Liu, "Closed-Form AoI Analysis for Dual-Queue Short-Block Transmission with Block Error," 2021 IEEE Wireless Communications and Networking Conference (WCNC), Nanjing, China, 2021, pp. 1-6, doi: 10.1109/WCNC49053.2021.9417272.
	
	GPW2019
	G. P. Wijesiri N.B.A., J. Haapola and T. Samarasinghe, "A Markov Perspective on C-V2X Mode 4," 2019 IEEE 90th Vehicular Technology Conference (VTC2019-Fall), Honolulu, HI, USA, 2019, pp. 1-6, doi: 10.1109/VTCFall.2019.8891331.
	
	MGM2019	
	M. Gonzalez-Martín, M. Sepulcre, R. Molina-Masegosa and J. Gozalvez, "Analytical Models of the Performance of C-V2X Mode 4 Vehicular Communications," in IEEE Transactions on Vehicular Technology, vol. 68, no. 2, pp. 1155-1166, Feb. 2019, doi: 10.1109/TVT.2018.2888704. 
	
	Wijesiri2021
	G. P. Wijesiri N.B.A., J. Haapola and T. Samarasinghe, "A Discrete-Time Markov Chain Based Comparison of the MAC Layer Performance of C-V2X Mode 4 and IEEE 802.11p," in IEEE Transactions on Communications, vol. 69, no. 4, pp. 2505-2517, April 2021, doi: 10.1109/TCOMM.2020.3044340.
	

	





 Paper Title: Optimal Sampling under Cost for Remote Estimation of the Wiener Process over a Channel with Delay.










Journal of  Class Files, Vol. 14, No. 8, August 2015
Shell et al.: Bare Demo of IEEEtran.cls for IEEE Journals


Optimal Sampling under Cost for Remote Estimation of the Wiener Process over a Channel with Delay
    Orhan T Yavaşcan, 
        Süleyman Çıtır, 
        and Elif Uysal, Fellow, IEEE
        J. Doe and J. Doe are with Anonymous University.

    August 5, 2024
==============================================================================================================================================





§ ABSTRACT 
In this paper, we address the optimal sampling of a Wiener process under sampling and transmission costs, with the samples being forwarded to a remote estimator over a channel with random IID delay. The goal of the estimator is to reconstruct an estimate of the real-time signal value from causally received samples. Our study focuses on the optimal online strategy for both sampling and transmission, aiming to minimize the mean square estimation error. We establish that the optimal strategy involves threshold policies for both sampling and transmission, and we derive the optimal thresholds. We utilize Lagrange relaxation and backward induction as our methodology, revealing the problem of minimizing estimation error, under the assumption that sampling and transmission times are independent of the observed Wiener process. Our comparative analysis demonstrates that the estimation error achieved by the optimal joint sampling and transmission policy is significantly lower than that of age-optimal sampling, zero-wait sampling, periodic sampling, and policies that optimize only the sampling times.




IEEE, IEEEtran, journal, paper, template.
















§ INTRODUCTION











Many real-time control and remote monitoring tasks in industrial automation, cyberphysical systems, autonomous vehicles, healthcare monitoring systems, and smart grids rely on state estimation of unstable processes, based on samples obtained over a communication network. Classically, the network is responsible for the transport of samples that it has been handed by the application. Recently, with the advent of goal-oriented networking, the function of the communication system has started to involve how to get the most timely and useful samples to the task of computation at the destination. This relies on timely sampling, jointly taking into account process characteristics and the network delay, to ensure the success of the remote decision making process. For instance, in smart grids, the status information about power demand, supply conditions, and grid health constantly changes. To ensure efficient, reliable, and safe system performance, controllers need accurate estimates of the grid's current status from nearby sensors<cit.>. This necessitates a decision policy on when to generate and transmit samples, to minimize estimation error. Designing such an optimal sampling strategy has been addressed in the previous literature<cit.>, which assumed that the process can be monitored continuously at the sender side.  Incorporating practical constraints on the cost of sampling and transmission makes the problem more challenging, but it is important to address this generalization for results that are more relevant to the design of networked sensing architectures.



In this paper, we study the optimal remote estimation of a Wiener Process from samples that are transmitted through a channel with random delay, as illustrated in Figure <ref>. The random delay models the packet transmission latency in a networked system, where the estimator has access to the data samples over a wired or wireless network. The delays in such a scenario may be caused by factors such as network congestion, the existence of links with long propagation delay, wireless channel fading, interference, and re-transmissions. Source process is modeled as a Wiener process W_t, which describes the values of a scalar measured quantity such as displacement, temperature, etc, that has time variation. The Wiener process is a well-known example of a non-stationary stochastic process, characterized by stationary and independent increments. It has broad applications in modeling the state of an unstable system, in fields such as pure and applied mathematics, economics, finance and physics. The remote estimator provides a minimum mean square error (MMSE) estimate Ŵ_t based on the received samples. The sampling times are denoted as S_n, and the samples as (S_n, W_S_n). 

Upon sampling at time S_n, the pair (S_n, W_S_n) is stored in a buffer by the controller before being transmitted over the channel. We assume that time-stamped samples are encoded and transmitted as individual packets, with the sample index of packet n denoted as k_n. The design of source and channel encoders are outside the scope of our treatment. The buffer only stores the most recent sample, and so only the most recent sample is transmitted. The transmission time might differ from the sampling time, that is, the head-of-line packet may be held in the buffer before transmission. In contrast to the previous literature <cit.> on this problem, in this paper we incorporate sampling and transmission costs. In other words, the objective is to find optimal sampling and transmission policies to optimize mean squared error at the estimator, under sampling and transmission costs. 

We outline the methodology for finding the optimal policies, utilizing Lagrange relaxation and backward induction to iteratively determine the optimal sampling intervals. 






§ RELATED WORK

This paper can be considered as an extension of the treatment in <cit.> as it explores optimal sampling of the Wiener process under delays for minimizing mean square estimation error under a sampling frequency constraint, establishing a connection between age of information (AoI) and estimation error. Online sampling of the Wiener process to minimize MSE was also studied in <cit.> under unknown transmission delays, proposing an adaptive algorithm with cumulative MSE regret O(ln k) and proving its minimax optimality. Other recent research <cit.>, considered the sampling of an Ornstein-Uhlenbeck process through a queuing system. The study showed that without knowledge of the signal, minimizing the age of information is optimal, whereas using causal knowledge of signal values can achieve lower estimation errors. In the field of goal-oriented communications for remote inference, the work <cit.> addresses the designing an optimal sampling and scheduling strategy over channels with random delays that exhibit memory, such as Delay and Disruption Tolerant Networks. The study proposes an index-based threshold policy that effectively utilizes delayed feedback to minimize the expected time-average inference error. In <cit.> the optimal sampler of the Wiener process was designed in the absence of random service time (i.e., Yi = 0). Our work extends the existing research by incorporating both sampling and transmission costs into the optimization problem, and employing a Lagrange relaxation and value iteration.

Along with those studies the paper can be considered as a contribution to the literature on remote estimation, e.g., <cit.>.


§ SYSTEM MODEL AND PROBLEM STATEMENT


We consider a system where an observer samples a Wiener process W_t and sends these samples to a remote estimator over a channel. The remote estimator reconstructs the signal Ŵ_t from the received samples. The sampling times are denoted as S_n, and the samples are W_S_n. 
We will denote transmitted samples as packets and sample index of the n^th packet as k_n. Upon sampling at time S_n, the pair (S_n, W_S_n) is stored in a buffer before being transmitted over the channel. The buffer only stores the most recent sample, and so only the most recent pair (S_k_n, W_S_k_n) is transmitted. 

We assume that the packets experience i.i.d. random transmission delays over the channel. The delay is characterized by the random variable Y satisfying E[Y^2] < ∞, where Y≥ 0 is the transmission delay experienced by the packet. Accordingly, let B_n represent the time at which the transmission of the n^th packet begins. Delivery time of the packet n is D_n = B_n + Y_n and S_n,j is the time of the j^th sample prior to the transmission of n^th packet,  j ∈1,2,...,k_n. The initial value W_0 = 0 is initially known by the estimator, which is represented by S_0 = D_0 = 0. At any time t, the estimator forms an estimate Ŵ_t using the samples received up to time t. The actions of sampling  and transmitting have a associated costs of c_s and c_τ, respectively. (c_s, c_τ > 0). Since c_τ > 0, not all samples are transmitted over the channel. 
 We use time-average mean square error (MSE) to measure the accuracy of estimation:

    MSE = lim sup_T →∞1/T E [ ∫_0^T (W_t - Ŵ_t)^2 dt ].


We can express Ŵ_t with the information available to the estimator at time t which contains the sampling time S_n, sample value W_S_n, and delivery time D_n of the samples delivered by time t and the facts that no sample has been received after the latest sample delivery:
 {(S_n , W_S_n , D_n ),    M_t(t) =max{n|D_n ≤ t}}

Denote the total count of samples that have been collected and processed up to the current time as M_s(t). Hence, the number of samples taken up to time t is given by:

    M_s(t) = ∑_n=1^M_t(t) k_n

where M_t(t) is the index of the last received packet by the time t.
Similar to <cit.>, we assume that the estimator neglects the implied knowledge when no sample was delivered. In this case, the minimum mean-square error
(MMSE) estimation policy <cit.> is given by 

    Ŵ_t    = 𝐄[ W_t | M_t ]
       = W_S_k_n,    if    t ∈[ D_n , D_n+1],    n = 0, 1, 2, . . . ,

which is illustrated in figure <ref>.



Let Z_n,j represent the waiting time between the delivery of sample n and the generation of sample n+1:


    Z_n,j =
         
        S_n,j-S_n,j-1   if  j > 1 
    
        S_n,1-D_n-1   if  j=1



Let the set Π consists of causal sampling policies, where a sampling policy π = (S_1, S_2, …) determines the sampling times S_i based on the available information up to those times. If (Y_1, Y_2, …) is given, therefore D_n, the waiting times (Z_1, Z_2, …) is uniquely determined by (S_1, S_2, …).
Hence, we can use π = (Z_n,1,Z_n,2…) to represent a sampling policy. Therefore, in other words, we aim to find a set of variables {Z_n,1, …, Z_n,k_n} minimizing the MSE. Together with that, a process is regenerative if there are points in time where the process probabilistically restarts itself, known as regeneration points <cit.> and the waiting times Z_n,j form a regenerative process <cit.>. For Z_n,j, we hypothesize the existence of integers 0 ≤ a_1 < a_2 < … such that the process after any a_i (post-a_i process) has the same statistical behavior as the process after a_1 (post-a_1 process). Mathematically, {Z_n,a_i + j, j = 1, 2, …} should have the same distribution as {Z_n,a_1 + j,   j = 1, 2, …}. Pre-a_i process should be independent of the post-a_i process, meaning {Z_n,j,   j = 1, 2, …, a_i - 1} is independent of {Z_n,a_i + j,   j = 1, 2, …}. Such process is well-defined as E[S^2_n,a_1] < ∞, and 0 < E[(S_n,a_i + 1 - S_n,a_i)^2] < ∞. 


If inter-sampling waiting time Z_n,j is a regenerative process, then the sum T_n = ∑_j=1^k_n Z_n,j, corresponding to inter-packet waiting time, is also a regenerative process. This is because the regeneration points of Z_n,j serve as regeneration points for T_n, and the segments of T_n between these points are independent and identically distributed, thereby maintaining the regenerative property.


Because T_n is a regenerative process, by following the renewal theory in <cit.> and <cit.> the problem for minimizing the time averaged MSE with sampling and transmission costs is formulated as:


    mse_opt = min_π∈Π∑_n=1^∞𝔼[ ∫_D_n^D_n+1(W_t - W_0)^2   dt + c_s k_n + c_τ]
    /∑_n=1^∞𝔼[ D_n+1 - D_n ]













§ FINDING OPTIMAL POLICY

The solution for (<ref>) involves introducing λ for normalization (Lagrange relaxation), and using backward induction to iteratively find the optimal sampling intervals {Z_n,1, …, Z_n,k_n}. 



 §.§ Lagrange Relaxation and Problem Simplification


To solve the (<ref>), we introduce a Lagrange multiplier λ and define λ^* as the minimum value of the objective function. Noting that D_n+1-D_n = Y_n+T_n redefine the problem to find the optimal policy for an arbitrary λ by minimizing the following cost function:


    J(λ)=minlim_N→∞1/N∑_n=1^N C_n(λ)


Where C_n, is the expected cost in one delivery epoch. 


    C_n(λ) = 𝔼[∫_D_n^D_n+1(W_t-W_0)^2 dt + c_sk_n + c_τ -λ(Y_n + T_n)]


For any policy π∈Π, we can write:

    ∑_n=1^∞𝔼[∫_D_n^D_n+1-10mu(W_t-W_0)^2 dt+c_s k_n+c_τ-λ^*(Y_n+T_n)]≥0



If we let  λ be nonnegative weight, we get penalized if we violate the constraint, and we are also rewarded if we satisfy the constraint strictly. The above system is Lagrangian relaxation of our original problem ensuring  the overall cost remains non-negative. 


The following assertions are true:

    
  (a) mmse_opt≥λ if and only if J(λ) ≥ 0.
    
  (b) mmse_opt≤λ if and only if J(λ) ≤ 0.
    
  (c) If J(λ) = 0, the solutions to (<ref>) and (<ref>) are identical.





See Appendix A.


Assume we obtain the optimal policy that minimizes J(λ), then we can characterize the behavior of the cost function as follows:

    
  * J(λ^*) = 0: There exists a value λ^* for which the cost function achieves its minimum value. This λ^* represents the optimal trade-off point where the MSE, sampling cost, and transmission cost are balanced most effectively.
    
  * J(λ) > 0 if λ < λ^*: For values of λ less than λ^*, the cost function is positive. This indicates that the policy underestimates the required trade-off, leading to a higher overall cost.
    
  * J(λ) < 0 if λ > λ^*: For values of λ greater than λ^*, the cost function is negative. This suggests that the policy overestimates the required trade-off, resulting in an overall reduction in cost.


The optimal policy is derived by iteratively adjusting λ and evaluating J(λ) until the condition J(λ^*) = 0 is met. This approach ensures that the selected λ^* provides the most efficient balance between estimation accuracy and the associated costs of sampling and transmission. It is important to note that the state of the system is reset at the beginning of each transmission. This reset implies that the decision process for each packet transmission is independent of previous packets, allowing the optimization to be performed on a per-packet basis. Then by value iteration, we derive the optimal waiting times and associated costs. We use Lemma <ref> to simplify the integral part of the cost function.


Let X and Y be independent random variables with finite second moments. Then,

    𝔼[ ∫_X^X+Y (W_t - W_0)^2   dt ] = Y^2/2 + Y(W_X - W_0)^2.





The proof follows from the properties of the Wiener process and it is a known equality in literature. 


 By lemma <ref>, denote sampling error E_j Δ= (w_Y+Z̅_̅j̅^2 - w_0)^2 , and (<ref>) can be written as:
 
 
    J^λ    = c_τ - λμ_y + 𝔼[ Z_1^2/2 + Z_1 Y  + c_s - λ Z_1 ] 
            + ∑_j=2^k𝔼[ Z_j^2/2 + Z_j E_j-1  + c_s - λ Z_j ]
            + μ_y 𝔼[ E_k ]


Note that, 𝔼[E_k] = 𝔼[Y+T_k] = μ_y + ∑_j=1^k𝔼[Z_j] where μ_Y is the mean transmission delay.  The cost function is then simplified to:

    J^λ =    c_τ - (λ - μ_Y)μ_Y 
    
        +   𝔼[Y^2/2] + 𝔼[Z_1^2/2 + Z_1(Y-λ + μ_Y) + c_s] 
    
        +   ∑_j=2^k𝔼[Z_j^2/2 + Z_j(E_j-1-λ + μ_Y) + c_s].

Next, we define h: ℝ_≥0×ℝ_≥0→ℝ as:

    h(x_1,x_2) =(x_1 + x_2 -λ+μ_Y)^2/2-(x_2 -λ+μ_Y)^2/2+ c_s.

and h_0 ≜ c_T - (λ - μ_Y)μ_Y + E[Y^2 / 2]. 
Then, cost function (<ref>) can be rewritten in recursive form as:

    h_0 + E[h(Z_1,Y)] + ∑_j=2^k E[h(Z_j, E_j-1)].



 §.§ Finding optimal policy

Define g_r as the minimum value of the sum in (<ref>) over a horizon of at most r samples given an initial sampling error, capturing the minimum cost achievable from a specific state onward, helping in the iterative backward optimization process. The optimal waiting times {Z_n,1, …, Z_n,k_n} for the n-th packet are determined iteratively using the function g_r(E_m).

    g_r (E_m) = min∑_j=1^r E[h(Z_m+j, E_m+j-1)].

where E_m+j = 𝒩(√(E_m+j-1), Z_m+j)^2.
Notice that, g_r depends only on c_S and λ - μ_Y. In a recursive form, g_r can be expressed as:

    g_r (E_m) = min_Z_m+1 h(Z_m+1, E_m) +  E[g_r-1(E_m+1)].

Following, the minimum value of (<ref>) is:

    h_0 + E[h(Z_1,Y) + g_∞ (E_1)],

where g_∞ (E_m) = lim_r →∞ g_r (E_m) and E_1 = 𝒩 (0,Y+Z_1)^2. 


    Let E' be the sampling error and let Z be the waiting time until the next sample. Then, probability distribution of the next sampling error, E”, is given by:
    
    E” = 𝒩 (√(E'),Z).

    



The error term E_m+j is updated as:

    E_m+j = (W_Y + Z_m+j - W_0)^2

Given E_m+j-1, we update E_m+j iteratively using the mean and variance of the Wiener process. Continue the iteration until g_r(E_m) converges to g_∞(E_m), representing the minimum cost over an infinite horizon. Evaluate the cost function iteratively and stop the iteration when |J^λ| is sufficiently small (e.g., less than a predefined threshold ϵ).



















§ NUMERICAL ANALYSIS

To provide a comprehensive evaluation of the proposed optimal sampling and transmission policy, we consider a variety of scenarios that cover both edge cases and typical operating conditions. These cases help illustrate the behavior of the system under different cost structures and transmission conditions. 

First we analyze the performance of our proposed sampling and transmission policy for Wiener process estimation, as characterized by the Mean Square Error (MSE). 



The results, depicted in Figure <ref>, compare the MSE of our MSE optimal policy with a conventional periodic sampling approach in relation to varying delay variances (σ^2).

The optimal policy was derived using Lagrange relaxation and iterative backward induction to minimize the time-average MSE, considering the costs associated with sampling and transmission. Optimal policy consistently outperforms periodic sampling across different costs, particularly in scenarios with higher delay variability. As the parameter σ grows, the tail of the delay distribution becomes heavier. We observe that MSE_periodic grows quickly with respect to σ^2, much faster than MSE_opt. Consequently, the estimator's performance degrades significantly, resulting in higher MSE. Conversely, the optimal policy exhibits a more gradual increase due to its adaptive nature, adjusting sampling intervals based on the current state and transmission delays. By balancing the trade-off between sampling frequency and delay variability, the optimal policy maintains lower estimation errors across a broader range of delay conditions.

The figure <ref> shows a clear convergence of the g values over the iterations, indicating that the optimization process is functioning correctly and the algorithm is finding the optimal policy. 

In the figure <ref> with σ^2 = 0, the optimal waiting time z decreases linearly with increasing error e, and the curves for different transmission costs (c_τ = 0.1, 1, 10) overlap completely. This is because the transmission delay Y is deterministic, eliminating any variability in transmission times. Consequently, the system can directly reduce the waiting time as the error increases to correct the estimation more frequently. The independence of the curves from the transmission cost indicates that, without delay variability, the primary concern is correcting the error. The linear decrease in the optimal waiting time z with increasing error e suggests a direct relationship between the error and the time to wait before taking the next sample. 



For low delay variance σ^2 = 0.1 the waiting time decreases with increasing error, with minor differences between c_τ values. For high delay variance σ^2 = 2 the waiting time decreases more sharply, with higher c_τ values resulting in significantly longer waiting times. 
When the delay variance is low, the waiting times are relatively short, and the system can afford to be more responsive to errors. The differences between the curves for different c_τ are less pronounced, indicating a smaller impact of transmission costs in low variance scenarios. On the other hand, with high delay variance, the system must wait longer to account for the increased variability in transmission times. The impact of transmission costs is more pronounced, with higher costs leading to longer optimal waiting times to balance the trade-off between error correction and cost.

For all cases, optimal waiting time decreases with increasing error. This behavior is expected as the system should react more quickly to larger errors to maintain accurate estimations. Beyond a certain error range, the optimal waiting time becomes very small or zero, indicating that for very high errors, immediate sampling is preferred, aligning with the need to correct large deviations as promptly as possible.









§ CONCLUSION


In this paper, we addressed the problem of sampling a Wiener process under cost for remote estimation over a channel with random IID delay. Utilizing Lagrange relaxation and iterative backward induction, we derived a policy that minimizes the time-average MSE while accounting for the costs associated with sampling and transmission. The results substantiate the efficacy of our approach in minimizing MSE under varying transmission conditions. Specifically, the robustness of the optimal policy to delay variance highlights its potential for practical implementation in communication systems where transmission delays are unpredictable. This robustness is particularly critical in applications such as remote sensing, networked control systems, and real-time monitoring, where maintaining low estimation error is paramount. Future research could explore the applicability of our findings to other types of stochastic processes and further refine the optimization techniques to accommodate more complex and dynamic network conditions.
























































































































§ PROOF OF (<REF>)
 
Part (a) and (b) is proven in two steps:

Step 1: We will prove that mmse_opt≤λ if and only if J(λ) ≤ 0.

If mmse_opt≤λ, then there exists a policy π = (Z_0, Z_1, …) ∈Π that is feasible for both (<ref>) and (<ref>), which satisfies

    lim_n →∞∑_i=0^n-1𝔼[ ∫_D_i^D_i+1 (W_t - W_S_i)^2   dt ]/∑_i=0^n-1 E[Y_i + T_i]≤λ.

Hence,

    lim_n →∞1/n∑_i=0^n-1𝔼[ ∫_D_i^D_i+1 (W_t - W_S_i)^2   dt - λ(Y_i + T_i) ] ≤ 0.

Because the inter-packet waiting times T_i are regenerative, the renewal theory <cit.> tells us that the limit

    lim_n →∞1/n∑_i=0^n-1𝔼[Y_i + T_i]

exists and is positive. By this, we get

    lim_n →∞1/n∑_i=0^n-1𝔼[ ∫_D_i^D_i+1(W_t - W_S_i)^2  dt -λ(Y_i + T_i) ] ≤ 0.


Therefore, J(λ) ≤ 0.

On the reverse direction, if J(λ) ≤ 0, then there exists a policy π = (Z_0, Z_1, …) ∈Π that is feasible for both (<ref>) and (<ref>), which satisfies (<ref>). From there, we can derive previous steps. Hence, mmse_opt≤λ. By this, we have proven that mmse_opt≤λ if and only if J(λ) ≤ 0.

Step 2: We need to prove that mmse_opt < λ if and only if J(λ) < 0. This statement can be proven by using the arguments in Step 1, in which “≤” should be replaced by “<”. Finally, from the statement of Step 1, it immediately follows that mmse_opt > λ if and only if J(λ) > 0. This completes the proof of part (a) and (b).

Part (c): We first show that each optimal solution to (<ref>) is an optimal solution to (<ref>). By the claim of the part (a) and (b), J(λ) = 0 is equivalent to mmse_opt = λ. Suppose that policy π = (Z_0, Z_1, …) ∈Π is an optimal solution to (<ref>). Then, mmse_π = mmse_opt = λ. Applying this in the arguments of (<ref>), we can show that policy π satisfies

    lim_n →∞1/n∑_i=0^n-1𝔼[ ∫_D_i^D_i+1 (W_t - W_S_i)^2   dt - λ(Y_i + T_i) ] = 0.

This and J(λ) = 0 imply that policy π is an optimal solution to (<ref>).

Similarly, we can prove that each optimal solution to (<ref>) is an optimal solution to (<ref>). By this, part (c) is proven.



§ PROOF OF (<REF>)
 
We start with the original problem of minimizing the long-term average mean square estimation error (MSE) subject to the costs of sampling c_s and transmission c_τ.

The original objective is to minimize the long-term average MSE:

    mse_opt=min_π∈Πlim_N →∞1/N∑_n=1^N𝔼[ ∫_D_n^D_n+1(W_t - W_0)^2   dt + c_s k_n + c_τ]


To introduce the Lagrange multiplier λ, we add a penalty term for the transmission delays and the waiting times. This helps balance the estimation error and the associated costs. The augmented objective function becomes:

    J_λ=min_π∈Πlim_N →∞1/N∑_n=1^N C_n(λ).

Where C_n(λ) is the expected cost in one delivery epoch. Let λ^* be the value of λ that minimizes the augmented objective function. The goal is to find λ^* such that:

    J_λ^*=min_π∈Πlim_N →∞1/N∑_n=1^N C_n(λ^*)


Since we are considering an infinite horizon, we can rewrite the objective function as an infinite sum:

    ∑_n=1^∞𝔼[ ∫_D_n^D_n+1 (W_t - W_0)^2   dt + c_s k_n + c_τ-λ^* (Y_n + T_n) ] ≥ 0.


This ensures that the policy π chosen does not violate the constraint imposed by the Lagrange multiplier λ^*. The existence of such a λ^* implies that there exists a balance point where the long-term average cost is minimized while ensuring that the policy adheres to the constraint. The inequality ∑_n=1^∞𝔼 [·] ≥ 0 guarantees the feasibility of the solution.












































IEEEtran  
   














Orhan T Yavascan
received the B.S. degree in
electrical and electronics engineering from Bilkent
University in 2019. He is currently pursuing the
M.S. degree with Middle East Technical University
(METU). His current research interests include communication networks and information freshness



Süleyman Çıtır
received the B.S. degree in Electrical and Electronics Engineering from Middle East Techincal University (METU) in 2023. He is currently pursuing the M.S. degree with METU. His current research interests include non-terrestrial networks.






Elif Uysal
(S’95-M’03-SM’13) is a Professor in the Department of Electrical
and Electronics Engineering at the Middle East Technical University (METU),
in Ankara, Turkey. She received the Ph.D. degree in EE from Stanford
University in 2003, the S.M. degree in EECS from the Massachusetts Institute
of Technology (MIT) in 1999 and the B.S. degree from METU in 1997.
From 2003-05 she was a lecturer at MIT, and from 2005-06 she was an
Assistant Professor at the Ohio State University (OSU). Since 2006, she
has been with METU, and held visiting positions at OSU and MIT during
2014-2016. Her research interests are at the junction of communication and
networking theories, with particular application to energy-efficient wireless
networking. Dr. Uysal is a recipient of the 2014 Young Scientist Award
from the Science Academy of Turkey, an IBM Faculty Award (2010),
the Turkish National Science Foundation Career Award (2006), an NSF
Foundations on Communication research grant (2006-2010), the MIT Vinton
Hayes Fellowship, and the Stanford Graduate Fellowship. She is an editor for
the IEEE/ACM Transactions on Networking, and served as associate editor
for the IEEE Transactions on Wireless Communication (2014-2018) and track
3 co-chair for IEEE PIMRC 2019. Her current research is supported by
TUBITAK, Huawei, and Turk Telekom.



















 Paper Title: CAVE: Crowdsourcing Passing-By Vehicles for Reliable In-Vehicle Edge Computing.
















CAVE: Crowdsourcing Passing-By Vehicles for Reliable In-Vehicle Edge Computing 

    Jiahe Cao, Qiang Liu 



University of Nebraska-Lincoln

qiang.liu@unl.edu

Dawei Chen, Kyungtae Han 



Toyota InfoTech Labs

{dawei.chen1, kt.han}@toyota.com


    August 5, 2024
=====================================================================================================================================================================



§ ABSTRACT 
In-vehicle edge computing is a much anticipated paradigm to serve ever-increasing computation demands originated from the ego vehicle, such as passenger entertainments.
In this paper, we explore the unique idea of crowdsourcing passing-by vehicles to augment computing of the ego vehicle.
The challenges lie in the high dynamics of passing-by vehicles, time-correlated task computation, and the stringent requirement of computing reliability for individual user tasks.
To this end, we formulate an optimization problem to minimize the end-to-end latency by optimizing the task assignment and resource allocation of user tasks.
To address the complex problem, we propose a new algorithm (named CAVE) with multiple key designs.
First, we reformulate the original problem into two subproblems while incorporating not only incoming but also in-progress tasks.
Second, we solve the task assignment subproblem with reliability constraints by using particle swarm optimization with the adaptive barrier function. 
Third, we solve the resource allocation subproblem by deriving the optimal allocation with Karush–Kuhn–Tucker (KKT) condition.
We build an end-to-end network and compute simulator and conduct extensive simulation to evaluate the performance of the proposed algorithm.
Simulation results show that, our CAVE algorithm reduces more than 15% end-to-end latency than state-of-the-art solutions, without degrading the reliability performance.



In-Vehicle Edge Computing, Reliable Computing, Task Assignment, Resource Allocation




§ INTRODUCTION





Vehicle computing <cit.> is much anticipated to be the next edge computing paradigm to enable and catalyze a wide range of applications on the drive, such as augmented and virtual reality (AR/VR) and large language models (LLMs) from passengers <cit.>.
With the electrification and intelligentization of vehicles, e.g., autonomous driving (AD) <cit.> and software-defined vehicles (SDV) <cit.>, modern vehicles are with much higher computation, networking, and storage capabilities, for example, NVIDIA DRIVE AGX delivers 200 TOPS.
Apart from the needed capability to perform vehicular-related computation (if enabled), e.g., AD and advanced driver-assistance system (ADAS), there remains non-trivial but varying computation and networking resources <cit.>, which can be utilized for accelerating generic computation tasks, such as AR/VR rendering and DNN inference.
This creates a unique possibility to leverage the sparse computation capability of vehicles as the edge computing servers, towards pervasive vehicle edge computing.


Vehicle edge computing has been widely investigated from various perspectives <cit.>, including computation offloading, resource allocation, content caching, security, and privacy.
As high-mobility vehicles have extremely high dynamics (especially wireless connectivity on the drive), one of the key considerations is the reliability of vehicle computation <cit.>, in terms of completing user tasks. 
For example, a vehicle may be disconnected from wireless networks (e.g., cellular vehicle-to-everything (C-V2X)), or experience a significantly low wireless data rate, which can fail its in-progress tasks.
Recent works <cit.> investigated assigning user tasks to multiple vehicles for increasing computation redundancy and thus improving computing reliability.
In existing reliability works, all vehicles are connected via the common wireless network infrastructure (e.g., C-V2X Uu and public cellular), where vehicles receive user tasks from the network side, and send computing results back after task completion, via one-hop wireless transmission (either vehicle-to-infrastructure or infrastructure-to-vehicle).
However, these approaches can be problematic when user tasks originate from individual vehicles (e.g., passengers), which will involve two-hop wireless transmission (i.e., vehicle-to-infrastructure-to-vehicle and reverse path) and result in highly varying and delayed end-to-end performance.


In this paper, we focus on the unique idea of crowdsourcing passing-by vehicles to augment computing of the ego vehicle, where user tasks originate from the ego vehicle.
The rationale is that, there are more computation demands originating from vehicles, including but not limited to, passenger entertainment (e.g., AR/VR), personalized agents (e.g., LLMs), and enhanced driving (e.g., connected AD). 
In this scenario, the ego vehicle wirelessly connects with passing-by vehicles and sends user tasks to them for reliable computation, if not locally computed. 
Here, the unique challenges lie in the high dynamics of passing-by vehicles, time-correlated task computation, and the stringent requirement of computing reliability for individual user tasks.


To address these challenges, we formulate an optimization problem to optimize the task assignment and resource allocation of user tasks, where each task can be assigned to multiple passing-by vehicles to improve computing reliability.
Specifically, we aim to minimize the average end-to-end latency of tasks originating from the ego vehicle, while ensuring the requirement of computing reliability of individual tasks.
We propose a new algorithm (named CAVE) to efficiently solve the optimization problem, with the following key designs. 
1) We reformulate the original problem into two subproblems at different locations, i.e., task assignment in the ego vehicle and resource allocation in individual passing-by vehicles.
In particular, we tackle the time-correlation issue by incorporating the existing in-progress tasks into both subproblems, where we do not modify their task assignments but re-optimize their resource allocation.
2) We convert the task assignment subproblem into unconstrained by using the adaptive barrier function, and solve the converted problem by using enhanced particle swarm optimization. 
3) We solve the resource allocation subproblem by deriving the optimal allocation with the Karush–Kuhn–Tucker (KKT) condition.
Note that, the algorithm does not require full controllability of the wireless network (which is more practical in real-world scenarios), and can be easily adapted to different wireless connectivities.
Extensive simulation results show that our proposed algorithm substantially outperforms existing solutions, in terms of convergence, reliability, and scalability.



    
    
    
    







§ SYSTEM MODEL






















We consider a generic vehicle edge computing network shown in Fig. <ref>, including an ego vehicle and multiple passing-by vehicles.
The ego vehicle serves as an edge computing platform to accelerate the task computation from its passengers (e.g., AR/VR headset and mobile gaming).
We consider the passengers' devices to be connected with the ego vehicle, either wireless (WiFi) or wired (plug-in), with extremely high and consistent data rate, and thus omit the modeling of such transmission latency.
When the ego vehicle drives on the road, it crowdsources passing-by vehicles via direct wireless connectivity (e.g., C-V2X PC5 interface) to accelerate its task computation.
Due to the high dynamics of passing-by vehicles, we aim to assure the computation reliability of tasks by assigning individual tasks to one or more vehicles. 
As tasks are completed in passing-by vehicles, they will be sent back to the ego vehicle, and then forwarded to the devices of passengers.
We denote ℐ as the computation tasks and 𝒥 as the participating vehicles.
We define α_i,j as the binary indicator of task assignment, where α_i,j=1 means the ith task is assigned to the jthe vehicle.
Moreover, we define g_i,j as the allocated computation resources to the ith task by the jth vehicle.
Denote 𝒜 = {α_i,j, ∀ i, j} and 𝒢 = { g_i,j, ∀ i, j}, which are the optimization variables.

Reliability Model.
From the perspective of the ego vehicle, other passing-by vehicles are highly dynamic throughout its driving period.
Assigning a task to only one passing-by vehicle may be unreliable, e.g., vehicles drive away from the coverage of the ego vehicle before they complete the assigned tasks.
To ensure the reliable computation of tasks, we allow each task to be assigned to one or more passing-by vehicles.
We consider, each passing-by vehicle has a reliability function <cit.> (denoted as P_j(·), ∀ j), which represents its probability of failing task computation (e.g., out of the coverage of the ego vehicle and other connectivity issues).
Note that, this reliability function is not constant but varies over time.
Then, we can calculate the reliability of computing a task in the jth vehicle as P_j(L_i,j), where L_i,j is the round-trip latency of the task (see Eq. <ref>).
For the sake of simplicity, we consider that the reliability function of all passing-by vehicles are known when solving the following optimization problem.
Hence, given the task assignment α_i,j, ∀ j, we express the unreliable probability of the ith task when it is computed by multiple vehicles as

    U_i = ∏_j∈𝒥(1-α_i,jP_j(L_i,j) ).




Latency Model.
Each task experiences three stages, i.e., downlink ego-to-vehicles transmission, vehicle computation, and uplink vehicles-to-ego transmission, where both transmission stages may be omitted if the task is locally computed by the ego vehicle.
Hence, we model the round-trip latency <cit.> of ith task in the jth vehicle as 

    L_i,j = D_i/R^d_i,j + C_i/g_i,j + E_i/R^u_i,j,

where R^u_i,j and R^d_i,j are the uplink (from other vehicles to the ego vehicle) and downlink (from the ego vehicle to other vehicles) wireless data rate.
Here, we do not assume that the ego vehicle has full controllability over the wireless communication, which is independently managed by other parties, e.g., cellular network operators.
In other words, the downlink R^d_i,j and uplink R^u_i,j wireless data rate experienced by the ith task at the jth vehicle are generally unknown.
Moreover, C_i, D_i, and E_i are the known computation complexity (e.g., GFLOP), downlink and uplink data size, respectively. 




Problem.
The objective is to minimize the aggregated latency of all the incoming tasks.
Therefore, we formulate the optimization problem ℙ_0 as follows:

    ℙ_0:   min _𝒜, 𝒢       ∑_i ∈ℐ∑_j ∈𝒥α_ij L_i,j
     s.t.        ∑_i ∈ℐα_ij g_i,j≤ G_j^max, ∀ j, 
           ∏_j ∈ℐ (1-α_i,jP_j(L_i,j))  ≤ H_i^min, ∀ i,

where G_j^max is the computation capacity of the jth vehicle and H_i^min is the threshold of failure probability for the ith task.
The optimization variables are the task assignment 𝒜 = {α_i,j, ∀ i, j} (binary) for each task and resource allocation 𝒢 = {g_i,j, ∀ i, j} (continuous) in each vehicle.
The first constraint in Eq. <ref> ensures that, the allocated computation resources to all tasks complies with the capacity G_j^max of individual vehicles.
The second constraint in Eq. <ref> ensures that, the failure probability of each task is below the given threshold H_i^min.



Challenges.
The technical challenges of addressing the above problem are multi-fold.
First, the optimization variables 𝒜 and 𝒢 are closely coupled in both the objective function and the second constraint.
Moreover, multiple parameters in the latency and reliability model, i.e., the experienced wireless data rate, are unknown, when solving the above optimization problem. 
Second, the vehicle edge computing system is tightly time-correlated, where the life-cycle of tasks spans non-negligible period, while incoming tasks could arrive at any time.
In other words, optimizing the incoming tasks could degrade the performance of existing in-progressing tasks, which leads to complicated far-reaching impacts over long-term system performance.
Third, the problem is NP-Hard. Even if the resource allocation 𝒢 is determined, the remained task assignment 𝒜 is binary.
Considering the optimization space of 𝒜 covers all the tasks and vehicles, the optimization problem turns out to be NP-Hard <cit.>. 





    
    
    





§ THE PROPOSED SOLUTION



    
    
In this section, we propose the CAVE algorithm to efficiently address the above optimization problem.
First, we reformulate the original problem into two subproblems at different locations, i.e., task assignment in the ego vehicle and resource allocation in individual passing-by vehicles.
In particular, we tackle the time-correlation issue by incorporating the existing in-progress tasks into both subproblems, where we do not modify their task assignments but re-optimize their resource allocation.
Second, we convert the task assignment subproblem into unconstrained by using the adaptive barrier function, and solve the converted problem by using particle swarm optimization. 
Third, we solve the resource allocation subproblem by deriving the optimal allocation with the Karush–Kuhn–Tucker (KKT) condition.
Note that, the algorithm performs the optimization for incoming tasks while taking care of existing in-progress tasks in both the ego and passing-by vehicles, which addresses the time-correlated issues.




 §.§ Problem Reformulation

To tackle the coupled optimization variables in the original problem ℙ_0, we decompose it into two subproblems, i.e., task assignment and resource allocation.
The rationale is that, task assignment is performed in the ego vehicle whenever there are incoming tasks from passengers, while the resource allocation is performed in the passing-by vehicles at any time slots.
In particular, to tackle the time-correlation issue, we incorporate existing in-progress tasks into the formulation of the following two subproblems.

On the one hand, when optimizing the task assignment subproblem for incoming tasks, we incorporate the impact from existing in-progress tasks into the objective function.
Denote 𝒦 = {𝒦_1, .., 𝒦_j, .., 𝒦_J} as the set of existing in-progress tasks, where 𝒦_j is the subset in the jth vehicle. 
Here, we do not re-optimize the task assignment for existing in-progress tasks, as they are already under computation by different passing-by vehicles. 
Thus, we build the task assignment subproblem ℙ_1 in the ego vehicle as 

    ℙ_1:   min _𝒜       ∑_i ∈ℐ+𝒦∑_j ∈𝒥α_ij L_i,j
     s.t.        ∏_j ∈𝒥 (1-α_i,jP_j(L_i,j)) ≤ H_i^min, ∀ i,

where we only optimize the task assignment 𝒜 for incoming tasks ℐ, although we consider both ℐ and 𝒦.


On the other hand, when optimizing the resource allocation for incoming tasks, we also re-optimize that for existing in-progress tasks.

Thus, we build the resource allocation subproblem ℙ_2 in the passing-by vehicles as  

    ℙ_2:   min _𝒢       ∑_i ∈ℐ+𝒦∑_j ∈𝒥α_ij L_i,j
     s.t.        ∑_i ∈ℐ+ 𝒦α_ij g_i,j≤ G_j^max, ∀ j,

where the optimization variables α_i,j are given, from the perspective of solving this subproblem.




 §.§ Task Assignment Subproblem


In this subsection, we aim to solve the task assignment subproblem ℙ_1 by assigning the incoming tasks to passing-by vehicles only.
Here, we identify the difficulties as 
1) unknown parameters (i.e., uplink and downlink wireless data rate) and non-determined resource allocation 𝒢.
2) the NP-Hardness of the subproblem with constraints.


Parameter Prediction.
First, we deal with unknown parameters in the subproblem ℙ_1.
Specifically, we create a simple prediction model for each passing-by vehicle to estimate its wireless data rate over time, which will be trained with all historical observations.
The prediction model will observe the number of in-progress tasks and historical wireless data in the passing-by vehicle, and generate the prediction of the next wireless data rate.
In this way, the impact of in-progress tasks will be considered in optimizing incoming tasks.
On the other hand, as resource allocation is performed more frequently after the task assignment, it is difficult to forecast how many computing resources will be allocated to each user task.
To balance the accuracy and complexity, we simply presume the resource allocation will be equally allocated, which will derive the fixed g_i,j, given the in-progress tasks in individual passing-by vehicles.

Adaptive Barrier Method.
Second, we deal with the constraints in the subproblem ℙ_1, under determined resource allocation and predicted parameters.
Specifically, we use adaptive barrier function inspired by the interior point method <cit.> to convert the constrained subproblem into unconstrained.
The basic idea is to adaptively incorporate the constraint into the objective function and then solve a series of unconstrained subproblems.
Hence, we build the unconstrained subproblem as

    ℙ_3: min _𝒜     ∑_i ∈ℐ + 𝒦∑_j ∈𝒥α_ij L_i,j
        + μ∑_i ∈ℐ ln [H_i^min - ∏_j ∈ℐ (1-α_i,jP_j(L_i,j)) ],

where μ is the non-negative factor, and we use ln(·) penalize if the constraint is violated.
Note that, we consider the impact from existing in-progress tasks in the objective function, while their reliability constraints are skipped.



Particle Swarm Optimization.
Third, we deal with the NP-Hardness in the above unconstrained subproblem ℙ_3 by using particle swarm optimization (PSO) <cit.>.
PSO is an efficient global searching algorithm, and has been applied and evaluated in a wide range of application domains <cit.>, e.g. energy sector and transportation systems. 
Generally, PSO first initializes a candidate solution and iteratively improves the candidate solution (aka. particles) by moving them towards the global optima as well as the local optima among particles in current iteration 
based on a given quality measurement of solutions Eq. <ref>.


Specifically, we first generate the initial task assignment by randomly sampling from its optimization space.
We observe that, to achieve the given requirement of computing reliability, only a partial of vehicles are needed in most scenarios.
Hence, accelerate the convergence of the PSO searching by reducing its optimization space into n vehicles, such as 5 or 10.
Second, as the particle moves during the search iterations, some particles may not satisfy the requirement anymore.
Hence, we dynamically delete some particles and re-sample them again from the reduced optimization space.
Finally, we stop the searching if reaching the given maximum iterations, where the best assignment strategy is chosen according to the quality measurement throughout the whole search iterations.
Note that, PSO can easily be implemented in parallel for further computing acceleration, which would benefit the real-time decision of task assignment.





























 §.§ Resource Allocation Subproblem

In this subsection, we aim to solve the resource allocation subproblem ℙ_2 in individual passing-by vehicles.
Given the task assignment, we observe that the subproblem ℙ_2 is fully separable, with respect to each passing-by vehicle. 
Hence, we apply the KKT condition to derive the optimal resource allocation in each vehicle.
This is based on our observation that, the subproblem is convex by evaluating the Hessian matrix, which are all positive. 


First, we build the Lagrangian function for the jth vehicle by including the constraint in Eq. <ref> as

    ℒ_j = ∑_i ∈ℐ + 𝒦α_ij L_i,j  + λ_j (∑_i ∈ℐ+ 𝒦α_ij g_i,j - G_j^max),

where λ_j is the Lagrange multiplier for the jth vehicle.

Then, we differentiate ℒ_j with respect to g_i,j and set the derivative equal to zero, which is expressed as

    ∂ℒ_j/∂ g_i,j = - ∑_i ∈ℐ+𝒦α_i,j C_i/ (g_i,j)^2 + λ_j ∑_i ∈ℐ+ 𝒦α_i,j = 0,

where the following condition must be satisfied for the inequality constraints, i.e., 

    λ_j (∑_i ∈ℐ+ 𝒦α_ij g_i,j - G_j^max) = 0.

Based on Eq. <ref> and Eq. <ref>, we can obtain that the optimal resource allocation at the jth vehicle for both incoming and in-progress tasks, expressed as

    g_i,j = √(C_i) G_j^max /∑_i ∈ℐ+ 𝒦α_ij√(C_i),

when α_i,j is non-zeros, otherwise g_i,j = 0.


    




 §.§ The CAVE Algorithm

Based on the above analysis, we propose the CAVE algorithm to solve the problem 𝒫_0, whose pseudocode is summarized in Alg. <ref>.
On the ego vehicle side, it optimizes the task assignment once incoming tasks arrive.
First, we estimate the unknown parameters based on their historical data points.
Second, we initialize the particles and continuously move them in PSO, where particles will be deleted and resampled if the reliability constraint cannot be satisfied.
Third, we stop the PSO searching under given iterations and use the historically best 𝒜^* as the task assignment. 
On the passing-by vehicle side, each of them optimizes its resource allocation at any continuous time (e.g., per milliseconds).
In each passing-by vehicle, we obtain all the tasks and calculate optimal resource allocation based on Eq. <ref>.






§ PERFORMANCE EVALUATION





End-to-End Simulator.
We build a simulator based on our prior work <cit.> with multiple modules, including ego vehicle computing, downlink wireless network, passing-by vehicle computing, and uplink wireless network.
The basic idea of the time-slotted simulator is to flow tasks among these sequential modules, e.g., if a task finishes its downlink transmission, it will be enqueued into the next module (i.e., passing-by vehicle computing).
The computation and transmission of tasks are simulated by deducting its remaining computing complexity and transmission data size, respectively.
In particular, the wireless network is simulated with an open-source 5G system-level simulator <cit.>, with the radio channel of urban micro (UMi - Street Canyon).

Simulation Parameters.
The default number of passing-by vehicles is 20, where their locations are randomly generated in a radius of 100 meters.
The default uplink and downlink bandwidth is 10 MHz, with a maximum transmit power of 20 dBm. 
We use the Poisson Point Process (PPP) to generate the user tasks in the ego vehicle, where the default intensity is 20 
per second.
Without loss of generality, transmission size and computation complexity of user tasks are uniformly sampled from [10, 100] Kbits and [1000, 2000] GFLOP.
By default, the computation capacity of vehicles is 10 TFLOPS, and reliability requirement H^min_i=0.2, ∀ i.
In addition, we use the exponential decaying function to represent the reliability function over time, where P_j(x) = exp(-x), ∀ j, and the unit of x is second.



Comparison Algorithms.
We compare the CAVE algorithm with the following works:

    
    
  * Baseline uses least-workload criteria to assign user tasks to passing-by vehicles, where computing resources in vehicles are equally shared by all assigned tasks.
    
  * FPSO-MR <cit.> is a fault-tolerant particle swarm optimization algorithm. As its problem is different from ours, we adopt its algorithm idea to solve our problem, where existing in-progress tasks are not considered during its optimization. Besides, computing resources in vehicles are equally shared by all assigned tasks. 





Latency.
Fig. <ref> shows the empirical CDF of the average latency of user tasks under different methods.
As we can see that, our proposed CAVE algorithm achieves the best latency performance (194.83ms on average), with a 15.25% and 33.60% reduction than the FPSO-MR and Baseline method, respectively.
Moreover, the CAVE algorithm obtains more than 80% tasks have the latency below 221ms, where the comparative percentile latency is 273ms and 408ms under FPSO-MR and Baseline method, respectively.
In particular, we observe the long tail of the Baseline method, which suggests the necessity of optimizing the task assignment, rather than simply finding the least workload vehicle.

Reliability.
Fig. <ref> shows the empirical CDF of the failure probability of user tasks under different methods.
We can see that, all methods cannot assure 100% failure probability below the given threshold H^min = 0.2.
This could be attribute to the complex system dynamics (e.g., traffic, communication, and computing), especially the lack of fully controllability of wireless transmissions.
For example, the FPSO-MR method obtains only reliability of 57th percentile for all the tasks (i.e., below 0.2), and the Baseline method fails to finish any tasks under the threshold H^min = 0.2.
In contrast, our proposed CAVE algorithm obtains more than 90th percentile for all the tasks (i.e., below 0.2), with a maximum of 0.431 failure probability. 




Redundancy.
To dissect the details behind the failure probability, we show the number of redundant assignment of user tasks over time in Fig. <ref>, under different methods.
The Baseline method always select only one vehicle, and the FPSO-MR method assign a task to 2.45 
vehicles on average.
In contrast, the CAVE algorithm tends to assign more vehicles (average 2.33 
vehicles), to assure the given threshold of the failure probability.
We observe that, although the FPSO-MR method assigns tasks to more vehicles, its achieved latency performance is still worse than our proposed CAVE algorithm.
This may be attribute to the reason that, it overlooks the in-progress tasks when optimizing the incoming user tasks, which can lead to non-trivial performance degradation of user tasks, including both latency and reliability.
























Traffic Intensity.
Fig. <ref> and Fig. <ref> show the average latency and failure probability of user tasks under various traffic intensities, respectively.


As more user tasks in the system, the average task latency of all methods generally increases, because the resource competition becomes more severe in both communication and computation.
In the CAVE algorithm, the average latency is increased from 186.1ms to 221.4ms, and the failure probability also increased, but still not exceed the given threshold of 0.2.
In contrast, we observe the other methods are with much higher task latency. 
This result justifies the CAVE algorithm in handling varying user traffic and assuring the computing reliability. 









Failure Threshold.
Fig. <ref> shows the average latency and failure probability of user tasks under different failure thresholds, respectively.
The lower failure threshold means that the more redundant vehicles per task needed to assure the computing reliability.
Hence, we found that the average number of redundant vehicles per task increase from
1.62 (H^min=0.4) to 2.96 (H^min=0.1) 
in the CAVE algorithm.
As a result, the latency performance generally increases under lower failure thresholds.

This results show that the CAVE algorithm can adapt to different reliability functions while assuring the reliable computing for user tasks.













































§ RELATED WORK


This work relates to the computation offloading, resource allocation, and reliable computing in the scenario of vehicle edge computing.
Computation offloading <cit.> is the widely used approach to exploit more powerful servers (e.g., edge or cloud) to accelerate the task computation of mobile devices and vehicles.
To achieve diverse objective (e.g., latency and energy), existing works have formulated various optimization problems and designed a wide range of algorithms and methods, via both model-based and model-free approaches.
Task assignment and resource allocation are extensively investigated and optimized to achieve more efficient and effective computation offloading <cit.>.
For example, Feng et. al. <cit.> proposed two algorithms that minimize the system latency under both binary and partial reverse offloading problems respectively, by optimizing both offloading decision and radio resource allocation.
However, most existing works <cit.> focused on offloading tasks from vehicles to infrastructural edge/cloud servers, where the wireless connectivity is more reliable and consistent.
In the scenario of in-vehicle edge computing, we explore the idea of crowdsourcing passing-by vehicles to serve passengers tasks, where the vehicle-to-vehicle wireless connectivity becomes much more volatile over time.
Several works <cit.> focused on the reliable computing in vehicle edge computing, by assigning single task to multiple vehicles to increase computation redundancy.
However, their works considered only snapshot-based task computation, where the complex time correlation among consecutive tasks in individual vehicles are overlooked.
In this paper, we focus on the reliable computing problem in the in-vehicle edge computing, where the time correlation of tasks are incorporated in the design of the CAVE algorithm.












§ CONCLUSION

In this work, we explored the possibility of crowdsourcing passing-by vehicles to accelerate the computation of the ego vehicle.
We formulate the optimization problem to reduce the average end-to-end latency by optimizing task assignment and resource allocation for user tasks.
We designed the CAVE algorithm with several key designs to efficiently solve the problem.
Extensive simulation results show that our proposed algorithm outperformed existing solutions, in terms of convergence, reliability, and scalability. 



IEEEtran




 Paper Title: Physically-consistent Multi-band Massive MIMO Systems: A Radio Resource Management Model.











The authors are with the Department of Electrical and Computer Engineering at the University of Manitoba, Canada. This work was partially supported by the project 5G-Enabled Trustworthy Common Operational Picture with Edge Server Data Engine (5G-TCOP) funded by Department of National Defense, Canada.

Physically-consistent Multi-band Massive MIMO Systems: A Radio Resource Management Model
    Nuwan Balasuriya, Amine Mezghani, and Ekram Hossain
    August 5, 2024
========================================================================================

§ ABSTRACT 
Massive multiple-input multiple-output (mMIMO) antenna systems and inter-band carrier aggregation (CA)-enabled  multi-band communication are two key technologies to achieve very high data rates in beyond fifth generation (B5G) wireless systems. We propose a joint optimization framework for such systems where the mMIMO antenna spacing selection, precoder optimization, optimum sub-carrier selection and optimum power allocation are carried out simultaneously. We harness the bandwidth gain existing in a tightly coupled base station mMIMO antenna system to avoid sophisticated, non-practical antenna systems for multi-band operation. In particular, we analyze a multi-band communication system using a circuit-theoretic model to consider physical characteristics of a tightly coupled antenna array, and formulate a joint optimization problem to maximize the sum-rate. As part of the optimization, we also propose a novel block iterative water-filling-based sub-carrier selection and power allocation optimization algorithm for the multi-band mMIMO system. A novel sub-carrier windowing-based sub-carrier selection scheme is also proposed which considers the physical constraints (hardware limitation) at the mobile user devices. We carryout the optimizations in two ways:  (i) to optimize the antenna spacing selection in an offline manner, and (ii)  to select antenna elements from a dense array dynamically. Via computer simulations, we illustrate superior bandwidth gains present in the tightly-coupled colinear and rectangular planar antenna arrays, compared to the loosely-coupled or 
tightly-coupled parallel arrays. We further show the optimum sum-rate performance of the proposed optimization-based framework under various power allocation schemes and various user capability scenarios.



Multi-user multi-band massive MIMO, physically-consistent model, carrier aggregation, joint optimization, block iterative water-filling 





§ INTRODUCTION

Carrier, aggregation (CA) has been a popular technique used in fifth generation (5G) communication systems which allows different communication sessions to select better quality sub-carriers in a fading environment <cit.>. Multi-band CA further extends this capability to select sub-carriers over multiple bands which in turn increases the spectral efficiency. Compared to single-band CA, multi-band CA can harness the diverse characteristics of the sub-carriers in different bands to produce a much diverse sub-carrier selection. For example, the high-band sub-carriers can have larger bandwidths than the low-band sub-carriers and also the high-band is less crowded as of today, hence have less interference[In this work we consider the spectrum proposed for beyond 5G systems and refer to sub-6GHz range as the low-band and 13GHz-20GHz range as high-band.]. However, the propagation and scattering properties of the high-band sub-carriers are poor and also larger bandwidth captures higher noise power. Thus, a combination of low and high-band sub-carriers would be beneficial for CA.


Meanwhile, massive multiple-input multiple-output (mMIMO) antenna systems have also been a very popular technique to achieve high spectral efficiency in modern communication systems. An extensive array of prior work on mMIMO has investigated the techniques to efficiently optimize the antenna selection, optimize beamforming and user scheduling in mMIMO systems. In  <cit.>, the authors optimize power allocation to the antennas and beamforming in mMIMO systems. In <cit.>, the authors  optimize digital and analog beamforming for a mMIMO system while <cit.> optimizes a mMIMO system to obtain a trade-off between the energy efficiency and spectral efficiency.


A combination of multi-band CA and mMIMO can be expected to provide a superior spectral efficiency over the systems using multi-band CA or mMIMO alone. Although <cit.> deals with the optimization of CA-enabled mMIMO systems, it is limited to beamforming optimization. While <cit.> deals with joint carrier aggregation and mMIMO system optimization, it does not consider multi-band transmission. Also, the precoder optimization  is limited to zero-forcing precoding. In <cit.>, we proposed a joint optimization framework for multi-band mMIMO systems in which the optimization is carried out for precoding, sub-carrier selection, user selection and power allocation simultaneously. This communication system and the optimization shows a superior spectral efficiency over all the aforementioned systems.

However, the multi-band operation is always challenging and requires wide-band antenna systems. Two straight forward options for wide-band operation is to use multiple antennas/arrays to cater for different frequency band of interest (as used in the base station in <cit.>) or to have wide-band antennas which can resonate over multiple frequency bands simultaneously <cit.>. The former is not a very attractive solution as the cost of implementation and the form factor will increase significantly. In certain instances the regulations do not allow to install multiple sets of antennas. Especially at a miniaturized mobile device, it is not practical to deploy multiple antennas to cater for carriers in different frequency bands. The latter is an attractive solution and is indeed the practically used solution at the single antenna user devices <cit.>. 

Recently, it was demonstrated in <cit.> that a large multiple-input single-output (MISO) antenna array system is capable of providing a super wide-band response when the transmit antennas are sufficiently tightly coupled, even when the individual antennas are not specifically wide-band antennas. This bandwidth gain is very promising in multi-user multi-band communication systems where an adequately tightly coupled single transmit antenna array suffices to cater to multi-band communication.


To the best of our knowledge there is no joint optimization work on multi-band carrier aggregated mMIMO systems deploying tightly-coupled antenna arrays to harness the aforementioned bandwidth gain and use simple antennas in the antenna array. It is interesting to investigate the optimum tight coupling setup in the transmit antenna array to maximize the overall sum-rate under antenna array size constraints.
Furthermore, a joint optimization framework for multi-band mMIMO systems where the optimization is carried out for precoding, sub-carrier selection, user selection, power allocation, and transmit antenna spacing simultaneously would be required. 
 

 In this paper,  we propose a joint radio resource and transmit antenna system optimization framework for multi-band mMIMO systems. Our key contributions can be summarized as follows:


  * Development of  a circuit-theoretic multi-user, multi-band communication system model to unify the wireless communication channel model and the antenna model (including mutual coupling effects).

  * Development of a joint optimization framework and the alternating simplification which produces a superior sum data rate over state-of-the-art frameworks. 

  * Solution of the optimization problem in two different ways, namely, the online (antenna spacing selection is carried-out dynamically together with other optimization parameters) and offline (antenna spacing is optimized only once, taking the average values for all the other optimization parameters, and then the other parameters are optimized dynamically for the set spacing value).



  * Investigation on the performance of the proposed framework under different power allocation schemes and user device constraints.


 The rest of the paper is organized as follows. In Section <ref>, we describe the CA-enabled, circuit-theoretic multi-band mMIMO system model we use, followed by the descriptions of information flow from input to the output of the above model in Section <ref>. Using the information flow relationship, we develop the proposed joint optimization framework in Section <ref>. In Section <ref>, we conduct a complexity analysis of the proposed optimization approaches. Section <ref> presents the simulation results to benchmark the sum-rate performance of the proposed joint optimization framework. Finally, Section <ref> concludes the paper.




§ BASIC SYSTEM MODEL AND ASSUMPTIONS




 §.§ CA-Enabled Multi-band mMIMO Transmission Model


We consider a multi-band downlink mMIMO communication system shown in Fig. <ref> comprising of a single base station (BS) having an N antennas transmitting in low-band and high-band simultaneously. We consider a uniform δ spacing between the base station antennas which is a parameter to be optimized. However,  the maximum antenna array linear dimension is fixed at D such that the number of antenna elements along one dimension in the transmit array is given by N=⌊D/δ⌋+1. Note that the optimum number of antennas N^* is decided by the optimum inter-element spacing δ^*. Each of the K (≤ N) user mobiles is equipped with a single antenna. We consider a set of independent M_L and M_H sub-carriers in the low-band and high-band, respectively, where the low-band sub-carriers have a bandwidth of B_L while high-band sub-carriers have a bandwidth of B_H in each.
Meanwhile,  each mobile user k, k∈{1,K} can simultaneously operate in η_k, (η_k∈{1,2}) number of bands selected by switching the PIN diodes attached to the filters in the antenna system in the user devices which enables the power-constrained user devices  to operate only in one band at a time. 

Furthermore, each user k is capable of handling a limited 𝒩_k number of sub-carriers selected contiguously within each band. The contiguous sub-carrier selection ensures high spectral efficiency and ease of implementation. We refer to the set of these sub-carriers, denoted by Ω_k, as a carrier window (Fig. <ref>). Note that {η_k,𝒩_k}^K is known a priori. 






The BS has a total power budget of P_T for the downlink communication. Due to the difference in bandwidth of low-band and high-band sub-carriers, the additive white Gaussian noise (AWGN) variance in each band differs and these variances are given by σ_L^2 and  σ_H^2(>σ_L^2), respectively. For simplicity of notation, let us denote the sub-carriers with a common index i spanning over both low-band and high-band such that f_i∈ F={f_L,1,f_L,2,…,f_L,M_L,f_H,1,f_H,2,…,f_H,M_H}. 



 §.§ Signal and Channel Model


 In each sub-carrier i, the input signal vector s_i=[s_i,1, s_i,2, …, s_i,K_i]^T is precoded using a N × K_i precoder matrix, W^D_i, which results in a signal vector x_i=W^D_is_i=[x_i,1, x_i,2, …, x_i,N]^T. The signal is fed into the N transmit antennas. 

The single antenna at a mobile device captures the superimposed signal modulated in the sub-carriers spanned over the two bands and delivers to the processing circuitry. We assume that each frequency sub-carrier undergoes independent quasi-static block Rayleigh fading where the fading coefficients are given by {F_i}^M_L+M_H, and the full knowledge of {F_i}^M_L+M_H is available both at the receivers and the transmitter. At the  start of each block, {F_i}^M_L+M_H is estimated and shared.



 §.§ Equivalent Circuit-Theoretic Model

Recall that in the considered communication system model, we propose to use a single transmit antenna array and single antenna at each of the users. Furthermore, we wish to harness the wideband characteristics resulting from tight coupling of transmit antennas rather than using sophisticated wideband antenna setups. Hence, the inter-element spacing needs to be optimized while considering the mutual antenna element coupling effects during the optimization. However, the mMIMO communication system defined in Sub-section <ref> defines a communication theoretic mMIMO channel while the antenna parameters lie in the domain of antenna theory. Inspired by <cit.>, we define an equivalent circuit theoretic unified model for both antennas and the mMIMO channel which is described as follows (Fig. <ref>). Note that this model uses the multi-port network models defined in <cit.> to jointly model the mMIMO channel and the antenna system.



We consider a set of binary generators v_s=[v_s1,v_s2,…,v_sK]^T which represents the input signal vector and a set of amplifiers with gains given by the matrix 𝐖^D. This amplifier operation is analogous to the conventional beamforming operation. This circuit operation results in a set of voltage signals v'_T=[v'_T1, v'_T2,…,v'_TN]^T which are the input voltage signals to the mMIMO antenna array. Let the input voltages, input currents, output voltages and output currents of the mMIMO system's equivalent multi-port circuit be denoted by v_T=[v_T1, v_T2,…,v_TN]^T, i_T=[i_T1, i_T2,…,i_TN]^T, v_R=[v_R1, v_R2,…,v_RN]^T and  i_R=[i_R1, i_R2,…,i_RN]^T, respectively. v_nR=[v_nR1, v_nR2,…,v_nRN]^T denotes the AWGN added at the receiver antennas while v_RL=[v_RL1, v_RL2,…,v_RLN]^T denotes the load voltages as experienced at the mobile users having matched load impedances given by the K× K diagonal matrix 𝐑_L with real diagonal entries [R_1, R_2, …, R_K]^T. 
The Z_MIMO matrix completely defines the mMIMO channel and the antenna system, where

    [ v_T(f); v_R(f) ]
    =
    [  Z_T(f) Z_TR(f); Z_RT(f)  Z_R(f) ]_Z_MIMO[ i_T(f); i_R(f) ].


Furthermore, the diagonal elements of N × N-sized Z_T(f) and K × K-sized Z_R(f) correspond to self-impedances of the transmit and receive array antennas, respectively, while the off-diagonal elements of Z_T(f) represent the pair-wise mutual-impedances between tightly-coupled antenna elements in the transmit array. We consider receiver mobile devices each equipped with a single antenna and the separation between the mobile devices is very large. Hence, the mutual-impedances between any two mobile device antennas is negligible and we denote this by zero-valued off diagonal elements in Z_R(f). Both Z_T(f) and Z_R(f) are symmetric matrices. N × K-sized Z_TR(f) and K × N-sized Z_RT(f) are transimpedance matrices representing the mMIMO channel, where Z_TR(f)=Z_RT^T(f). However, in the far-field, the signal attenuation is very large, and due to no power input at the receiver, the effect of the receiver on the transmitter side is negligible. Thus, it is reasonable to represent this situation by Z_TR(f) ≈0. 


In this work, we assume all the antenna elements to be simple canonical minimum scattering (CMS) antennas given in <cit.>. Note that the lowest Q factor (hence the maximum bandwidth) is obtained when the antenna operates in the lowest scattering mode, hence we consider the CMS antennas operating only in TM_1 lowest mode which is expected to provide the widest possible bandwidth response. We hereafter refer to the CMS antennas only radiating in TM_1 as TM_1 antennas. Most of the simple antennas such as dipole antennas can be approximated to TM_1 antennas. An equivalent circuit model for a TM_1 antenna is given in Fig. <ref> <cit.>, where the current and the voltage at the input port are given by

    i(f)   =   √(8π/3)√(μ/ϵ)A_1/k_0[1+1/jk_0a]e^-jk_0a, 
    
    v(f)   =   √(8π/3)√(μ/ϵ)A_1/k_0R_a[1+1/jk_0a-1/(k_0a)^2]e^-jk_0a,

with amplitude A_1 and where, k_0=2π f/c and μ, ϵ, k_0, a and c  are the permeability, permittivity, TM_1 mode's complex coefficient, antenna element size (as in <cit.> an antenna which can be contained within a sphere having radius a), and the speed of light. Moreover, L_a, C_a, and R_a represent the equivalent inductance, capacitance and the equivalent resistance in the TM_1 antenna model.



Hence, the self-impedance of the antenna is given by

    Z(f)=v(f)/i(f)=[c^2+j2π fca-(2π fa)^2/j2π fca-(2π fa)^2]R_a,

which can be used to determine the diagonal elements of Z_T(f) and Z_R(f).
Furthermore, the mutual-impedance between two TM_1 antennas p and q separated by a  distance δ_pq and with orientation angles α and β, as shown in Fig. <ref>, is given by

    Z_p,q(f)   =-3 √({Z_p}{Z_q})×
       [1/2sinαsinβ(1/jk_0δ_pq+1/(jk_0δ_pq)^2+1/(jk_0δ_pq)^3).
       . +cosαcosβ(1/(jk_0δ_pq)^2
        +1/(jk_0δ_pq)^3)] e^-jk_0δ_pq,

where {Z_p} and {Z_q} refer to the real parts of the self-impedances of the two antennas. (<ref>) can be used to determine the off-diagonal elements of Z_T <cit.>. In our work, we consider three different antenna element arrangements: 
  * colinear array where α=π, β=0, 
  * parallel array where α=π/2, β=π/2, and 
  * N_1 × N_2 rectangular planar array, which is a combination of colinear arrays and planar arrays (Fig. <ref>).


The colinear configuration results in

    Z_p,q(f)   =3 √({Z_p}{Z_q})×
       (1/(jk_0δ_pq)^2
        +1/(jk_0δ_pq)^3)e^-jk_0δ_pq,

while the parallel configuration simplifies Z_p,q(f) to

    Z_p,q(f)   =3/2√({Z_p}{Z_q})×
       (1/jk_0δ_pq+1/(jk_0δ_pq)^2+1/(jk_0δ_pq)^3)
        e^-j(k_0δ_pq-π).

Here, δ_pq=|p-q|δ.In the rectangular planar array configuration, we find α=π-β, hence the mutual-impedance simplifies to

    Z_p,q(f)   =-3 √({Z_p}{Z_q})×
       [1/2sin^2β(1/jk_0δ_pq+1/(jk_0δ_pq)^2+1/(jk_0δ_pq)^3).
       .-cos^2 β(1/(jk_0δ_pq)^2
        +1/(jk_0δ_pq)^3)] e^-jk_0δ_pq,

where 
    δ_pq={[(p-q)+N_1(⌊q/N_1⌋ -⌊p/N_1⌋) ]^2δ_1^2
    +[⌊p/N_1⌋..
 
    ..-⌊q/N_1⌋]^2δ_2^2}^1/2, β=tan^-1(|⌊p/N_1⌋-⌊q/N_1⌋/(p-q)+N_1(⌊q/N_1⌋ -⌊p/N_1⌋)|δ_2/δ_1).




We assume a Rayleigh faded wireless channels and we model the downlink channel by the transimpedance matrix. However, the mutual coupling at the transmitter antenna array affects the overall transimpedance matrix. A realization of the channel matrix under the influence of mutual coupling at the transmitter and receiver is given in <cit.> and we adopt the same Rayleigh fading model:

    H(f)   =c/2π f d^γ/2({Z_R(f)})^-1/2{Z_R(f)}^1/2F
          ×{Z_T(f)}^1/2({Z_T(f)})^-1/2,

where F is a matrix of size K × N with elements drawn from a complex Gaussian distribution with mean zero and variance 1/2 per dimension, representing Rayleigh fading. Moreover, d and γ represent the distance between the transmit and receive antenna arrays in a far-field communication and the path-loss exponent, respectively.

Following the relationship between channel realization and the transimpedance given in <cit.>, the effective transimpedance matrix can be determined as,

    Z_RT(f)=c/2π f d^γ/2{Z_R(f)}^1/2F{Z_T(f)}^1/2.

 Since the multi-user downlink consists of receiver antennas having negligible mutual coupling, (<ref>) simplifies to  
 
    Z_RT(f)=c/2π f d^γ/2ΥF{Z_T(f)}^1/2,

with Υ=2π fa/√((2π fa)^2+c^2)√(R_a). Since we consider carriers in the GHz range, Υ→√(R_a) when the receiver antennas are made sufficiently large to obtain matching.
With the knowledge of the full Z_MIMO we can now derive a relationship between the input and output signals of the communication system taking the effect of mutual coupling  
into consideration.



§ INPUT TO OUTPUT INFORMATION FLOW
 
Considering (<ref>) and the equivalent full circuit theoretic model shown in Fig. <ref>,


    v_T=Z_Ti_T,


    v_R=Z_RTi_T+Z_Ri_R,


    v_R=-v_nR-R_Li_R,


    v_T=v'_T-R i_T,


    v_RL=-R_Li_R,


    v'_T=W^Dv_s.



Using (<ref>) and (<ref>),

    Z_Ti_T   =   v'_T-R i_T, 
    i_T   =   (Z_T+R I)^-1v'_T.

Using (<ref>) and (<ref>) we obtain,

    Z_RTi_T+Z_Ri_R   =   -v_nR-R_Li_R, 
     
        (Z_R+R_L)i_R   =   -v_nR-Z_RTi_T.

Applying i_T of (<ref>) in (<ref>),

    (Z_R+R_L)i_R   =   -v_nR-Z_RT(Z_T+R I)^-1v'_T,
    i_R   =   -(Z_R+R_L)^-1Z_RT(Z_T+R I)^-1v'_T 
                  -(Z_R+R_L)^-1v_nR.

By applying (<ref>) and (<ref>) in (<ref>) we obtain the relationship between v_RL and v_s given by

    v_RL   =R_L(R_L+Z_R)^-1Z_RT(Z_T+R I)^-1W^Dv_s
                   +R_L(R_L+Z_R)^-1v_nR.


(<ref>) can be further represented in the form of a conventional wireless communication channel model as follows:

    v_RL=H_eqW^Dv_s+v_n,

with H_eq=R_L(R_L+Z_R)^-1Z_RT(Z_T+R I)^-1 and v_n=Av_nR=R_L(R_L+Z_R)^-1v_nR. Note that 𝐀 is a diagonal matrix with non-zero diagonal entries, thus 𝐀 is non-singular. Therefore, we can further normalize (<ref>) by multiplying both sides by 𝐀^-1 to formulate

    ṽ_RL=H̃_eqW^Dv_s+v_nR,

with H̃_eq=A^-1H_eq and ṽ_RL=A^-1v_RL. Note that H̃_eq represents a modified channel coefficients matrix taking the effect of the mutual coupling in the antenna arrays into consideration.


 
Result 1: The maximum radiation efficiency of the transmitter antenna system is given by the largest generalized eigenvalue of the pair {Z_T,(Z_T+R I)}.


Proof: Since we consider lossless antenna elements, the radiated power from the transmit antenna array is 

    P_Rad=i^H_T{Z_T}i_T.

We also calculate the power loss in the transmitter as

    P_Loss=R i^H_Ti_T.

Thus, the radiation efficiency can be calculated as

    η_=P_Rad/P_Loss+P_Rad=i^H_T{Z_T}i_T/i^H_T[{Z_T}+R I]i_T.

Note that both {Z_T} and {Z_T}+R I are real symmetric matrices, hence i^H_T{Z_T}i_T/i^H_T[{Z_T}+R I]i_T is the generalized Rayleigh quotient. The maximum value of the Rayleigh quotient is equal to the largest root ϑ=ϑ_max of the equation |{Z_T}-ϑ[{Z_T}+RI]|=0 which proves Result 1.


Result 2: The mutual information carried in the output and input voltage signals v_RL and v_s in relationship (<ref>) is equal to the mutual information carried in the normalized output and input voltage signals ṽ_RL and v_s in relationship (<ref>).


Proof: Consider the relationship given in (<ref>). Mutual information,

    I(v_s;v_RL)   =   ħ(v_RL)-ħ(v_RL|v_s)
       =   ħ(H_eqW^Dv_s+v_n)-ħ(v_n)
       =   log_2 |H_eqW^DW^D^H H^H_eq+σ^2AA^H|
                           -log_2 |σ^2AA^H| 
       =   log_2 |1/σ^2(A^-1H_eqW^DW^D^H H^H_eqA^-H)+I|.

Similarly, considering (<ref>), the mutual information,

    I(v_s;ṽ_RL)   =   log_2 |H̃_eqW^DW^D^H H̃^H_eq+σ^2I|
                           -log_2 |σ^2I| 
       =   log_2 |1/σ^2(H̃_eqW^DW^D^H H̃^H_eq)+I|.

Using the relationship H̃_eq=A^-1H_eq,

    I(v_s;ṽ_RL)   =   log_2 |1/σ^2(A^-1H_eqW^DW^D^H H^H_eqA^-H)+I| 
       ≡   I(v_s;v_RL).

Hence, when calculating the input-output information flow capacity in the downlink communication system in Fig. <ref>, it is sufficient to calculate I(v_s;ṽ_RL). 



§ PROPOSED RESOURCE MANAGEMENT FRAMEWORK
 

Our aim is to determine the optimal BS antenna element spacing δ, the set of optimum precoding matrices {W^D_i}, i∈ F, the optimal power allocation for different users {P_i,k}, the optimum sub-carrier set (together with the set of window position {Ω_k}), and the maximum sum-rate via joint optimization. For this purpose, we use the circuit-theoretic model we introduced in Subsection <ref>, and in order to maximize the sum-rate, we maximize the sum of mutual information between the input voltage signal v_s and individual single antenna users received signal v_RLk for each of the K users. With Result 2, it is equivalent to maximizing the sum of mutual information between the input voltage signal v_s and normalized individual single antenna users received signal ṽ_RLk for each of the K users.
 


 §.§ Problem Formulation


Let us consider a K_i × N equivalent channel coefficients matrix for each sub-carrier i given by H̃_eq_i  where the rows of H̃_eq_i  are given by h̃_i,k,  k∈[1,K_i] representing the equivalent (mutual coupling effects incorporated) channel coefficient vector from BS antenna array to k-th user's antenna. The equivalent multi-user mMIMO system possesses the uplink downlink dual capacity relationship <cit.>, thus, the optimum downlink sum-rate can be determined considering an uplink with channel coefficient matrix H̃^H_eq_i, AWGN noise with zero mean and σ^2 variance per dimension, added at each BS antenna and the same sum power at the transmitters. Furthermore, let w_i,k represent the k-th row of the dual uplink precoding matrix W_i. Under minimum mean square error-successive interference cancellation (MMSE-SIC) detection, multi-user mMIMO uplink sum-rate per unit bandwidth in sub-carrier i can be calculated as in <cit.>, and is given by log_2 |𝐈+∑_k=1^K_i(h̃^H_i,kw_i,kw^H_i,kh̃_i,k)/σ_i^2|.
Hence, the joint optimization problem can be stated as follows:


    <ref>max_δ_1,δ_2, P_i,k,
    w_i,k, Ω_k R=∑_i=1^M_L+M_H B_i log_2 |𝐈+
    ∑_k=1^K_i(h̃^H_i,kw_i,kw^H_i,kh̃_i,k)/σ_i^2|

                 s.t.

    P_i,k=tr (w_i,kw^H_i,k)


    h̃_i,k=f_i,k(δ_1,δ_2)


    ∑_i=1^M_L+M_H∑_k=1^K_iP_i,k≤ P_T,    P_i,k≥0


    P_i,k=0      f_i∉Ω_k


    (N_1-1)δ_1≤ D_1    (N_2-1)δ_2+a≤ D_2.



Constraints  (<ref>) and (<ref>) are valid for a rectangular planar array with horizontal and vertical dimensions given by D_1 and D_2. In the case of linear antenna arrays, constraint (<ref>) has to be replaced by h̃_i,k=f_i,k(δ) and (<ref>) by only one of the inequalities.
P_i,k is the power allocated to k-th user's communication over sub-carrier i. The constraint (<ref>) on power allocation can be replaced with ∑_j=1^M_L∑_k=1^K_jP_j,k≤1/1+βP_T and ∑_i=1^M_H∑_k=1^K_iP_i,k≤β/1+βP_T, 0≤β < ∞ when band-wise power allocation constraints are implemented while it can be replaced with P_i,k≤P_T/M_L+M_H, when sub-carrier-wise power constraints are implemented.



 §.§ Inner and outer Optimization Sub-problems

It is apparent that this optimization problem is a non-convex problem when a joint optimization is considered for {δ_1, δ_2} (contained within h_i,k), and P_i,k, w_i,k together with Ω_k. Hence, we divide the full optimization problem into two sub problems:

    
  * Optimization for δ={δ_1,δ_2}
    
  * Optimization for P_i,k, w_i,k and Ω_k.

Note that a parallel alternating optimization is infeasible in this particular problem as the dimensions of the equivalent channel gain vectors and the precoding vectors are dependent on the value of δ. Hence, we consider a nested optimization approach as described below.For a given δ value set preserving the validity of constraint (<ref>), corresponding h_i,k vectors can be calculated using (<ref>)-(<ref>) and (<ref>). Then, for the calculated h_i,k, we can carryout a joint optimization for P_i,k, w_i,k and Ω_k. The resultant optimal sum-rate value represents a single point in a one-dimensional function of optimum sum-rate versus independent variable δ (two dimensional function in the case of planar arrays). Now our problem converts to the optimization of this single dimensional optimal sum-rate function with respect to δ (two dimensional optimal sum-rate function with respect to δ={δ_1,δ_2}) and finding the maximum value of optimum sum-rate when δ is varied.
Let us denote the optimization sub-problem where the δ is optimized as outer optimization and the optimization of other variables for a known δ as inner optimization. 





 §.§ Simplified Alternating Inner Optimization

The inner optimization problem (i.e., the constrained optimization problem in (<ref>) for a given δ, hence for a given set of h_i,k) can now be converted to an alternating optimization as shown in Algorithm <ref> <cit.>. Here, R_  R_ refer to the sum-rate calculated at the end of the previous iteration and sum-rate calculated at the current iteration.



Note that the sub-carrier allocation and power allocation here in steps 5 and 6 are carried out using an iterative water filling approach discussed in Subsection <ref>. 



 §.§ Block Iterative Water-filling Joint Precoding and Power Allocation

One of the important components of the proposed joint inner optimization algorithm is the iterative water-filling-based power and sub-carrier allocation algorithm. 

We adopt the iterative water-filling algorithm presented in <cit.> and modify the same to fit the multi-band systems <cit.>. As in <cit.>, the dual uplink sub-carrier and power allocation optimization problem can be simplified as:
 

    max_P_i,k,w_i,k∑_i=1^M_L+M_H B_i log_2|Z_i,k+(h̃^H_i,kw_i,kw^H_i,kh̃_i,k)|
    
    -B_ilog_2|Z_i,k|,


where  Z_i,k=∑_k'=1,k'≠ k^K_ih̃^H_i,k'w_i,k'w^H_i,k'h̃_i,k'+σ^2_i𝐈 is the effective interference and noise variance as seen by the BS in dual-uplink corresponding to k-th user communication, considering all the other communications intended for other users as interference.
Since Z_i,k is a positive semi-definite matrix, Z_i,k can be expressed as Q_i,kΔ_i,kQ_i,k^H, where Q_i,k is a unitary matrix and Δ_i,k is a diagonal matrix with eigenvalues. Thus, we can define ḣ_i,k=h̃_i,kQ_i,kΔ_i,k^-1/2 such that the effect of noise and interference is now absorbed to the channel gain matrix. Now the optimization can be written as:

    max_P_i,k, w_i,k∑_i=1^M_L+M_H B_i log_2|𝐈+(ḣ^H_i,kw_i,kw^H_i,kḣ_i,k)|.


The precoder optimization and power allocation is carried out iteratively as follows. First, we initialize the precoder vector set in carrier i, given by W_i=[w_i,1,w_i,2,…,w_i,K_i], i∈{1,2,…,M_L+M_H}. Then we calculate the sum interference from all other users and the noise in sub-carrier i, corresponding to user k, Z_i,k and the noise and interference absorbed channel coefficient vector, ḣ_i,k. Let the single non-zero eigenvalue correspond to ḣ_i,kḣ^H_i,k  be Λ_i,k (Λ_i,k=ḣ_i,k^2). Similarly, we can calculate the corresponding eigenvalues and the equivalent interference and noise at each of the users in each of the sub-carriers. Let U_i,kΛ^1/2_i,kV_i,k^H and S_i,kΣ^1/2_i,kT_i,k^H be the singular value decompositions of ḣ_i,k and w_i,k, respectively, where  U_i,k,V_i,k,S_i,k,T_i,k are unitary matrices each. We can maximize the objective function in (<ref>) by selecting S_i,k=U_i,k, hence we can rewrite the optimization problem as:

    max_P_i,kR=∑_i=1^M_L+M_H B_i ∑_k=1^K_ilog_2(1+Λ_i,kP_i,k).

Now we are in a position to consider the system as a parallel set of channels communicating independently and allocate power using a standard water-filling algorithm where the optimum power allocation to user k in i-th carrier is given by P^opt_i,k=[κ_i-1/Λ_i,k]^+ such that ∑_i=1^M_L+M_H∑_k=1^KP^opt_i,k≤ P_T. [α]^+ denotes a function whose output value is α if α≥ 0 and 0 when α<0. Here, κ_i can take two different values based on the band the carrier i belongs to. However, the ratio between the κ_i values for high-band to low-band sub-carriers, ϵ is the same as the bandwidth ratio of the sub-carriers. Hence, the total sum-rate can be determined as  R^opt=∑_i=1^M_L+M_H∑_k=1^K_iB_i[log_2(Λ_i,kκ_i)]^+. Furthermore, we can calculate the corresponding precoding vector for user k in i-th sub-carrier using w_i,k=U_i,k[P^1/2_i,k 0]^T.
Note that, at the end of the iteration, we calculate a set of updated uplink precoder matrices which we treat as the starting uplink precoder set for the next iteration. This iterative precoder optimization is carried out until the sum-rate does not show a significant further improvement. 
Also note that, this sub-optimal precoder optimization allocates zero power to some user-carrier combinations so that some sub-carriers can be fully unused by any of the users. Thus, the iterative water-filling sub-algorithm produces an effective joint sub-carrier selection and precoder optimization. It is worth mentioning that the dual uplink precoder W_i is different from the downlink precoder W^D_i. However, W^D_i can be determined via a linear transformation of W_i. For brevity, we omit the details in this paper (see <cit.>).



 §.§ Selecting Windowed Channels

We use the same subcarrier windowing technique as in <cit.>, which can be described as follows. As mentioned in Section <ref>, some devices are capable of supporting only a limited set of sub-carriers at a time. During the joint precoder optimization and sub-carrier selection step described in Subsection <ref> we also accommodate a sub-carrier selection constraint to represent the aforementioned device limitations. While selecting sub-carriers for power allocation, we also optimize the carrier windows vector Ω={Ω_1,…,Ω_K}. One exhaustive approach to perform this Ω optimization is to run the joint inner optimization algorithm for each and every possible carrier window set combination satisfying {η_k,𝒩_k}^K. However, this approach requires us to determine the sum-rates for a total of ∏_k=1^Kϕ_k different combinations, where ϕ_k is the total number possibilities for Ω_k. Therefore, we propose a simplified approach to determine the optimal Ω. Recall that during each iteration we calculate the value Λ_i,k which is a good measure of how much power is allocated to each user in each sub-carrier. A large Λ_i,k indicates a large power allocation and vise-versa. Furthermore, recall that κ_i is different for low-band and high-band sub-carriers, and as a result, there is an ϵ times bias for power allocation for high-band sub-carriers than for low-band sub-carriers. Thus, for each user k we select the Ω_k such that ∑ϵ̅Λ_i,k is a maximum, where ϵ̅ is ϵ for high-band sub-carriers and 1 for low-band sub-carriers. This approach reduces the complexity of the search for the optimum window to ∑_k=1^Kϕ_k. Power allocation considers only sub-carriers in the selected windows only.



 §.§ Outer Optimization and the Overall Algorithm

During the outer optimization we maximize the sum-rate function (let us denote this function by g(δ)) with respect to δ under the constraint (<ref>). However, the function itself is non-convex, hence we follow a global optimization approach for the outer optimization. We primarily deploy Particle Swarm Optimization algorithm <cit.> to find the global maximum. In comparison to a conventional optimization, particle swarm optimization simultaneously initiates search from a multiple randomly located points over the domain of the objective function which are known as particles. Moreover, each particle is assigned a random velocity which represents the direction and the amount of movement in the search space. Then, during each iteration, the velocity is updated such that these particles are pushed towards a global optimum. Due to the fact that the multiple particles are optimized simultaneously, the algorithm converges to the global minimum with high probability. However, particle swarm algorithm requires a very large number of iterations for convergence, hence a large number of function evaluations within each of the iterations. Recall that each of these function evaluations requires to run the Algorithm <ref> which itself is a complex operation. Thus, it is prohibitively expensive to resort only to a particle swarm based optimization for the outer optimization. Here, to have an affordable complexity we use several iterations of the particle swarm optimization to bring the search to a point within the neighbourhood of the global maximum and then follow a Swan bracketing and Golden rule bisection-based line search <cit.> to find the global maximum. In the case of planar antenna arrays, with the presence of two optimization variables we replace the line search with the Gradient Ascent Algorithm <cit.> with a stopping criteria of reaching a minimum given sum-rate improvement per iteration.


 The overall optimization algorithm can be stated as shown in Algorithm <ref>[g(.) requires a single run of inner optimization given in Algorithm <ref>, δ̃ and δ̇ are two predefined threshold constants.].


Overall optimization algorithm.





 §.§ Offline and Online Optimizations

The joint optimization shown in Algorithm <ref> can be implemented in two different approaches. In the first, one can generate a set of F, user device distances d={d_1,d_2,…,d_K} and optimize for an average sum-rate and to find an optimal value for antenna spacing given by δ^*. Then we can fix the antenna spacing at δ^* (design and implement the antenna system to have δ^* element spacing) and carryout only the optimization in Algorithm <ref> periodically while in operation. We refer to this approach as offline optimization. This approach is attractive in terms of low complexity and is more viable in practical operation. However, another more efficient, but high complexity approach is to have a densely populated antenna element array at the base station with a maximum array dimension D and carryout full joint optimization periodically. Then the elements in the dense antenna array are activated (non-activated elements are to be electronically terminated to avoid passive radiation effects) to maintain the determined optimum δ^*, while power allocation, precoder optimization and the subcarrier selection are also carried-out at the same time. We refer to this approach as online optimization. Due to the fact that this approach considers the instantaneously available Rayleigh fading coefficients and device locations with respect to the base station, it is expected to provide a superior average optimum sum-rate. However,  due to the large number of function evaluations (hence, massive run of Algorithm <ref>) this is relatively complex, thus can be considered to determine an upper bound for the sum-rate performance. When low-complexity is desired in lieu of superior sum-rate performance, one can opt for the offline optimization.



§ COMPLEXITY ANALYSIS
 
Recall that the proposed optimization algorithm performs function evaluations during its outer optimization while each of these function evaluations involve a single run of the inner optimization. Inner optimization itself is a alternating iterative process, thus, it is worth to investigate the complexity involved in the offline and online optimization processes.
During each function evaluation we require to calculate Ĥ_eq once, followed by an iterative run of a number of steps listed in Table <ref>. Table <ref> also lists the number of mathematical operations required for each of the different steps. Let us denote the average number of iterations required for the inner optimization algorithm to converge by ℐ_IO and the average number of iterations required for the  iterative power allocation step by ℐ_PA. Then the overall average number of operations required for the inner optimization is N_1+{N_2+N_3+N_4+N_5+(N_6+N_7)ℐ_PA}ℐ_IO.

The outer optimization involves function evaluations, each of which is a single run of the inner optimization algorithm. In global search algorithms it is customary to investigate the complexity in terms of function evaluations, as each of the function evaluations requires a very large number of mathematical operations compared to the non-function evaluations based mathematical operations. Let the pre-fixed number of iterations in the particle swam optimization and the average number of iterations for Swan bracketing and Golden Ratio bisection be ℐ_PS,ℐ_SB, and ℐ_BS, respectively. With the number of particles ζ in the particle swarm optimization, we require an average of ζ(1+ℐ_PS) function evaluations for the particle swarm optimization. We also require 2+ℐ_SB and ℐ_BS function evaluations for bracketing and bisection algorithms, respectively. Thus, we require an average total of [2+ζ(1+ℐ_PS)+ℐ_SB+ℐ_BS]×[N_1+{N_2+N_3+N_4+N_5+(N_6+N_7)ℐ_PA}ℐ_IO] mathematical calculations for the overall online optimization.However, recall that in the offline optimization, we carryout the outer optimization only once and only the inner optimization is run during the operation. Thus, we perform only a single run of the inner optimization which requires only N_1+{N_2+N_3+N_4+N_5+(N_6+N_7)ℐ_PA}+ℐ_IO mathematical operations resulting in a reduced computational complexity by a factor of 1/2+ζ(1+ℐ_PS)+ℐ_SB+ℐ_BS.





§ SIMULATION RESULTS AND DISCUSSION
 
Let us consider two carriers centered at 3.5GHz and 17.5GHz for the low-band and high-band operation, respectively. 
Moreover, let the bandwidth of the low-band sub-carriers be B_L=120kHz each and the bandwidth of the high-band sub-carriers be B_H=480kHz each. The transmit power at the base-station, P_T is equal to 2 Watts and the antenna element size is a=0.0025m.
Since the noise power is proportional to the channel bandwidth, the high-band channel noise power to low-band channel noise power ratio takes the same value as the bandwidth ratio. 
All sub-carriers undergo quasi-static Rayleigh block fading where each element of the Rayleigh fading channel matrix F_i is drawn from a complex Gaussian distribution having zero mean 1/2 variance per dimension.
We also assume that full knowledge of {F_i}^M_L+M_H is available at both the transmitter and the receivers. 
Without loss of generality we select the window size vector 𝒩 to be having integer entries drawn from a uniform distribution in the range [1,M_L+M_H]. 

We first consider the average equivalent channel gain variation over different frequencies for the optimized antenna spacing and Fig. <ref> shows the magnitude Bode plots for the three antenna configurations considered together with that for a mutually uncoupled transmit antenna array. It is clear that the colinear configuration has a superior wideband response compared to the parallel arrangement. Meanwhile, the rectangular planar array produces the highest response of the three configurations. Parallel configuration produces only a slightly better response than the uncoupled array. 

We also show the radiation efficiency against frequency, for the considered antenna configurations in Fig. <ref>. Radiation efficiency variation remains almost the same for the mutual coupled configurations while it is relatively inferior at low frequencies when uncoupled antennas are used.Then we investigate the sum-rate performance of the proposed optimization framework where only the inner optimization is carried out for different BS antenna element spacing values and for different overall array dimensions. Here we perform the inner optimization where power is allocated among all carriers from a common pool. 

Fig. <ref> depicts the sum-rate variation with different antenna spacing where the sum-rate improves when the antenna elements are brought closer which is a combined result of the increase of the number of elements resulting in a diversity gain and also the bandwidth gain due to the mutual coupling between antenna elements. However, the sum-rate starts to reduce after a certain optimum spacing value. The reduction is a result of the antenna impedance mismatches and correlations starting to dominate over the aforementioned gains, when the antenna elements are brought closer than the optimum spacing. Furthermore, in Figs. <ref> and <ref> we observe that the optimum sum-rate is higher for colinear array configuration than that for the parallel array configuration. Due to the poor mutual coupling effects existing in the parallel configuration compared to the colinear configuration, the former shows lesser additional bandwidth gain. Thus the resultant optimum sum-rate value becomes  comparatively smaller.











Fig. <ref> shows the colour map for the sum-rate of a planar BS antenna array of size 0.05m×0.05m vs different antenna element spacing values. It is apparent that the sum-rate performance improves with the reduced antenna spacing and reaches a saturated value, in contrary to the linear arrays where peak sum-rate behaviour is observed. Unlike in the linear arrays, in the planar array antennas in a 2D plane (from different angles) creates mutual coupling effects such that the mutual coupling effects always increases. However, after a certain minimum antenna spacing the spacial correlation takes over the effects of mutual coupling resulting in a saturation behaviour.

Next, we investigate the optimum sum-rate performance for different number of sub-carriers and for different average signal to noise ratio (SNR) values at the receiver antennas. Note that this investigation is an extension of our sum-rate performance comparison in <cit.> for uncoupled antenna arrays.
We compare the total sum-rate for the


  * Proposed joint resource optimization under carrier-wise power constraints (i.e., each sub-carrier gets P_T/M_L+M_H), denoted as CWPA.

  * Proposed joint resource optimization under band-wise power constraints (i.e., high-band gets β/1+β P_T and low-band gets 1/1+β P_T, 0≤β < ∞), denoted as BWPA.

  * Proposed joint resource optimization under full range power constraints (i.e. all carriers are allocated power from a common pool having P_T power), denoted as JPA. We consider η=2 for all users (Case 1), η=1 for all users with 75% users in high-band and 25% users in low-band  (Case 2), η=1 for all users with 25% users in high-band and 75% users in low-band  (Case 3), and η=2 for 75% of the users and η=1 for 25% of the users (Case 4).

















From Figs. <ref>, <ref>, <ref>, and <ref> for linear antenna arrays, it is apparent that the JPA scheme (Case 1) is superior to the CWPA and BWPA (β=1) schemes resembling the same comparison as in the case of uncoupled arrays. This is due to the fact that, in JPA, carrier-user blocks for allocating power are selected from an overall pool which enables to select the best blocks for higher power allocation. On the contrary, when the power allocation is constrained band wise or sub-carrier wise, it hinders the efficient selection of carrier-user blocks for power allocation. It is also apparent from Figs. <ref> and <ref> that, under all schemes the sum-rate improves with the increased number of sub-carriers. 
Figs. <ref> and <ref> further show that JPA (Case 4) is producing only a marginally inferior sum-rate compared to JPA (Case 1) and JPA (Case 2), whereas, JPA (Case 3) produces a significantly less sum-rate. The higher sum-rate in Cases 1, 2 and 4 are due to the larger unrestricted carrier space to select the sub-carriers than those in Case 3. 

Moreover, from Figs. <ref> and <ref> it is apparent that when the the power allocation ratio, β is increased, the sum-rate performance also increases at all the SNR values and reaches a maximum. However, the maximum sum-rate achievable via band-wise power allocation is significantly less than the sum-rate which can be achieved via the JPA scheme. This also implies that the proposed joint power allocation and the associated optimization framework over multiple bands can harness a considerable additional sum-rate than which can be achieved using only the high-band (β→∞) or the low-band (β=0).The sum-rate performances at all instances when deploying colinear arrays are superior over the cases using parallel arrays. This is a direct result of the bandwidth gain existing in colinear tightly coupled BS array configurations. 
In Figs. <ref> and <ref> we also show the optimum sum-rate performance for different number of carriers per frequency band and also for different SNR values. Although the sum-rate variation is different to that of a linear array, planar array shows a similar behaviour in sum-rate performance over different number of sub-carriers per band and for different SNRs. Joint power allocation where all the users are allowed to allocate power from a common pool, without any restriction outperform all the other power/user allocation schemes.While generating the results in Figs. <ref>-<ref>, we perform inner optimization a number of times to find an average g(δ) during each of the function evaluations. It is obvious that we have optimized for an average sum-rate (offline optimization). However, when the instantaneous Raleigh fading realizations and user position sets are considered each time, followed by an online optimization, we can have a superior average optimum sum-rate in linear arrays. Due to the saturating maximum sum-rate behaviour in planar array, which approximates a continuous aperture at small inter-element spacing, the aforementioned additional sum-rate improvements is absent in planar arrays even when the online optimization is performed. In Fig. <ref>, we show this optimal sum-rate performance of the online optimization, obtained at the cost of added complexity as discussed in Section <ref>.





§ CONCLUSION

We have proposed a  novel joint resource optimization framework for a physically-consistent multi-band mMIMO system model where BS antenna spacing selection, precoder optimization, sub-carrier selection and power allocation is carried out simultaneously. Antenna spacing optimization allows to harness the favourable effects of mutual coupling to produce a wideband response in the mMIMO antenna array (even with simple individual antenna elements) which in turn allows a more efficient communication in the multi-band scenario. The proposed antenna spacing selection can be carried out either only once at the design stage or dynamically during the operation. We also have investigated the optimum sum-rate performance (obtained via varying the antenna element spacing) variation for different  power allocation and user selection schemes. Through extensive simulation results we have shown that the proposed framework performs the best with power allocation considering all users in a common pool over all frequency bands and sub-carriers. 

7

Jod21 R. Joda et al., “Carrier aggregation with optimized UE power consumption in 5G," in IEEE Networking Letters, vol. 3, no. 2, June 2021, pp. 61-65.

Jod21_2 R. Joda et al.,  “QoS-aware joint component carrier selection and resource allocation for carrier aggregation in 5G," in IEEE ICC 2021, 2021, pp. 1-6.

Nid22 Nidhi et al.,  “A study on cross-carrier scheduler for carrier aggregation in beyond 5G networks," AT-AP-RASC, 2022, pp. 1-4.

Fu13 W. Fu, Q. Kong, Y. Zhang and X. Yan,  “A resource scheduling algorithm based on carrier weight in LTE-Advanced system with carrier aggregation," WOCON, 2013, pp. 1-5.

Kib20 M. G. Kibria, E. Lagunas, N. Maturo, H. Al-Hraishawi, and S. Chatzinotas, “Carrier aggregation in satellite communications: Impact and performance study," in IEEE Open Journal of the Communications Society, vol. 1, 2020, pp. 1390-1402.

Chi21 D. Chikhale, S. Deosarkar, and M. Munde, “Carrier aggregation in 5G using millimeter range communication," 2021 IEEE GUCON, 2021, pp. 1-6.

Ken22 B. A. Kenney, A. J. Majid, H. Moradi, and B. F.-Boroujeny,  “Resource allocation for single carrier massive MIMO systems," IEEE ICC, 2022, pp. 950-955.

Ayu20 M. S. Ayub, L. Wuttisittikulkij, P. Adasme, and I. Soto, “Hybrid precoding design for two carriers aggregated in 5G massive MIMO system," 2020 SACVC, 2020, pp. 1-5.

Ali19 M. Y. Ali, T. Hossain, and M. M. Mowla,  “A trade-off between energy and spectral efficiency in massive MIMO 5G system," 2019 ICECTE,  2019, pp. 209-212.
Bog15
T. E. Bogale, L. B. Le, and A. Haghighat, “User scheduling for massive MIMO OFDMA systems with hybrid analog-digital beamforming," in IEEE ICC, 2015, pp. 1757-1762.

Abd23
A. Hussein, Operating multi-user transmission for 5G and beyond cellular systems,  Ph.D. Thesis, University of Waterloo, Canada, 2023.

Bal24 —,  “Radio resource management in downlink multi-band mMIMO systems," submitted to IEEE 10th WFIoT, 2024.

Kin18
S. Kingsly et al., “Multi-band reconfigurable filtering monopole antenna for cognitive radio applications," in IEEE Antennas and Wireless Propagation Letters, vol. 17, no. 8, Aug. 2018, pp. 1416-1420.

Akr23 M. Akrout, V. Shyianov, F. Bellili, A. Mezghani, and R. W. Heath, “Super-wideband massive MIMO," in IEEE Journal on Selected Areas in Communications, vol. 41, no. 8, Aug. 2023, pp. 2414-2430.

Ivr14 M. T. Ivrlac and J. A. Nossek, “The multiport communication theory," in IEEE Circuits and Systems Magazine, vol. 14, no. 3, 2014, pp. 27-44.

Chu48 L. J. Chu,  “Physical limitations of omni-directional antennas," in Journal of Applied Physics, vol. 19, no. 12, Dec. 1948, pp. 1163-1175.

Akr23_2 M. Akrout, V. Shyianov, F. Bellili, A. Mezghani, and R. W. Heath, “Achievable rate of near-field communications based on physically consistent models," in IEEE Transactions on Wireless Communications, vol. 22, no. 2, Feb. 2023, pp. 1266-1280.

Saa22 S. Saab, A. Mezghani, and R. W. Heath, “Optimizing the mutual information of frequency-selective multi-port antenna arrays in the presence of mutual coupling," in IEEE Transactions on Communications, vol. 70, no. 3, March 2022, pp. 2072-2084.

Hea19
R. Heath and A. Lozano, Foundations of MIMO Communication, Cambridge University Press, 2019.

Wei04
W. Yu, W. Rhee, S. Boyd, and J. M. Cioffi, “Iterative water-filling for Gaussian vector multiple-access channels," in IEEE Transactions on Information Theory, vol. 50, no. 1, Jan. 2004, pp. 145-152.

Cho13
E. Chong and S. Zak, An Introduction to Optimization, John Wiley & Sons Inc., 4th Ed., 2013.






 Paper Title: Towards Variable-Length In-Network Caching.















Towards Variable-Length In-Network Caching
    
Gyuyeong Kim 
 Sungshin Women's University

    
================================================

§ ABSTRACT 

We present StarCache, a new in-network caching architecture that can cache variable-length items to balance a wide range of key-value workloads.
Unlike existing works, StarCache does not cache hot items in the switch memory.
Instead, we make hot items revisit the switch data plane continuously by exploiting packet recirculation.
Our approach keeps cached key-value pairs in the switch data plane while freeing them from item size limitations caused by hardware constraints.
We implement a StarCache prototype on an Intel Tofino switch.
Our experimental results show that StarCache can balance highly skewed workloads with various key and value sizes. 




§ INTRODUCTION 

Key-value stores are the fundamental building blocks for online services owing to fast access to variable-length data <cit.>.
Workloads are generally read-intensive, and most items are hundreds of bytes <cit.>.
A challenge in scaling key-value stores is mitigating load imbalance among storage servers caused by skewed key popularity (e.g., trending events).
The load imbalance leads to the overloading of hot item servers, causing significant performance degradation <cit.>.

In-network caching is a promising load-balancing approach that builds a cache in the switch by leveraging programmable switch ASICs like Intel Tofino <cit.>.
The idea is to store hot items in the switch memory.
Requests read the item value by referring to the cache lookup table where the item key is the table index.
Compared to server-based caching, in-network caching provides much higher performance without extra cache nodes or accelerators thanks to a Tbps-scale processing throughput.

Unfortunately, existing solutions <cit.> can cache only tiny items due to hardware constraints.
Specifically, the key size is limited to 16 bytes due to the maximum match-key width of the match-action table.
A limited number of match-action stages and a small accessible byte size per stage make it difficult for the value size to exceed 128 bytes.
These numbers are far from typical workloads where key and value sizes are still small enough to be stored in a single packet, but larger than the limit <cit.>.
For example, in 54 Twitter workloads <cit.>, most keys are tens of bytes and many values are less than 1024 bytes.
However, the existing solutions cannot cache even a single item in 42 out of 54 workloads because the keys and values are generally larger than the limits.
Between the other 12 workloads, only 2 workloads have a portion of cacheable items larger than 50%.
This trend is similar in Facebook workloads as well <cit.>.
In this context, we ask the following question: how can we cache variable-length items in the programmable switch for balancing a wide range of key-value workloads?

We present , a new in-network caching architecture that can cache variable-length key-value items in the switch data plane.
Our idea is to make hot items keep visiting the switch data plane continuously in the form of cache packets, instead of caching the hot items in the switch memory.
We efficiently use packet recirculation, a built-in feature of the programmable switch ASIC.
This enables a packet to revisit the switch data plane through an internal loopback port without going outside. 
The switch can cache variable-length key-value pairs without being limited by the hardware constraints owing to circulating cache packets.
For cache serving, the switch maintains small request metadata in the switch memory and forwards the matched cache packet to the client if there is a pending request.
To support variable-length keys, we use key hashes for cache lookups and resolve hash collisions at the client by comparing the requested key and the returned key, which is contained in the cache packet.

The key insight behind the idea is that it is hard to overcome the memory access constraint of switch hardware if we stick to the approach of caching items in the switch memory.
For example, the idea of using key hashes for variable-length keys is difficult to realize in the existing architecture.
This is because the switch does not have enough hardware resources to store keys and values together.
In , cache packets contain both keys and values, allowing to handle hash collisions at the client.
Meanwhile, we may recirculate requests multiple times to read large values.
However, this limits scalability because the number of in-flight packets in the recirculation port increases proportionately to the number of requests.
In addition, the switch has tens of ports but only one recirculation port.
In ours, the recirculation port is not a performance bottleneck. This is because 1) requests are never recirculated; 2) the number of in-flight cache packets is small; 3) the switch can process multiple cache packets simultaneously with massive hardware-level parallelism, thus minimizing queueing between cache packets.

Realizing the idea imposes various technical challenges.
First, the switch should buffer multiple request metadata for cached items.
We design a request table as a circular queue structure using multiple register arrays. 
Our request table provides isolated data access among different keys, which means that requests for a key do not have to wait until requests for other keys are handled.
Second, a cache packet should serve multiple requests once fetched, rather than a single request.
To achieve this, the switch clones the cache packet before forwarding it to the client using the packet replication engine (PRE), a special module of the switch ASIC.
Next, we should ensure cache coherence.
We design an invalidation-based coherence protocol that can fetch new values and reply to the client simultaneously.
Lastly, the switch should deal with dynamic workloads where key popularity and value size change over time.
We design a switch control plane mechanism that considers dynamic workloads.

We have implemented a prototype on an Intel Tofino switch.

To evaluate , we build a testbed consisting of commodity servers, which are interconnected via an Intel Tofino switch.
Our experimental results show that provides load balancing for skewed workloads having diverse item sizes.

In addition, is robust to various workload conditions.

We also show that can handle dynamic workloads where key popularity changes over time.

In summary, this work makes the following contributions.


  * 
We design , an in-network caching architecture where the switch can cache variable-length key-value items to balance a wide range of workloads.


  * 
We propose several techniques to address various challenges in designing a custom switch data plane.


  * 
We implement a prototype on Intel Tofino switches and show the efficiency and robustness of through extensive testbed experiments.



The remainder of the paper is organized as follows.
In Section <ref>, we describe the motivation of this work.
Section <ref> provides the design of .
We present implementation and evaluation results in Section <ref> and Section <ref>, respectively.
We discuss related work in Section <ref>.
Lastly, we conclude our work in Section <ref>.


§ BACKGROUND AND MOTIVATION 



 §.§ In-Network Load-Balancing Caches

Balancing key-value stores with a small cache.
Today's online services rely on key-value stores since a user request may generate hundreds of key-value store requests.
In distributed key-value stores, balancing imbalanced loads between storage servers caused by different key popularity is a primary challenge.
Caching is a powerful technique to address the challenge, which is based on a theoretical foundation that we can balance loads for N servers (or partitions) by caching the O(Nlog N) hottest items, regardless of the number of items <cit.>.

Caching on commodity servers is a natural option to build the load-balancing cache <cit.>.
However, the performance of a cache node is not sufficient to handle many requests due to the limited packet processing throughput of CPUs.



Why in-network caching?
Building a load-balancing cache in the switch by leveraging programmable switch ASICs like Intel Tofino <cit.> is an attractive approach.
Unlike server-based caching, this approach can provide high-performance load balancing without additional nodes.
This is because the switch is highly optimized for packet I/O and can process a few billion packets per second, which is an order of magnitude higher than that of the CPU in servers.
The switch also provides a low packet processing delay within hundreds of nanoseconds.

Limited cacheable item size.
Existing works <cit.> demonstrate the efficiency of in-network caching, but they limit the cacheable item size.
Specifically, they support items of up to 16-byte keys and 128-byte values.
In the reconfigurable match table (RMT) switch architecture <cit.> like Intel Tofino, the switch data plane consists of n match-action stages <cit.>.
Each stage has a static memory and a few ALUs that can perform simple arithmetic operations on k bytes.
The existing works store the value of cached items across multiple stages after fragmentation, limiting the maximum value size to n× k bytes.
Unfortunately, n× k of the current switch is quite small, and the available number of stages for accessing the value is less than n since other non-caching functions also consume match-action stages.
The key size is also limited by the maximum match-key width of the match-action table.
This is because the existing works use a match-action table to implement the cache lookup table where the item key is the match key.


Why is it not enough?
Many key-value items are indeed small, but typically exceed the existing size limits. 
We have analyzed 54 Twitter workloads <cit.>, and observe that existing solutions are not enough to deal with typical workloads.
For example, only 3.7% of the workloads have over 80% of keys ≤ 16 B.
38.9% of the workloads have over 80% of values ≤ 128 B.
The existing works can cache less than 10% of items for 85% of the workloads because, to be cacheable, both key and value sizes must be within the limits.
Furthermore, they cannot cache even a single item for 77.8% of the workloads.
Facebook workloads <cit.> show similar size distributions.
The average key size is 27.1 bytes, and the median value size is 235 bytes.
These numbers exceed the size limits of the existing architecture.



 §.§ Variable-Length In-Network Caching

Design rationale.
We argue that it is hard to overcome the memory access constraint if we stick to the approach of caching data in the switch memory because the constraint is determined at the time of manufacture.
Therefore, we propose , a different approach to in-network caching.

Our high-level idea is to keep cached key-value pairs circulating within the switch data plane through packet recirculation, a built-in feature of the programmable switch that makes the packet revisit the switch data plane.
In our approach, cached items read requests rather than the requests read the cached items.
Specifically, clients submit requests, and the switch maintains small request metadata like the client IP address.
Cache packets, which are reply packets containing the key and the value of cached items, continue to revisit the data plane via an internal recirculation port while actively checking pending requests.
If a request for the key is found, the matched cache packet is forwarded to the client.
Otherwise, the cache packet is recirculated.
In Figure <ref>, we illustrate the difference between and the architecture <cit.>, which is the reference architecture of existing in-network caching solutions <cit.>.

Variable-length keys.
Our approach enables us to support variable-length key-value pairs.
For variable-length keys, one possible solution within the limited match-key width of the match-action table is to use a fixed-size key hash as the match key.
To handle potential hash collisions, we should compare the requested key and the key of the value that has been read.
To do this, the switch data plane should also store the item keys alongside values.
Unfortunately, it is hard to realize in the existing architecture because there are not enough match-action stages to store both keys and values.
can realize the idea of using key hashes since the switch data plane maintains cached key-value pairs in the form of circulating reply packets.
Therefore, the client can get the correct value from the storage server if the requested and returned keys differ.

Variable-length values.
For variable-length values, a possible approach in the existing architecture is recirculating requests.
Specifically, requests can access the switch memory multiple times using packet recirculation.
However, this approach is not scalable because the number of in-flight packets per second in the recirculation port increases in proportion to the number of requests.
For example, if every request is recirculated 8 times to read a 1024-byte value, the effective throughput of the recirculation port is reduced to 1/8 of the bandwidth.
Unfortunately, a pipeline in the programmable switch has only one internal recirculation port, while there are tens of regular front ports.
This means that the recirculation port can limit the performance.

In , the recirculation port is not a bottleneck.
The switch never recirculates requests as they are in the memory until cache packets read them.
We recirculate only a small number of cache packets.
The time to recirculate and process a cache packet is a few hundred nanoseconds like normal packets.
Furthermore, the switch can process multiple packets simultaneously thanks to hardware-level parallelization, which reduces queueing between cache packets.

Trade-off.
There is a trade-off of using recirculation to achieve variable-length in-network caching.
Although we are free from size limitations, we should reasonably limit the number of cache entries to avoid long request wait times.
Otherwise, the switch may not have enough space to store requests due to many pending requests, or the request latency would increase excessively even if it is stored.
This is because, although the switch can process many packets simultaneously, the queueing between cache packets may still happen due to many cache packets beyond the capability of switch hardware.
Fortunately, caching a small number of hot items is enough to balance skewed workloads <cit.>.


Technical challenges.
To balance the trade-off and translate the idea into a working system, we should address several technical challenges as follows.


  * 
The switch should maintain multiple requests, especially for the same cached item.
If not, many requests for cached items would be forwarded to the server as they cannot be stored in the data plane.



  * 
A cache packet should serve multiple requests once fetched.
Otherwise, the switch must fetch the cache packet from the server again, degrading performance.


  * 
We should design a cache coherence mechanism between the switch and storage servers.
If not, a request may read a stale value for the requested key.


  * 
The switch should adapt to changes in the workload, such as key popularity.


 



§ DESIGN 



 §.§ Architecture

Figure <ref> shows the overview of the architecture.

Switch data plane.
The switch data plane in consists of several custom tables and modules as follows.


  * 
Lookup table is for cache lookups, which is a match-action table that uses a hash of item keys as the match key.
The table returns a table index needed to access other tables and modules.
Cache entries are managed by the controller in the switch control plane.


  * 
State table, implemented as a register array, maintains the validity of the value of cached items.
This is needed to prevent read requests from obtaining the stale value when there are pending writes for the requested key.


  * 
Request table, implemented with multiple register arrays and registers, stores request metadata like client IP address, L4 port number, and request sequence number.


  * 
Key counters consist of two registers and one register array.
The key popularity counter is a register array that tracks the key popularity for each key.
The cache hit counter and the overflow request counter are registers that track the total number of cache hits and the total number of overflow requests for all cached keys.
The controller uses these for cache sizing.


  * 
Cloning module comprises a few match-action tables for cloning reply packets.
Packet cloning is done by the PRE, a special hardware module in the switch ASIC <cit.>.
The switch clones packets with negligible overhead since the switch duplicates only a packet descriptor pointing to the packet instead of the whole packet.



Meanwhile, the switch invokes the custom processing logic only for packets by referring to reserved L4 ports.
We use UDP to handle messages for better latency like existing works <cit.> while using TCP for top-k item reports in cache updates.
For normal packets, the switch only applies the traditional packet forwarding logic.

Switch control plane.
The controller in the switch control plane performs cache updates and cache sizing.
It evicts the least popular keys and inserts new hot keys based on server-side periodic top-k reports for uncached keys and the switch-side report for cached keys.
Note that while the controller handles key insertion, fetching values is done via the switch data plane.
The controller also adjusts the cache size dynamically to deal with dynamic workloads.
It also handles switch configurations.

Clients and servers.
Clients should specify the operation type, the item key to request, and the key hash.
Storage servers run a server application that acts as a shim layer that translates messages to API calls for key-value stores and vice versa.
The server returns replies for read and write requests like regular storage servers.
However, in cases of write requests for cached items, the server returns a write reply with the item value to fetch the latest value.




 §.§ Packet Format

Figure <ref> depicts the packet format.

The switch only parses the header.
The payload of message consists of the key and the value.
Our header is 22 bytes.
Therefore, supports a key-value pair of up to 1438 bytes for a single packet.
For example, the switch can cache an item with a 16-byte key and a 1422-byte value.
Our header fields are as follows.



  * 
 (1 byte): the operation type, which can be  (Read request),  (Write request),  (Read reply),  (Write reply),  (Fetch request),  (Fetch reply), and  (Correction request).



  * 
 (4 bytes): a sequence number as request IDs for handling hash collisions.


  * 
 (16 bytes): the key hash as cache lookup index.


  * 
 (1 byte): a versatile flag field 1) to distinguish write requests for cached items from those for uncached items; 2) to store the number of packets comprising a multi-packet item.





 §.§ Basic Request and Reply Processing


In Figure <ref>, we illustrate the packet processing logic.


Request generation at clients.
To send a request, clients should specify the  and  fields.
The  field is increased by one for every request, which is used to resolve hash collisions.

As we consider hash-partitioned storage, the destination storage server is determined by hashing the key.

Read requests.
The switch first refers to the lookup table using a key hash to get a table index, which is used to access the slots in other tables.
If missed, the switch forwards the packet to the server as the request is for an uncached item.
In case of a cache hit, the key popularity counter and the cache hit counter are incremented by one.
This key popularity is collected by the controller in the switch control plane periodically for cache updates.
Next, the switch checks the validity of the requested key by looking into the state table.
The state is binary: valid or invalid.
Being invalid means that there are pending write requests for the key.
In this case, the switch forwards the read request to the server to avoid reading stale item values.
If the key is valid, the switch checks the request table to see whether there is a free slot.
The switch puts the request metadata into the table when a free slot is found.
Otherwise, the request is destined to the server after the overflow request counter is increased.
Request metadata includes the client IP address, L4 port number, and  as request IDs.
After insertion, the switch drops the packet.
This is acceptable since a cache packet will soon service the stored request.

Read replies.
When receiving a read reply, the switch first checks to see if the ingress port is the recirculation port.
If it is, the reply is a cache packet; otherwise, it is a reply for an uncached item sent by the server.
For cache packets, if the cache misses or the state of the value is invalid, the switch drops the packet.
A cache miss for a cache packet means that the controller evicted the key from the lookup table due to a change in key popularity or the cache size.
The invalid state indicates a write request with a new value is in progress.

The switch goes through the lookup table and the state table, and then looks for pending requests in the request table.
If a request is found, the switch forwards the cache packet to the client after updating the header with metadata and removing the metadata from the table slot.
Here, to make the cache packet serve more requests, the switch clones the packet before packet forwarding.
The switch forwards the original packet to the client and the cloned one to the recirculation port.
The switch also recirculates the packet if there are no pending requests in the table.

Write requests.
The first switch checks whether the requested key is in the cache lookup table.
If it is, the value for the key is invalidated to prevent reading the outdated value.
In addition, the switch sets the  field to 1 so that the storage server can append the value in the write reply.
Regardless of a cache hit, the switch forwards the request to the storage server to update the value of the key in the server.

Write replies.
If the key is cached, the switch validates the value to allow read requests to get the latest value.
Similar to reads, the switch clones the packet so that the client receives the write reply while the switch has a new cache packet simultaneously.
The switch updates the  field to  of the cloned packet after the first recirculation since cache packets are read replies.
For a cache miss, the switch forwards the packet to the client since the reply is for an uncached item.

Other types of messages.
Fetch requests, fetch replies, and correction requests are delivered to the server as standard packets.
The fetch reply is forwarded to the controller, who has sent a fetch request for cache updates. 






 §.§ Buffering Request Metadata

Circular queue-based request table.
We should store multiple request metadata for cached items because the requests should wait until cache packets serve them.
The switch should be able to buffer concurrent read requests for the same cached item.
If not, many requests for the cached item would be forwarded to the storage server due to the lack of vacancies.
Furthermore, a request for a key should be isolated from other requests for other keys since the request may undergo excessive queueing delay due to the different requests.

We design a request table based on a circular queue structure.
Our request table provides a logical queue for each cached key and the queue can be accessed in O(1) by efficient indexing.
This provides fast and isolated queue access for a key regardless of other keys.
The request table consists of 6 register arrays: the first three arrays are for storing request metadata for each key (i.e., client IP address, request sequence number array, and L4 port number).
The other three arrays are for managing queue operations.
The queue length array maintains the number of stored request metadata for each key.
The front pointer array handles dequeue operations by tracking the first request metadata for each key.
The rear pointer arrays perform enqueue operations by keeping track of the last request metadata for each key.

The queue management arrays are indexed using a table index (CacheIdx) returned by the cache lookup table.
The request metadata arrays are accessed via a request index, which is defined as ReqIdx = CacheIdx× S + i where S is the maximum queue size per key and i is the offset for ith slot in the logical queue for a key and is given by the pointer arrays.
The switch uses three match-action stages for a request table.
Specifically, the switch first checks the queue status (Stage 1), and performs en/dequeue operations if the queue is not empty or not full (Stage 2).
Next, the switch gets/puts metadata (Stage 3).

An example for table operations.
Figure <ref> shows a simple example of table operations.
We assume that the request table can maintain up to 4 requests for each key.
We consider enqueueing and dequeueing operations using read requests and cache packets, respectively.
In stage 2, the rear pointer changes to 0 from 3 since we implement a circular queue.
It is also easy to see that the request metadata for different keys does not collide since we partition the metadata arrays using the indexing formula of ReqIdx=CacheIdx× S + i.



 §.§ Cache Serving

Handling multiple requests via packet cloning.
In , cache packets pass through the switch data plane repeatably to serve pending requests for cached items.
One challenge is serving multiple requests with a cache packet fetched only once because the cache packet is forwarded to the client after updating the header using the retrieved metadata.
A strawman is to fetch the cache packet from the server again, but this approach is inefficient as the switch cannot serve pending requests for the key until the fetching is completed.
Fetching multiple copies of the cache packet may be a solution, but this incurs excessive queueing delay between cache packets. 
In addition, it is hard to know how many cache packets should be fetched because we cannot predict the number of requests for each cached item in advance.



To address this, we utilize packet cloning, another built-in feature of the programmable switch ASIC besides packet recirculation.
Packet cloning is done by the PRE, a hardware module specialized for packet cloning in the switch ASIC.
Cloning has a negligible overhead for the following reasons.
First, the PRE is located after the ingress pipeline.
This means the switch does not repeat the ingress pipeline processing for the cloned packet, not causing extra ingress processing delay.
Second, the switch does not copy the entire packet.
It only copies the small descriptor pointing to the memory location of the packet and reuses the packet.

We use multicast to forward the original and clone packets.
Specifically, through a match-action table using the destination IP address acquired from the request table, the switch gets a multicast group ID that specifies the regular port number directed to the client and the internal recirculation port number.
The switch forwards the original packet to the client and the cloned one to the switch data plane again, making it serve more requests.

An example of cache cloning.
In Figure <ref>, a cache packet gets the client IP address of 10.0.1.102 and other metadata from the request table.
The packet obtains the multicast group ID as 2 using the destination IP address (i.e., the client IP address).
The cache packet then gets a pair of destination port numbers.
52 is the port number directed to the client, and 68 is the recirculation port number.
After that, the switch clones the packet and forwards the original and cloned ones to each port.





 §.§ Handling Hash Collisions


The existing works use the item key as the match key of the cache lookup table, which is implemented as a match-action table.
Since the match-key width is limited by hardware, we cannot use the key exceeding the size limit.
We may use a register array, but the size limitation in the register index is stricter than in the match-action table.
Therefore, we use the fixed-sized key hash as the match key.
A challenge is how to resolve potential hash collisions.
We should handle this because a request may read the wrong value of other keys.

Client-side collision resolution.
We handle hash collisions at the client level by maintaining a list of the keys for each request that has not yet received a reply.
The list is indexed by pkt.seq, a request ID.
Request packets contain both the original key and the key hash since we should use the original key to get the value in servers.
Upon receiving read replies, the client checks whether the requested key in the list and the returned key in the reply header are identical.
If different, the client sends a correction request to the storage server.
The switch then forwards the request without applying the cache logic so that the client gets the correct item value from the storage server.
One issue is that requests for uncached but hash-collided keys always undergo this process.
However, this is uncommon since our 128-bit key hash provides a low collision probability. 
In our experience, hash collisions only occur occasionally during cache updates.

An example of resolving hash collisions.
We plot an example of handling hash collisions in Figure <ref> where the requested key is  but the returned key is .
Upon receiving the reply, the client detects that the retrieved value is a wrong value by referring to the key list using pkt.seq=1.
The client then sends a correction request to the storage server.
The switch bypasses the cache logic, and forwards the packet to the server.
The storage server returns the value as a read reply, and the client finally gets the correct value for the key .
The client removes the key from the list.
pkt.seq is finite and wraps around if it reaches the maximum value.



 §.§ Cache Coherence

To ensure cache coherence between the switch and servers, we design an invalidation-based coherence protocol illustrated in Figure <ref> (c) and (d).
The switch invalidates the value when handling a write request for a cached item and revalidates the value upon receiving a write reply.
Ideally, the outdated cache packets should be replaced with the latest cache packet.
However, requests may be returned with the stale value since the outdated packets are still circulating in the switch data plane even if the item value becomes invalid.
To address this, the switch drops the cache packet if the item is cached but its value is invalid.
Owing to the fact that the cache packet is dropped before accessing the request table, we can prevent read requests from retrieving the stale value until a new cache packet is fetched.

In , the server sends a single reply packet, and the switch updates the value and replies to the client simultaneously by cloning the packet.
Note that the existing architecture <cit.> sends two different packets for updating the value and replying to the client, which degrades write performance.








 §.§ Cache Updates

can handle dynamic workloads by tracking workload changes as follows.

Dealing with key popularity changes.
To handle key popularity changes over time, we update cache entries based on periodic popularity reports from the switch and storage servers.
In particular, we count the popularity of cached keys in the switch data plane using the key popularity counter.
The controller keeps track of key popularity by reading the counter periodically.
Meanwhile, storage servers periodically report the top-k keys to the controller, which are popular uncached keys.
The servers use a count-min sketch with five hash functions to track key popularity in a memory-efficient manner while ensuring accuracy.
To reflect the recent status only, we reset all the counters to zero after reporting.

If cache entries need to be updated due to a change in key popularity, the controller removes the victim key from the cache lookup table and inserts a new popular key instead.
The controller then sends a fetch request to the storage server containing the latest value for the key.
Upon receiving the request, the server fetches a new cache packet to the switch data plane and replies to the controller.

Dynamic cache sizing.
Since uses recirculation, it is important to avoid the queueing between cache packets.
If not, many requests for cached items may be forwarded to the storage server since they did not find a free slot in the request table where cache serving is delayed due to the queueing between cache packets.
Although we try to set the best cache size at first, the cache size may cause a negative effect as the workload changes over time.

To address this, the controller dynamically changes the cache size to maximize the performance while avoiding the adversarial effect.
The controller calculates the request overflow ratio, which is defined as the number of requests for cached items that were sent to the server divided by the total number of requests for cached items.
The controller compares the ratio with a pre-defined threshold (e.g., 1%).
If the ratio exceeds the threshold, the controller decreases the cache size by one.
Otherwise, the cache size is increased by one.



 §.§ Handling Practical Requirements

Multi-packet items.
Many key-value items are less than the MTU size.
However, some items may exceed the MTU size, such as articles and photo objects (e.g., Wikipedia and Flickr workloads <cit.>).
To cache multi-packet items, we fetch multiple cache packets with fragmented values for the same key. 
We maintain the number of forwarded cache packets for each item by placing another register array alongside the request table, the ACKed packet counter.
The initial value of each slot is 1 since most items are single-packet.
When fetching item values, the storage server puts the number of packets comprising the item in the  field.
When handling cache packets, the switch retrieves metadata as usual, but it does not manipulate the slots of all register arrays in the request table if the slot value in the ACKed packet counter is not equal to the value in .
Instead, the switch increases the slot value by one.
When the slot value in the counter equals , the switch removes request metadata as it is ready to be finished.

However, this may reduce the performance gains since has to limit the cache size to avoid queueing between cache packets, and multiple cache packets occupy the cache for one large item.
Therefore, the balancing efficiency may decrease if many items consist of multiple packets.
This can be mitigated if the bandwidth of the recirculation port increases in next-generation switch hardware.


Failure handling.
For packet loss, we can use an application-level mechanism.
For example, our controller uses UDP with a timeout-based mechanism to send fetch requests and receive fetch replies.
Storage servers use TCP to report top-k items to the controller.
Meanwhile, server failures do not cause a -specific problem.
Switch failures result in the loss of cached items, but the cache can be reconstructed quickly after the switch is recovered.


Multi-rack deployment.
can scale to multiple racks as items of servers are only cached in the ToR switch.
For example, assume that we have rack 1 and rack 2 where ToR switches, ToR1 and ToR2, are interconnected via a spine switch SPN.
When CLI, a client in rack 1, sends a request to SRV, a server in rack 2, the path is CLI-ToR1-SPN-ToR2-SRV-ToR2-SPN-ToR1-CLI.
If the requested item is cached, ToR2 is the only switch that applies the cache logic.
There is no issue even if rack 1 is the replicated rack of rack 2 where the two ToR switches have the same cache entries because the item is returned by ToR1.







 §.§ Discussion




Write-back caching.
Although most key-value workloads are read-intensive, some workloads are write-intensive.

Since we use write-through caching that updates the switch and the server simultaneously with invalidation, our performance gain decreases as the write ratio grows.
FarReach <cit.> is a recent work that enables write-back caching.
Although it still cannot serve many workloads due to the size limitation, it provides high performance regardless of the write ratio.
The difference between write-through caching and write-back caching is whether a write request for cached items updates the storage server or not.
Therefore, can use write-back caching as well by letting the switch return write replies upon receiving write requests after updating the cache only.
However, the switch with Intel Tofino 1 does not have sufficient stages to implement other key modules like snapshot generation alongside the modules.
We expect that we may fully implement write-back caching in using Tofino 2 <cit.> whose the number of stages is much higher than Tofino 1 by roughly double, but this is beyond the scope of this work.


Multi-pipeline deployment.
The multi-terabits performance of the programmable switch ASIC comes from the multi-pipelined architecture.
Each pipeline consists of tens of regular ports and one internal recirculation port.
In a switch with 4 pipelines and 64 front ports, each pipeline has 16 ports and one recirculation port statically.
The switch resources are partitioned, and metadata and memory data are not shared between the pipelines.
Therefore, the pipelines of client-directed ports and server-directed ports should be the same.
Otherwise, cache packets in a pipeline may not read requests since they are in another pipeline.
This can be achieved by mapping each pipeline of the ToR switch to a client-directed port on the spine switches, which is feasible only when the clients and servers are located in different racks.

We expect this to be mitigated with a new programmable switch architecture like MP5 <cit.> that achieves high performance with a logical single pipeline.

We describe how can ensure cache coherence between switch pipelines.
First, the switch forwards write requests to the recirculation ports of other pipelines and the server.
In the other pipelines, the switch drops the request after invalidation as the request has already been delivered to the server in the source pipeline.
Meanwhile, write reply packets are cloned in the PRE to fetch the new cache packet for all pipelines.
The switch forwards the original reply to the client while recirculating the cloned reply via the recirculation ports of all pipelines.
We have implemented this and validated the functionality and performance, but do not include this in the current design.
This is because write requests for cached items in a pipeline should be propagated to the other pipelines to invalidate the item value via the recirculation ports, and this may make the port a bottleneck.

  


§ IMPLEMENTATION

Client-server application.
We develop an open-loop application in C using NVIDIA Messaging Accelerator library (VMA) <cit.> that reduces host packet processing time.
This enables the processing of packets in user space by implementing native RDMA verbs and bypassing kernel network stacks.
The client application measures throughput and latency by generating requests.
The time gap between consecutive requests follows an exponential distribution.
The server application has multiple threads where each thread is pinned to a disjoint CPU core.
To emulate multiple storage servers, we assign a partition per thread so that each thread acts as an independent server.
This technique is used in  <cit.> as well.
In a similar vein, we implement a key-value store with TommyDS <cit.>, a high-performance hash table library.


Switch.
We implement the switch data plane in P4_16 <cit.> for Intel Tofino <cit.>.
We use Intel P4 Studio SDE 9.7.0 to compile the switch data plane.
Our prototype uses 10 match-action stages and 12.38% SRAM.

The request table has 64K slots that can maintain up to 4K cached keys with a maximum queue size of 16 for each key.
The controller is written in Python 3.
In our prototype, we use an additional register array for the request table to store the timestamp of the request for latency measurement.
The header also has 3 extra fields of 1-B , 4-B , and 1-B .
The first two fields are required to separately measure the latency of requests for cached and uncached items.
The last field is to store the partition ID as we emulate multiple storage servers as dedicated threads in a physical node.


§ EVALUATION






 §.§ Methodology

Testbed setup.
We build a cluster consisting of 8 nodes, which are connected by an APS BF6064X-T switch with the Intel Tofino 1 <cit.>.

The servers are equipped with a 10-core CPU (Intel i5-12600K @ 3.7 Ghz), 32 GB of DDR5 memory, and a 100GbE NVIDIA CX-5 NIC.
The servers run Ubuntu 22.04 LTS with Linux kernel 6.5.0.
The 5 nodes act as clients to generate requests, and the remaining 3 nodes are used to emulate multiple storage servers.

Compared schemes.
We compare our work with and  <cit.>.
is a mechanism without any cache logic.
is the representative in-network caching architecture.
FarReach <cit.> and DistCache <cit.> also adopt the architecture.
We implement the core caching logic of , but our implementation provides items up to 64-byte values across 8 stages with an 8-byte accessible size per stage.
We find that the P4 compiler allocates only two cache read tables per stage, even with pragma statements.
We suspect that this is due to compiler restrictions on our code.
We clarify that this does not mislead the conclusions of our experiments, as there is a negligible difference in the latency required to read 64 bytes and 128 bytes at line rate using the same number of match-action stages.

Workloads.
We emulate 36 storage servers using 12 partitioned threads per node.
We basically consider a workload with 10M key-value pairs whose key popularity skewness follows a Zipfian distribution with α=0.99, since it is regarded as typical skewness <cit.>.
Although both key and value sizes impact whether an item is cacheable by , we use fixed 16-byte keys by default for simplicity.
Instead, we represent the portion of cacheable items using different ratios between 64 bytes and 1024 bytes values.
The 64-byte value represents a cacheable item value of .
The 1024-byte value is a typical-sized item value in many workloads and is considered in many research papers <cit.>.
We use a bimodal distribution with 12% 64-byte and 88% 1024-byte values by considering the  workload of Twitter <cit.>.
In the workload, 12% of values are ≤128 B and 87% of values are >128 and ≤1024 B.
1% of values are between 1024 B and 1280 B, but we consider them 1024 B for simplicity.
Most experiments are for read-only workloads like  as we target read-intensive workloads.

Except for dynamic workloads, we try to preload 10K and 32 of static hottest items for and , respectively.
32 is a nearly optimal cache size for with the given workload.
Since can cache only 12% of items in the workload, the actual number of cached items is 1.2K.
We choose 12% of keys among the 10K hottest keys with a uniform distribution. 
We store the chosen keys as a text file to make experimental results consistent.
The controller loads the file and puts the keys to the switch cache.







 §.§ Main Results

Throughput with different key access distributions.
We measure the throughput with different skewness and plot the results in Figure <ref>.
 (switch) is the throughput of the cache in the switch.
In other words,  (servers+switch) is the total throughput, the sum of server throughput and switch throughput.
We can see that provides high throughput regardless of skewness, unlike the others whose throughput decreases as the workload becomes more skewed.
For example, provides high throughput even for highly skewed workloads with Zipf-0.99.
does not provide high throughput as expected since many hot items are not cacheable.
In the Zipf-0.99 workload, has higher throughput than and by 1.46× and 1.39×, respectively.

Throughput on individual servers.
We plot the load on individual servers in Figure <ref>.


We can see that and do not balance loads well.
This is due to the lack of caching logic for and many uncacheable hot items for .
However, can balance the loads between the servers sufficiently since can cache variable-length items.


Latency.
We measure the latency by varying target throughput.
Figure <ref> shows the median and the 99th percentile latencies as a function of throughput.
Since there is no cache logic, does not provide low latency with limited throughput.
is better than , but still does not provide the best performance.
On the other hand, provides low latency in both the median and 99th percentile latencies while providing the highest throughput. 

Impact of write ratios.
To investigate the performance of in various workloads, we measure the throughput by varying the write ratios.
Figure <ref> reports the throughput as a function of write ratios.
As expected, the throughput of decreases as the write ratio grows.
This is because the switch invalidates the cached key when handling a write request for a cached item to ensure cache coherence.
The switch cache provides benefits for only read requests, the throughput is decreased with higher write ratios since read requests for invalid cached items are forwarded to storage servers.
With the 100% write ratio, the throughput of converges to since the cache does not provide any benefit.
Meanwhile, similar to , offers reduced throughput as the write ratio grows since it also invalidates the cached key if there is a write for the key.







Impact of item size.

Figure <ref> shows the throughput with different key sizes and value sizes.
In Figure <ref> (a), we can see that the throughput of decreases as the key size grows.
This is because larger keys consume more resources in storage servers and the switch.
The latency to recirculate and process a cache packet especially increases, though it still remains at a few hundred nanoseconds.
and also show similar results.


The results with different value sizes in Figure <ref> (b) are similar to those with different key sizes.
shows the best performance except in a case with 64-B values.
provides the highest throughput for the case since all items can be cacheable.









 §.§ Deep Dive

Impact of number of servers.

Figure <ref> shows the individual throughput of 9, 18, and 36 storage servers.
We also present the result with 36 servers for the uniform workload as the baseline.
We can see that can balance loads sufficiently regardless of the number of servers.
does not achieve perfect load balancing.
This is because there are trade-offs to achieve variable-length caching, like limited cache size and extra latency caused by packet recirculation.




Impact of cache size.

Figure <ref> plots the throughput breakdown of with different cache sizes and the corresponding overflow request ratios.

We can see that the total throughput increases as the number of cached entries grows.
However, the throughput is saturated around 16-32 cached items.
The throughput of the switch cache is also saturated as well.
Figure <ref> (b) clarifies the reason.
From 64 cached items, the overflow request ratio rapidly increases, where the overflow requests indicate the requests for cached items forwarded to the storage servers due to the lack of free slots in the request table.
Although the request table can maintain 256 request metadata for each key simultaneously, the slots get full quickly if the queueing between cache packets occurs due to too many cache packets.












Latency breakdown.
Figure <ref> (a) and (b) plot the median and the 99th percentile latency breakdowns, respectively.
For the median latency, we can see that there is no distinct difference between and for servers.
However, for , the latency of requests handled by the switch is slightly higher than by 1 
 μ s.
This is because requires requests to wait until circulating cache packets handle them.
The difference in the caching logic between and results in more different tail latency as throughput grows.
In Figure <ref> (b), we can see that the tail latency of requests handled by the switch cache in gradually increases as throughput grows, unlike showing a deterministic latency even if throughput is saturated.
This is because the queueing delay between requests for the same cached item occurs when the throughput gets closer to the saturated throughput.






Impact of multi-packet items.
Figure <ref> shows the performance of by considering that all cached items are multi-packet items.
For one packet, we use a 1416-B value.
For example, a 4-packet item has 16-B key and 5664-B values.
We can see that generally balances loads even with multi-packet items.
We observe that the balancing efficiency slightly decreases as the value sizes grow.
This is because we decrease the cache size by considering that multiple cache packets take up the cache for a single item.

Impact of cacheable item ratios for .
Figure <ref> shows the throughput and the balancing efficiency of and with different ratios of 64-B values.
This is to inspect whether is better than when it can cache more items.
The balancing efficiency is defined as LowSrv/HighSrv where LowSrv is the storage server where the throughput is the lowest between the storage servers, and HighSrv is the storage server with the highest throughput (i.e., the server with the hottest item).
We use a near-optimal cache size for .
The number of cached items is 32 for the ratios between 0-60%.
We use 64 cache entries for 70-90%.
For the ratio of 100%, we cache 128 hot items.
Figure <ref> (a) shows mixed results.
outperforms until 50%, but is beaten by after 60%.
We note that the throughput of in 80-100% is limited by the Tx throughput.
However, does not balance loads sufficiently.
In Figure <ref> (b), we see that has poor balancing efficiency even if only 10% items are uncacheable.
Unlike , can balance loads across all the ratios.
We recall that the number of workloads where the portion of cacheable items for is higher than 50% is only 2 between 54 workloads <cit.>.



Handling dynamic workloads.
We investigate the performance of with dynamic workloads.
We use a hot-in pattern, which is the most radical workload change.
Specifically, every 10 seconds, the popularity of the 256 coldest items and the 256 hottest items is swapped.
This means that all cache entries should be replaced.
These sudden popularity changes are not common in practice, but this enables us to evaluate the robustness of to dynamic workloads.

Figure <ref> (a) shows the throughput during 60 seconds.
We can see that the throughput decreases when item popularity changes but recovers within a few seconds.
This is because the controller in the switch control plane quickly updates the cache entries and fetches cache packets based on the top-k report of servers and the cache popularity counter of the switch data plane.
Figure <ref> (b) plots the change in the cache size over time with a given threshold of 0.05% and a minimum cache size of 32.
The controller adjusts the cache size dynamically by monitoring the overflow request ratio.
We can see that the cache size fluctuates between 32 and 36, hence the switch can maintain the overflow request ratio within 0.05%.





 


§ RELATED WORK

In-network caching.
We have witnessed a surge of domain-specific acceleration through in-network computing (e.g., consensus <cit.>, coordination <cit.>, machine learning <cit.>, 
 replication <cit.>).
Storage systems are major target domains of recent works (e.g.,  <cit.>).
 <cit.> is an early work that has a great impact not only on in-network caching but also on the entire field of in-network computing.
It shows the potential and limitations of the programmable switch as an in-network accelerator for distributed systems.

DistCache <cit.> resolves inter-rack load imbalance and FarReach <cit.> realizes write-back caching, but they can still cache a limited size of items.

enables variable-length in-network caching by leveraging built-in features of the programmable switch while sacrificing the cache size reasonably.

Other alternatives.
SwitchKV <cit.> employs the OpenFlow switch for cache lookups while items are cached in cache nodes, which provide limited performance.
Pegasus <cit.> is a selective replication solution where the switch replicates hot items across storage servers for load balancing, functioning similarly to caching.
Unfortunately, its performance is limited by the aggregate throughput of storage servers.
provides high-performance caching beyond the aggregate throughput with no extra hardware costs. 


§ CONCLUSION

We proposed , an in-network caching architecture that is capable of variable-length caching in programmable switches.
The key idea of is to make cached key-value pairs revisit the switch data plane continuously by leveraging packet replication efficiently.
We addressed technical challenges to realize the idea of .
We have implemented a prototype on an Intel Tofino switch, and experimental results demonstrated that can balance skewed workloads with diverse sizes of items.
We believe that is a fine example of overcoming fundamental switch hardware constraints that put limits on application semantics, which are major hurdles of in-network computing research.  


§ ACKNOWLEDGEMENT

The author thanks Yeon-sup Lim for providing constructive feedback on the earlier draft of the paper.


plain




 Paper Title: Kuramoto oscillators in random networks.



A Simple Low-bit Quantization Framework for Video Snapshot Compressive Imaging
    Miao Cao1,2 Lishun Wang2Huan Wang2Xin Yuan2()
    August 5, 2024
==============================================================================







§ INTRODUCTION 

The phenomenon of collective synchronization of an ensemble of interacting entities can be observed in different areas. For example in biology, in processes involving the regulation of the cell cycle <cit.>; in physics, in the study of coupled lasers<cit.> and Josephson junctions<cit.>; but also in human <cit.> and ecological networks <cit.>. One of the first important works on the subject was written by A. Winfree <cit.> in the 1960s who, studying a population of interacting oscillators, discovered that if the natural frequencies of the oscillators were very close, there existed a threshold condition for the coupling constant that, if exceeded, caused a phase transition in which some oscillators suddenly synchronized. Winfree's ideas were taken up by Kuramoto <cit.> who proposed a model whose analytical solution could be studied in the mean-field approximation. The equations of dynamics, for a collection of N oscillators, are:


    θ̇_i = ω_i + K/N∑_j=1^N(θ_j - θ_i) .


The temporal evolution of the phase θ_i of each oscillator depends on its natural frequency ω_i, and on the phases of the other oscillators with which it interacts. If ω_i = ω_j   ∀ i,j the model is called homogeneous, otherwise inhomogeneous. The magnitude of the interaction is given by the coupling constant K ≥ 0, which is the same for all oscillators. In order to have a geometric representation of the dynamics, it is convenient to imagine that the motion occurs on a circle of unit radius so that the phase of an oscillator is represented by the azimuthal angle defined by its position, see Fig. <ref>. 



Collective motion can be studied by introducing the order parameter


    R e^i ψ = 1/N∑_j=1^N e^i θ_j .


The modulus 0 ≤ R ≤ 1 measures phase coherence. If the oscillators are scattered along the entire circle then R ≈ 0  while if they are in phase coherence (θ_i ≈θ_j, ∀ i,j) then R ≈ 1. The angle ψ is the average phase. If K = 0  the oscillators decouple and the phase of each of them evolves according to the equation θ_i(t) = ω_i t + θ_i(0) and synchronization, that is θ̇_i = θ̇_j = θ̇ ∀ i,j, can only occur if ω_i = ω_j ∀ i,j. Note that by summing the members of the Eq.( <ref>) we obtain ∑_i=1^Nθ̇_i = ∑_i=1^Nω_i, which implies that if all the oscillators synchronize they will have a frequency equal to the average of the natural frequencies: θ̇= ω̅= N^-1∑_i=1^Nω_i. In terms of the order parameter Eq.( <ref>) can be written as:


    θ̇_i = ω_i + K R  (ψ - θ_i)


where it is evident how the single oscillator can be thought as interacting with a mean-field with effective coupling constant equal to K R and phase ψ.
The numerical analysis of the Kuramoto model shows the existence of a threshold value of the coupling constant below which the oscillators evolve while remaining in a state of phase incoherence; gradually increasing K beyond this threshold the incoherent state becomes unstable, a small nucleus of oscillators is formed which synchronize at a frequency equal to ω̅ while the remaining part moves non-uniformly around the unit circle; finally, if K reaches a critical value K_c, collective synchronization begins. Kuramoto proved that K_c = 2/π g(0) in the limit  N →∞, where g(ω) is the distribution of natural frequencies, see <cit.> for details.

The Kuramoto model refers to a fully (all-to-all) connected network. However, it is interesting to study collective synchronization also in the case of heterogeneous networks, because of this kind are those of particular importance in nature, social interactions and technological infrastructures. Considering the network topology, Eq.( <ref>) becomes:



    θ̇_i = ω_i + K/k_i∑_j=1^N A_ij(θ_j - θ_i)


where A_ij is the adjacency matrix (A_ij = 1 if θ_i and θ_j are coupled, 0 otherwise) and k_i is the number of connections of oscillator θ_i. In this paper, only connected and symmetric networks (A_ij = A_ji) are considered.


As far as we have been able to ascertain, there is no general theory that predicts for each type of network the value of the critical threshold K_c for the onset of collective synchronization and, therefore, it is necessary to resort to numerical analysis. In some cases the system goes from a state of phase incoherence to one of coherence as soon as K > K_c and it can be said that it undergoes a phase transition <cit.>.

In this paper we focus on the study of the Kuramoto model for random networks. This type of networks are extensively studied both for their theoretical interest and for their statistical properties that make them a basic model in different  application fields. The novelty presented in this paper is the accurate numerical analysis of the synchronization process in the connectivity regime of random networks in the inhomogeneous Kuramoto model. In particular, the numerical results support the conjecture according to which the critical threshold of the coupling probability of oscillators above which they almost certainly synchronize is that of the network connectivity. Furthermore, to the best of our knowledge, this analysis has never been conducted for networks of the size of the largest network analyzed in this paper.



§ RANDOM NETWORKS

The structure of a random network G(N,p) of N nodes, also known as Erdős-Rényi,  is determined by the probability 0 ≤ p ≤ 1 that a link exists between two randomly chosen nodes. The degree distribution of a random network, which is the probability that a randomly chosen node has degree k, is a binomial P(k)= N-1k p^k (1-p)^N-1-kand the average value of the degree is k̅ = p (N-1). For p < 1/N (subcritical region) the network is mainly composed of isolated nodes, tiny clusters and k̅ < 1. The largest cluster is a tree whose size is N_G ∼ N. For p = 1/N (critical point) k̅=1, the network is composed of many small disconnected components, mainly trees, and N_G ∼ N^2/3. For 1/N < p ≤N/N = p_c (supercritical region) k̅ > 1 and a giant component is formed which contains a finite part of the network and grows rapidly as p increases. However, the network is not yet connected in this region. The probability p_c is called connectivity threshold because  the network is supposed to became connected for p just above p_c. For p > p_c one has k̅≥N  and N_G ∼ N. At the end of the connected region p = 1 and the network becomes fully (all-to-all) connected. See<cit.> for details.





§ METHOD

In this work, networks of size N = 50, 100, 500 and 1000 are considered. For each type of network analyzed 10 simulations are performed  and  for each of them the distribution of natural frequencies is a Gaussian with ω̅ = 0 and σ = 0.1.
 Since isolated oscillators evolve independently, their dynamics do not affect the collective synchronization of those connected to the network.
Therefore, we consider the region with p > p_c. To study the limit p → p_c, we set p = p_c in the random generator of the numerical library used to  generate the  networks <cit.>, exclude disconnected configurations and accept only connected ones that  correspond to the case in which the giant component absorbs all the nodes of the network and has the minimum number of links.

The range of values of the coupling constant examined in this work is 0 ≤ K ≤ 1 and we choose to divide it into 100 steps. For each value of K the system of Eq.( <ref>) is numerically integrated up to 5000 time iterations of amplitude 0.1 s. The order parameter R is calculated by averaging the last 1000 iterations in order to exclude values due to initial transient phase fluctuations. The value of K_c was estimated by finite-size scaling analysis <cit.>. 




§ RESULTS

To study how the density of links affects synchronization we consider the probability increment Δ p = (1-p_c)/10 and calculate the order parameter R(K,p) for p = p_c + n Δ p for n = 0,...,10. The results are shown in Fig. <ref>. 






What is observed is that the oscillators synchronize and reach the state of phase coherence for all values of p examined in the connected region. The behavior of R(K) is very similar for all values of p > p_c. For p = p_c, the system reaches phase coherence, although more slowly. As the size of the network grows, for K > K_c the transition from state R ≃ 0 to R ≃ 1 is more and more sudden, as the shape of R(K) shows, which means that the system has a phase transition. This also happens when p = p_c, as shown in more detail in Fig. <ref>, where the derivative dR/dK is also shown. It can be seen that as N increases the maximum value of dR/dK increases, while its variation around the maximum shrinks.





   
To estimate K_c we consider that for K < K_c the oscillators do not synchronize and R decays around a value of O(1/√(N)). For K> K_c R increases and saturates at a value R_∞ < 1, although with fluctuations of O(1/√(N)). Therefore K_c can be estimated numerically considering that at the onset of synchronization R(N, K_c) has the smallest variation with N. Fig. <ref> shows R(N, K) for all values of p examined in this work. For p > p_c we have that K_c ∼ 0.16 ÷ 0.19. This suggests that the onset of synchronization does not depend too much on the level of connectivity of the network itself. The theoretical value predicted by Kuramoto is K_c ∼ 0.16. For p = p_c the best estimate is K_c ∼ 0.21, which confirms that for this network configuration the onset of synchronization is slower.



§ DISCUSSION

The conjecture that the probability threshold above which synchronization is highly probable is p_c = N/N has been considered by several authors. In <cit.>, in the context of optimization of nonconvex cost functions, authors proved, for the homogeneous Kuramoto model, that if p ≫N/N^1/3 the oscillators synchronize with high probability for N →∞. In <cit.>, with arguments from spectral graph theory, authors proved that the same occurs if p ≫^2 N/N. The issue has also been addressed in the context of graphon theory <cit.>. In <cit.> authors proved, still for the case of the homogeneous model and for N →∞,  that the oscillators reach phase synchronization with high probability if p > p_c. They performed numerical simulations for networks of size N ≤ 100.
In this work the above conjecture has been proved numerically even in the more general case of inhomogeneous Kuramoto model and for larger network sizes.



§ CONCLUSION

We have numerically studied the collective synchronization of coupled phase oscillators in the Kuramoto model in the connected regime of random networks of sizes N = 50, 100, 500 and 1000. The oscillators synchronize and reach phase coherence for all connectivity levels of the networks. The behavior of the order parameter R as a function of the coupling constant K does not show significant discrepancies between different levels of link density. For the limiting case in which oscillators are coupled with probability around the connectivity threshold of the network, synchronization still occurs, although in a more attenuated way. As the size of the network increases, the transition from the incoherent to the coherent phase state occurs more and more abruptly and the oscillators have a phase transition. The critical threshold value  for the onset of synchronization varies in the range K_c ∼ 0.16 ÷ 0.19  for all the connectivity levels of the networks. In the limiting case when the network goes from disconnection to connection  K_c ∼ 0.21.
A random network of a certain size can be obtained by perturbing a complete network of the same size by randomly eliminating links. Therefore, from this analysis it can be deduced that a random perturbation of a complete network of oscillators coupled according to the Kuramoto model does not affect the synchronization as long as the perturbation does not disconnect the network.


The computing resources and the related technical support used for this work have been provided by CRESCO/ENEAGRID High Performance Computing infrastructure and its staff <cit.>. CRESCO/ENEAGRID High Performance Computing infrastructure is funded by ENEA, the Italian National Agency for New Technologies, Energy and Sustainable Economic Development and by Italian and European research programmes, see http://www.cresco.enea.it/english for information

0

Banfalvi_2011
  Banfalvi G.
  Methods Mol Biol.76120111.

Manukyan_2011
  Manukyan A., Abraham L., Dungrawala H. Schneider B. L.
  Methods Mol Biol.7612011173.

Ziping_1993
  Ziping J. McCall M.
  J. Opt. Soc. Am.101993155.

Kourtchatov_1995
  Kourtchatov S. Yu, Likhanskii V. V., Napartovich A. P., Arecchi F. T. Lapucci A.
  Phys. Rev. A5219954089.

Takemura_2021
  Takemura, N., Takata, K., Takiguchi, M. Notomi M.
   Sci. Rep.1120218587.

Wiesenfeld_1998
  Wiesenfeld K., Colet P. Strogatz S. H.
  Phys. Rev. E.57199815631.


Shahal_2020
  Shahal, S., Wurzberg, A., Sibony, I., Duadi H., Shniderman E., Weymouth D., Davidson N. Fridman M.
  Nat. Commun.1120203854.

Vandermeer_2021
  Vandermeer J., Hajian-Forooshani Z., Medina N. Perfecto I.
  R. Soc. Open Sci.82021210122.


Winfree_1967
  Winfree A. T.
  J. Theoret. Biol.16196715.



Kuramoto_1984
  Kuramoto Y.
  Chemical Oscillations, Waves and Turbulence
  Springer
  19
   Berlin
  1984.

Strogatz_2000
  Strogatz S. H.
  Physica D: Nonlinear Phenomena14320001.


English_2007
  English L. Q.
  European Journal of Physics292007143.


Barabasi_2016
  Barabási A.
  Network science
  Cambridge University Press
  2016.


Networkx
  Hagberg A. A.,  Schult D. A. Swart P. J.
  Proceedings of the 7th Python in Science Conference (SciPy2008)200811-15.


Marro_1999
  Marro J. Dickman R.
  Nonequilibrium phase transitions in lattice models
  Cambridge University Press
  1999.

Moreno_2004
  Moreno Y. Pacheco A. F.
  Europhysics Letters682004603.

Ling_2019
  Ling S., Xu R. Bandeira A. S.
  SIAM J. Optim.2920191879.

Kassabov_2022
  Kassabov M., Strogatz S. H. Townsend A.
  Chaos: An Interdisciplinary Journal of Nonlinear Science322022093119.

Lovasz_2012
  Lovász L.
  American Mathematical Soc.602012.


Borgs_2019
  Borgs C., Chayes J., Cohn H. Zhao Y. 
  Transactions of the American Mathematical Society37220193019-3062.


Medvedev_2014_a
  Medvedev G. S.
  SIAM Journal on Mathematical Analysis4620142743-2766.

Medvedev_2014_b
  Medvedev G. S.
  Archive for Rational Mechanics and Analysis2122014781-803.

Medvedev_2019
  Medvedev G. S.
  Communications in Mathematical Sciences172019883-898.


Nagpal_2024
  Nagpal S. V., Nair G. G., Strogatz S. H. Parise F.
  https://doi.org/10.48550/arXiv.2403.13998
  2024


Iannone_2019
  Iannone F., Ambrosino F., Bracco G., De Rosa M., Funel A., Guarnieri G., Migliori S., Palombi F., Ponti G., Santomauro G. Procacci, P.
  Proceedings of the 2019 International Conference on High Performance Computing   Simulation (HPCS)20191051-1052.







 Paper Title: LoRaWAN Based Dynamic Noise Mapping with Machine Learning for Urban Noise Enforcement.


	
	
	
	
	How to Use the IEEEtran  Templates
	
	Based Dynamic Noise Mapping with Machine Learning for Urban Noise Enforcement
    
		
			H. Emre Erdem
			, Henry Leung
		
		This work was supported in part by the Alberta Jobs, Economy and Innovation Ministry’s Major Innovation Fund to the Centre for Autonomous Systems in Strengthening Future Communities and in part by the Natural Sciences and Engineering Research Council of Canada.
	
    
======================================================================================================================================================================================================================================================================================================================

§ ABSTRACT 
		Static noise maps depicting long-term noise levels over wide areas are valuable urban planning assets for municipalities in decreasing noise exposure of residents.
		However, non-traffic noise sources with transient behavior, which people complain frequently, are usually ignored by static maps.
		We propose here a dynamic noise mapping approach using the data collected via low-power wide-area network (LPWAN, specifically LoRaWAN) based internet of things (IoT) infrastructure, which is one of the most common communication backbones for smart cities.
		Noise mapping based on LPWAN is challenging due to the low data rates of these protocols.
		The proposed dynamic noise mapping approach diminishes the negative implications of data rate limitations using machine learning (ML) for event and location prediction of non-traffic sources based on the scarce data.
		The strength of these models lies in their consideration of the spatial variance in acoustic behavior caused by the buildings in urban settings.
		The effectiveness of the proposed method and the accuracy of the resulting dynamic maps are evaluated in field tests.
		The results show that the proposed system can decrease the map error caused by non-traffic sources up to 51% and can stay effective under significant packet losses.
	
	
	
		Acoustic localization, IoT, LoRaWAN, noise mapping, noise monitoring, wireless sensor networks.
	
	
	

§ INTRODUCTION

	
	Noise exposure is long proven to be a primary health problem due to its negative impacts on main body functions such as sleep, hearing, and mental capacity <cit.>.
	Thus, diminishing noise exposure plays a key role in improved quality of life.
	For this purpose, municipalities around the globe utilize noise management practices to constrain the exposure within healthy limits <cit.>.
	These practices include passive measures such as consideration of static traffic noise maps for new development plans as well as active noise enforcement measures such as elimination of noise limit violations.

	

	Static (or strategic) noise maps used by passive measures depict noise levels of desired areas over extended periods (a year) <cit.>.
	These levels are calculated based on the methodological frameworks such as CNOSSOS-EU, CRTN, and sonROAD18 created by various government authorities <cit.>.
	Since static noise maps present noise levels over extended periods, their outputs represent only perpetual noise sources such as road traffic and industrial facilities and not the intermittent sources with random emergence at arbitrary locations such as construction, loud music or car alarm.
	However, these temporary non-traffic sources constitute a significant portion of civil noise complaints, rendering a more capable alternative to static noise maps essential for effective noise enforcement <cit.>.
	Moreover, relying on noise complaints to mitigate noise violations is a slow process.
	For these reasons, internet of things (IoT) based dynamic noise maps incorporating higher frequency noise level changes are sought for elevated noise enforcement capabilities.

	Dynamic noise maps can be considered as collections of noise maps created at consecutive time steps with short intervals.
	At each time step, either a completely new map or a partial update to the previous map can be created based on the environmental parameters collected from a sensor network.
	No matter what method is used for map generation, existing dynamic mapping research focus on decreasing temporal error of traffic noise and omit the spatiotemporal error caused by non-traffic sources.
	As a result, the noise from non-traffic sources is either filtered out or incorrectly attributed to roads, damaging effectiveness of these maps for noise enforcement.
	
	

	To have a dynamic noise map with a short time step for a low-cost city-wide noise monitoring system, low-power wide-area networks (LPWANs) can be used.
	Although LPWANs provide lower deployment and maintenance costs by eliminating tethered infrastructure via battery-powered wireless nodes, they limit communication data rates significantly.
	For example, LoRaWAN, the most frequently used LPWAN protocol, enables extremely long single-link communication ranges using its LoRa modulation technique in return for extremely low data rates <cit.>.
	Combined with hundreds of nodes having to share the same channel using ALOHA-like medium access, the rate and size of messages each node can transmit is significantly constrained to prevent collisions.
	Using only limited data available, estimation of non-traffic source locations on the dynamic map also becomes challenging.
	
	To fill the gap for dynamic maps with non-traffic sources using LPWANs, this study proposes a dynamic mapping system with machine learning (ML) based acoustic localization to overcome the data rate limitation.
	The localization model uses only periodically transmitted 2-byte sound pressure level (SPL) readings and combines them with its prior knowledge of the acoustic behavior of the environment to predict the non-traffic source location.
	The final dynamic map is created as a joint visualization of traffic and non-traffic sources.
	The main contributions of this article can be listed as follows:
	
	
  * A scalable based dynamic noise mapping approach with non-traffic sources is proposed.
	
  * An ML based acoustic localization approach using simple SPL indicators to overcome the data rate limitations of LoRaWAN is provided.
	
  * A acoustic node hardware design targeting low cost noise mapping applications is presented.
	
	
	The remainder of this article is organized as follows.
	Related work is summarized in Section <ref>.
	The details of our proposed system are provided in Section <ref>.
	Performance evaluations based on field-tests are presented in Section <ref>.
	Finally, the paper is concluded in Section <ref>.



§ RELATED WORK


	Existing noise mapping literature can be grouped into simulation and measurement based approaches <cit.>.
	While simulation based approaches use computational methods to estimate noise levels, measurement based approaches use IoT technologies to collect measurements from acoustic sensors to generate a seamless noise level distribution over a wide region.
	Static noise maps are the most prominent examples of simulation based maps and their update interval of several years provide sufficient flexibility to execute time consuming acoustic simulations unsuitable for dynamic maps with frequent updates.
	Using the computational approach, the need to collect scattered SPL measurements are eliminated.
	Studies adopting this approach usually focus on improving the accuracy and computational efficiency of simulations <cit.>.
	On the other hand, dynamic maps are enabled with measurement based approaches using the spatiotemporal information provided from distributed acoustic sensors.

	A unique IoT-based approach for noise data collection is crowdsensing where microphones and network capabilities of smartphones are used to measure and transmit SPL measurements for map generation <cit.>.
	Although crowdsensing approaches eliminate the usage of a dedicated sensor network, calibration of microphones from different manufacturers and varying availability of up-to-date data may pose significant challenges.
	For example, <cit.> reports almost half of the users removing their application after using it only once.
	Moreover, privacy becomes a significant concern when users' accurate location information is attached to noise measurements <cit.>.
	
	On the other hand, dedicated acoustic sensor networks with cloud connectivity are used as a more robust IoT backbone for noise measurements, eliminating the shortcomings of crowdsensing in return for increased costs.
	However, existing noise mapping studies focus mostly on demonstrating their mapping techniques using pre-recorded measurements without considering the networking aspect <cit.>.
	For example, <cit.> demonstrates the integration of SPL readings into static maps using measurements logged at eight locations without any loss of information.
	Similarly, Aumond use SPL values recorded using a mobile measurement device that attaches position information to its logs <cit.>.
	Analyzing network conformity is another important aspect for effective evaluation of IoT-based mapping systems to demonstrate their feasibility in real world.
	While many noise monitoring studies prefer spatially sparse SPL values, they do not generate a map with fine-grained representation of noise levels.

	Several noise monitoring studies adopt commercial off-the-shelf IoT-compatible platforms such as Raspberry Pi and BeagleBone as their edge platform <cit.>.
	Although these single-board computers provide elevated capabilities with low cost, their power consumption figures prevent long-term battery-powered operation as reported in <cit.> with a daily energy consumption of 43 Wh.
	Hence, <cit.> uses power over ethernet technology to jointly meet network and power requirements of the nodes.
	However, tethered infrastructures are usually not preferred due to their higher long-term costs.
	
	For truly untethered IoT-based noise mapping/monitoring applications, low-power microcontroller based architectures with wireless communications are preferred.
	For this goal, various communication technologies such as WiFi, Zigbee, and can be considered <cit.>.
	While WiFi provides the highest data rates, its power consumption is also the highest, preventing long-term battery-powered operation.
	Zigbee on the other hand provides mediocre performance in both data rate and range, while or another low-power wide-area network (LPWAN) protocol combines the longest range and low power consumption in return for low data rates <cit.>.
	For example, <cit.> reports triple the lifetime for their node with compared to a similar node using WiFi.
	Thus, usually becomes the first choice for non-data-hungry applications.
	Moreover, <cit.> shows that large datasets can be sent with LPWANs if divided into smaller pieces and sent over longer periods.
	In a different based study, LoRa modulation is combined with their unique MAC protocol to enable multi-hop communication unlike the star topology of when sending small-sized SPL and class label <cit.>.
	Similarly, <cit.> uses LoRaWAN to collect SPL measurements to redirect road traffic to decrease noise at noise-sensitive locations such as hospitals.

	No matter which technology is used to transmit noise data, the assumptions on the collected data have a strong impact on the accuracy of generated maps.
	Hence, <cit.> uses a classifier to detect non-traffic noise events and discards audio signals containing these events in SPL calculation to leave traffic noise only for their intended map.
	Although they create event labels in situ, the manual process to remove non-traffic sources prior to SPL calculation prevents it from being realized as a real-time application.
	Conversely, other existing dynamic noise mapping approaches mostly assume contribution from non-traffic sources to noise levels to be negligible, limiting the variety of noise violation types against which dynamic noise maps can be used.

	For a more effective noise enforcement, non-traffic noise sources should also be depicted on the map.
	Such a task is not trivial since the locations of non-traffic sources are arbitrary.
	To overcome this challenge, acoustic localization algorithms can be used to estimate the position of non-traffic sources <cit.>.
	These algorithms estimate source locations using indicators such as steered response power (SRP), direction of arrival (DoA), energy level, or time difference of arrival (TDoA).
	Due to data rate and energy budget limitations, these algorithms may not be realistic for microcontroller and LPWAN-based IoT systems.

	Among the existing acoustic localization methods, energy based methods offer the lowest computational complexity at the edge nodes while requiring small amount of information transmission (energy level only).
	Least squares and maximum likelihood are among the commonly used methods for energy based acoustic source localization <cit.>.
	However, studies adopting these approaches usually assume free-field propagation <cit.>.
	In <cit.> and <cit.>, the localization methods use the energy level ratios of sensor pairs in a non-reverberating environment.
	Similarly, the authors in <cit.> use a feed-forward neural network (FNN) for acoustic localization using energy measurements in free-field environments.
	However, it is shown that omittance of acoustic reflections is known to cause poor performance in urban environments <cit.>.

	Another IoT-compatible localization indicator is TDoA where cross-correlations of audio signals are used to estimate the time difference of these signals.
	Using TDoA information from multiple sensor nodes, source localization in urban environments becomes possible <cit.>.
	Despite slightly higher edge computation requirements, TDoA based approach is an important alternative to energy based methods for urban acoustic localization especially when combined with a phase transform (GCC-PHAT) to prioritize phase over magnitude during correlation calculation.
	Instead of using cross-correlation to extract time delay, SRP based localization approaches calculate signal power at candidate source positions and returns the position with the highest SRP as the estimated location <cit.>.
	Since this approach requires cross-correlations to be calculated for all signal pairs, it requires audio signals from all the nodes to be collected at a central location.
	In <cit.> the authors use 2-second audio with 10 kHz sampling rate, thus each node transmits at least 80 KB of data (using 4-byte floating points) for each localization task which would take hours to be transmitted in LPWAN-based systems.

	In <cit.>, Faraji use DoA for localization, but their DoA calculation method require evaluation of beamformed signal energy in each discrete direction for each pair of 8 microphones whose computational load prevents battery-powered operation beyond 26 hours.
	As an alternative to signal processing based DoA estimation methods, end-to-end ML models eliminate feature extraction step and enable low-computation DoA prediction.
	However, the best offerings of the current literature focus on platforms with significantly higher power consumption than battery-powered nodes used in low-cost IoT-based noise monitoring systems <cit.>.
	
	

§ SYSTEM OVERVIEW

	

	In providing a low-cost LPWAN based dynamic noise mapping system, limited data rate of LPWANs poses a big challenge and the system must be designed to overcome this barrier.
	Our proposed system uses ML models to predict the specifications of the non-traffic noise sources on the dynamic map using sparsely transmitted small-sized SPL information from the end nodes.
	Since the acoustic behavior of outdoor environments may vary drastically due to the different sizes and shapes of  physical objects such as buildings, which makes accurate location prediction difficult.
	Therefore, our proposed dynamic noise mapping system divides the application region into 250 m × 250 m areas and uses area-specific ML models to predict non-traffic noise events as well as their locations and SPLs.
	This way, ML models can learn the acoustic behavior of the specific environment during training, and non-traffic noise sources can be more accurately localized given small amount of SPL data, increasing LPWAN compatibility.
	Such bottom-up approach enables elevated scalability for cities at different sizes.

	In the proposed approach, each area is monitored by nine sensor nodes and the data transmitted by the nodes are collected by multiple gateways.
	Each gateway covers a circular area with a radius of the gateway's communication distance d_gw.
	The frequently used terms of region, cell, and area are visualized in Figure <ref>.	
	
	
	
	For LoRaWAN, although battery-powered nodes eliminate power infrastructure requirement, they may significantly increase maintenance costs if frequent battery replacements are needed.
	To keep power consumption low, the sensor nodes in our proposed system calculate only fast time weighted (LAF) and 15-minute long-term equivalent (LAeq,15min) A-weighted SPL values.
	These measurements constitute only 2 bytes in total and are transmitted with only 5 to 15-minute intervals (Δ t).
	As a result, the transmission of these data consumes only between 0.36 to 0.84 bps of physical bitrate for each node when used with Δ t = 15 minutes and DR0 or Δ t = 5 minutes and DR3, respectively.
	
	To extract useful information from the scarce sensor data, our system uses two ML models.
	While the first model predicts non-traffic noise events (p̂), the other predicts the properties (location and SPL, x̂, ŷ, and l̂) if the first model returns positive.
	This way, computation requirements of the server are decreased by running the secondary model only when a non-traffic noise event exists.
	The predicted non-traffic source is integrated into the map after being simulated as a point source enabling the distinctive output of our system.
	An overview of our proposed approach is shown in Figure <ref>.
	
	

	When used against noise violations, the time until the dynamic map includes the new noise sources is an important metric.
	In addition to transmission interval, the lack of time synchronization among sensor nodes may contribute to this delay if packet arrivals from all the nodes are waited.
	Moreover, packet losses may cause SPL information temporarily unavailable.
	To overcome these issues, dynamic map update tasks are executed with a 1-minute interval using the latest measurements available to stay functional despite possibly lower output accuracy.
	This approach not only increases system robustness but also eliminates the need for acknowledgements, helping decrease network congestion by reducing the use of valuable downlink windows of LoRaWAN.
	
	It is noted that as a result, each area can have at most one non-traffic noise source at a time.
	Considering the size of an area, the probability of this assumption being violated should be very low and can be omitted.
	Alternatively, the size of the areas can be shrunk until this assumption is fully satisfied.

	We make the sizes of our ML models small so that running the models with a 1-minute interval for small regions can be carried out by regular computers.
	If city-wide coverage with hundreds of areas and 1-minute update interval is desired, a powerful server can be used.
	It is noted that the power required for the proposed system is much less than those mapping systems with complete simulations.

	

 §.§ Sensor Node

	To process acoustic signals and provide extended battery lifetime, we need to develop our own acoustic sensor.
	These sensor nodes contain dual microcontrollers, stereo microphones, battery and LoRa transceiver as shown in Figure <ref>.
	All the tasks listed in Figure <ref> are performed on the primary microcontroller while the more powerful secondary microcontroller and microphone are reserved for more complicated edge artificial intelligence (AI) functions for our future studies.

	
	
	
	
	The MEMS microphones located under windscreens at the tip of the extended arms have an overload limit of 120 dB and provide digital audio signal using protocol.
	Thanks to this protocol, nodes can easily switch between mono and stereo audio.
	Also, calibration is not mandatory thanks to the ± 1 dB of sensitivity tolerance <cit.>.
	The nodes first sample audio signal s_t at time t with a sampling rate of 31.25 kHz using direct memory access (DMA) for lower power consumption.
	s_t contains 1024 samples which takes 4096 bytes in the memory as floating-point values.
	However, the default frequency response of the microphone does not fully lie within the specified tolerances of ANSI specifications for the targeted Type 2 sound level meters and requires an equalizer <cit.>.
	Hence, an equalizing filter with frequency-based gain is applied to move the response at non-conforming frequencies into the specified limits.
	In addition, an A-weighting filter is consecutively applied to better mimic the human perception of audio levels as required by most noise enforcement bylaws.
	These two filters are implemented in second-order recursive structure with minimal memory footprint.
	Following the filtering, LAF and LAeq are calculated using:
	
	
    m^lf_t = 20log_10(
    		s_RMS,t/
    		10^(C_s/20))
    		+ C_sr + C_c

	
	
    m^leq_t = 10log_10(1/N_h∑_i=t-N_h + 1^t 10^(m^lf_i / 10))

	
	where C_s, C_sr, and C_c represent microphone sensitivity, reference sensitivity, and calibration constants, respectively.
	In (<ref>), T represents the preferred time duration for the long-term equivalence and N_h represents the length of m^lf storing historical LAF values measured within this duration.
	The calculated SPL measurements are within ±1 dB tolerance as tested in LAeq,1h.
	Our system implementation adopts 15-minute LAeq duration as a good trade-off between response latency and accuracy <cit.>.
	After the calculation of SPL indicators audio data is deleted, the indicators are transmitted as 1-byte integers if transmission interval is reached.
	Otherwise, the node goes to sleep for 15 seconds as a good tradeoff between measurement accuracy and power consumption <cit.>.
	The primary microcontroller can provide power consumption as low as 300 nA in low-voltage retention sleep contributing to daily energy consumption of as low as 50 mWh.
	For detailed consumption analysis readers may refer to <cit.>.
	
	Using the LoRa transceiver with 125 kHz bandwidth, our nodes can theoretically reach uplink data rates of 980 (with SF=10) to 5470 (with SF=7) bps on an unoccupied channel.
	Also, our nodes do not utilize automatic data rate (ADR) functionality to preserve valuable downlink channels.

	

 §.§ Machine Learning Models


	

  §.§.§ Binary classifier model for non-traffic event detection

	For event detection p̂, a binary classification model using a single long short-term memory (LSTM) layer with 100 units is used to utilize LSTMs' high capability in learning temporal changes.
	The only input feature is the change in the latest discrete LAeq measurement at each node (Δ m^leq_n).
	These 9 differences are fed into the network as if they are part of a single feature over discrete time steps instead of individual features to eliminate the need to store historical values and simplify the model.
	The model is trained using an additional dropout layer with 0.5 rate and 1e-4 learning rate.
	
	To save computational resources, the secondary model is executed only when this model predicts a non-traffic noise event.
	Hence, the correct representation of non-traffic source is only possible with high true positive rate of this model.
	
	

  §.§.§ Regression model for source property prediction

	To predict properties (easting (x̂), northing (ŷ), and level (l̂)) of the non-traffic source, a regression model using a dual-branch FNN is used.
	Different from the first ML model, the second ML model uses both LAF and LAeq foreground measurements fused via Kalman filter (KF) to combine the low latency of LAF and low variance of LAeq measurements.
	Here, the term foreground represents the values where background values are subtracted from the measured values (m^lf,fg_n = m^lf_n - m^lf,bg_n), with background levels representing the minimum levels observed at each sensor location.

	Since we are using cross-validation with leave-one-out method to evaluate the performance of the proposed system, a secondary set of features is needed to inform the model on the leave-one-out status.
	This way, the ambiguity on whether the measurement being 0 due to the long distance to the source or leave-one-out is eliminated.
	This extra set of features is called mask and is basically an indicator for the presence of the measurement.
	Mask value of zero means the respective measurement is reserved for cross-validation:
	
	
    k_n = 
    		0,    if node n is used for leave-one-out
    
    		1,    otherwise.

	Using two different groups of features, we adopt the dual-branch structure so that the network can learn distinct representations for each group before concatenating them.
	For this goal, each feature is connected to a layer of 320 neurons before concatenation.
	The resulting layer is connected to a hidden layer with 100 neurons before the final output layer.
	Between consecutive layers, a dropout rate of 0.2 is used to prevent overfitting.
	The model is trained using a learning rate of 1e-3.
		
	

 §.§ Noise Map Generation

	Our system utilizes individual noise maps for traffic and non-traffic sources before combining them for dynamic map iterations.
	Both traffic and non-traffic maps are generated using NoiseModelling simulation software <cit.>.
	While a traffic map uses multiple point sources representing the vehicles on roads, a non-traffic map uses single point source at the predicted location.
	Resulting vector maps are converted to 50 px × 50 px raster images using the nearest neighbor grid in QGIS software before they are combined <cit.>.
	Since our system divides an entire region into multiple areas, map generation is scripted to adapt different areas based on the coordinates given as parameters.
	
	To embed area-specific acoustic behavior into ML models, each model is trained using noise maps generated for their target area.
	Since collecting high amount of real-world noise violation data is only theoretically possible due to strict legislation around such practice, simulated synthetic data is used in our training.
	Although this approach requires additional computation, it is a one-time preliminary process, and it decreases edge computing complexity enabling localization using only the SPL measurements.

	Since the LSTM model predicts non-traffic noise events, it is trained using a scenario where 500 non-traffic noise sources randomly emerge and disappear within the target area at different times.
	The output maps without reaching a mean value of 35 dB are discarded since they do not possess useful information for training.
	After the resulting map is combined with the base traffic map, Δ m^leq_n values at node locations are extracted with a 1-minute interval.
	
	The second ML model is trained using single-shot noise levels discarding the time domain.
	Random sources in 10,000 trials are used to simulate SPL at sensor nodes.
	The resulting maps are further augmented by adding Gaussian noise (Z ∼𝒩(0, 1)).
	
	Finally, the directivity of the simulated sources is randomized to prevent possible bias.
	The contribution of each frequency bin is randomized while dB magnitude of the directivity (v) for each frequency bin at a specific direction (θ) is calculated using:
	
	
    v(θ,i_f) = 40 (cos(θ) - 1/2 i_f + 1)

	
	where i_f ∈ {1,2,…,7} represents frequency bin index for frequencies of {8000,4000,2000,1000,500,250,125} Hz, respectively.
	Generated random directivities are fed into NoiseModelling software to be used in simulations.
	Training, validation, and testing splits use 80%, 20%, and 20% of all the simulated map data, respectively.
	
	

 §.§ LoRaWAN Communication

	
	To evaluate system performance, the efficiency of the communication protocol represented as packet success rate (PSR) must be known.
	This study considers two factors affecting PSR: packet collisions due to time overlap and packet losses due to low link quality.
	
	To analyze collision based packet success rate r_s,c, packet airtime is needed to analyze time overlaps during channel utilization.
	Packet size in symbols and LoRa symbol rate can be used.
	While LoRa modulation uses a fixed preamble size of 8 symbols, the packet payload size (n_pl) varies based on the physical layer payload (PL), spreading factor (SF), redundancy check (CRC), data rate optimization (DE), and code rate (CR):
	
	
    n_pl = 8 + max(0, ⌈2 PL - SF + 7 + 4 CRC/SF - 2 DE⌉(CR + 4) )


	Assuming that a node uses fixed parameters for packet transmissions, packet airtime (a_pkt) is calculated based on preamble and payload sizes (n_pa=8 symbols and n_pl) as well as symbol rate (R_s = BW / 2 ^SF):
	
	
    a_pkt = n_pkt,sym / R_s
    		= (4.25 + n_pa + n_pl) / R_s

	
	Having packet airtime values, packets from two independent nodes sharing the same channel are assumed to collide if the midpoints of the packets' airtime (t_x and t_y) are not as apart as half of their total airtime:
	
	
    c(x, y) = 
    		1,    if  |t_x - t_y | < (a_pkt,x + a_pkt,y)/2
    
    		0,    otherwise.

	
	Consequently, the collision based packet success rate in each cell is calculated as the mean collision rate over all the link pairs:
	
	
    r_s,c = 1 - 1/N_cell×(N_cell-1)∑_x ∈ N_cell∑_y ∈ N_cell
     y≠ x c(x, y)

	

	The PSR based on link quality r_s,q is calculated considering the unrecoverable received packets due to insufficient SNR or signal strength below the receiver (gateway) sensitivity.
	For this analysis, path loss of link n at distance d_n is calculated based on the mean loss at the reference distance d_0, path loss exponent (η), and shadowing modeled as X_σ∼𝒩(0, σ^2):
	
	
    L_p,n(d_n) = L_p(d_0) + 10 ηlog_10(d_n / d_0) + X_σ


	Using the transmit power (P_t), transmitter and receiver antenna gains (G_t and G_r), link path loss (L_p), bandwidth (BW), and noise figure of the receiver hardware (NF), received signal power (P_r) and its SNR can be calculated using <cit.>:
	
	
    P_r = P_t + G_t + G_r - L_p

	
	
    SNR = P_r + 174 - 10 log_10(BW) - NF


	The energy per bit to noise power spectral density ratio can be calculated using SNR, spreading factor (SF) and CR using:
	
	
    E_b/N_0 = SNR - 10 log_10(SF × 4 / (4 + CR)/2^SF)

	
	Furthermore, link bit error rate is calculated as in <cit.>:
	
	
    r_e,bit = Q(log_12(SF)/√(2)×E_b/N_0)

	
	It should be noted that bit error rate is set to 1 for cases where SNR falls below LoRa's demodulation limits or received signal strength is below the modem sensitivity.
	
	To move from bit error rate to packet error rate, packet size in bits must be known.
	This value can be calculated using channel bitrate and packet airtime:
	
	
    R_b = (SF × BW) / 2^SF

	
	
    n_pkt,bit = ⌈ a_pkt× R_b⌉

	

	The conversion from bit error rate to packet success rate for individual link quality is performed with:
	
	
    r_s,q,n = (1 - r_e,bit)^ n_pkt,bit

	
	Therefore, the overall link quality based packet success rate is calculated as the average over all the independent links:
	
	
    r_s,q = 1/N_cell∑_n ∈ N_cell r_s,q,n


	Finally, the PSR of a cell (r_s,cell) with N_cell many nodes is calculated as a combination of success rates with time overlap collisions and imperfect link qualities:
	
	
    r_s,cell = r_s,c× r_s,q

	
	Although we have empirical individual link quality data from the field tests, we are using the above analytical model to define the channel behavior beyond the distances observed in our tests.
	The empirical data is used to verify this model.
	
	As a result, the quality of each link mainly depends on the distance and the SF used.
	This study uses pre-defined assignments (SF ∈{7,8,9,10}) instead of adaptive updates since downlinks diminish the number of possible uplinks significantly.
	For this goal, SF values are assigned based on the link SNR without shadow fading and minimum SNR requirements of LoRa modulation <cit.>.
	Since varying link quality with shadow fading may cause disconnection, each link uses the next SF (SF + 1) whenever possible.
	Although increasing SF by 1 causes longer airtime, it improves link quality beyond the losses caused by time overlap collisions.
	The other parameters used in the channel are as follows: BW=125 kHz, CR=1, DE=0, P_t=20 dBm, G_t=2.5 dBi, G_r=6 dBi, and NF=3 dB.
	For the path loss model, we adopt the parameters from <cit.> (η = 2.08, d_0=40 m, L_p(d_0)=127.41 dB, and σ=3.57) since they enable the best fitment to our empirical data.
		
	

§ EXPERIMENTAL RESULTS

	
	
	The performance of our proposed system in reducing the error caused by non-traffic source is evaluated in field tests using a speaker shown in Figure <ref> positioned at 6 random locations within the selected application area.
	In each test, the same music tracks were played for 15 minutes with the same order with the sound level of around 100 dBA.
	The reason for such duration is to focus on the entire transitional period of the selected SPL indicator (LAeq,15m).
	Each test was followed by a 15-minute break so that the LAeq measurements can settle back to background levels and allow the equipment to be moved to the next location.

	

	The nodes in the area were deployed as shown in Figure <ref> at 2 m height and at randomly selected existing pole locations that satisfy deployment constraints.
	These constraints force the nodes to be deployed with at least 50 meters distance from each other but within 25 meters of a building facade facing away from the building.
	Although these constraints were not mandatory for this study, they were implemented for a fair performance comparison with our prospective research.
	
	

	For the field test performance evaluation, the proposed dynamic mapping method was performed in consecutive fashion, where the data for all the field tests were collected and then the mapping was performed on the collected data.
	Although our system can be run in real time, this approach is preferred to augment more field tests with varying transmission intervals and packet success rates to limit the disturbance caused in the tests.
	During field tests, the data was transmitted with a 1-minute interval and then processed for varying transmission intervals and packet losses in 25 trials each.
	In each trial, each node was assigned with a random time offset to better mimic the lack of time synchronization.
	Also, each trial was augmented further by zeroing the measurements from one sensor at a time and setting its mask for cross validation.
	The zeroed values were used as reference values in error calculations.
	Finally, packet loss rates for the respective Δ_t and d_gw values were applied.
	As a result, the data collected from 6 field tests was expanded into 16,200 tests.
	
	Table <ref> shows PSRs calculated using (<ref>), (<ref>), and (<ref>) and the average number of packets reaching the gateway per area per hour.
	Since time-overlap collision probability increases with shorter transmission intervals, r_s,c decreases with shorter .
	Moreover, longer transmission distances require higher SF, hence longer packet airtime, causing more collisions.
	Thus, the values decrease as the gateway distance increases.
	On the other hand, link quality is independent from transmission interval.
	Due to higher path losses with longer transmission ranges, decreases for higher gateway distances despite using higher SF as explained in Section <ref>.
	The best overall cell packet success rate is achieved at 500 m of gateway distance and 15 minutes of transmission interval.
	However, lower transmission intervals enables the highest number of packets reach gateway despite their lower values, making them more useful for system since they provide more information to the ML models.
	It should be noted that the listed successful packets vary slightly from expected values due to random node deployment and shadowing.

	
	
	While the performance of our LSTM classification model is evaluated using prediction accuracy, the regression (FNN) model is evaluated based on the mean absolute error (MAE) of its predicted locations.
	In case of false negatives from the first model, our system misses its chance to correct the map error.
	On the other hand, our system may worsen the map error trying to attribute small variances to a noise event by predicting the location of a non-existent source.
	In such cases, the regression model usually predicts only a low source SPL (l̂) due to small changes in model inputs, hence the increase in the map error is minimal.
	Nevertheless, prevention of false positives is rendered more important than those of false negatives.
	The performance results of 0.75 precision and 0.57 recall reflect this ambition (higher precision than recall).
	The individual performance results of the ML models are listed in Table <ref> using the metrics of event detection (p̂) accuracy, MAE of individual and joint predicted location (e_x̂, e_ŷ, and e_loc = √(e_x̂^2 + e_ŷ^2)), as well as MAE of predicted source level (e_l̂).
	The trends in these metrics are observed to be highly correlated with the number of successful packets presented in Table <ref>.
	More packets reaching the gateway allow more up-to-date information being available for ML model predictions resulting in better location estimation.
	For example, the classifier provides 0.73 event detection accuracy when 105 packets are successfully transmitted, and this value goes down to 0.55 with 35 packets.
	It should be noted that 0.55 accuracy is close to random guess, but such a low accuracy is still useful in decreasing map error thanks to higher precision than recall as well as false positives' destructive impact being negligible due to low source SPL prediction.
	Similarly, location estimation values get better with more packets available.
	The exception of e_l̂ sometimes getting better with less packets is a result of synthetic data having lower SPL values compared to the real-world data.
	This is also the reason why e_l̂ is always around 7 dB.
	More successful packets cause the prediction to be better according to the synthetic training data, but this implies a larger discrepancy compared to field-test values.
	Compared with the best acoustic localization algorithms in the literature, 54 to 78 m of location estimation error in a 250 m × 250 m area may seem large, however the size of the data used to provide these results is extremely small and the data contains information delay up to 15 minutes.
	For the noise mapping application considered here, these localization results are effective in reducing the map error caused by non-traffic sources which is the ultimate goal here.

	
	
	Figure <ref> shows the map error decrease provided by the system as a result of joint utilization of ML models with red background for active test periods in the plot and samples of generated maps below the plot.
	For all the performance evaluations, the first test was clipped to provide equal total duration for different transmission intervals, leaving 5 tests in total.
	Also, cross validation was performed by using the measurements from 1 out of 9 sensors as reference for error calculations and replacing these measurements with zeros for other system tasks in each of 9 cross validation trials.
	For each test, prior error was considered as the maximum minus minimum SPL level throughout the test in dB representing the impact of non-traffic source.
	In addition, post error represents the mean difference in dB between the measured value (1-minute interval data) and the output map value over all the cross validations and random time offset trials at each time step.
	It should be noted that errors can only be measured at node locations since reference measurements are not available outside the node locations.
	As a result, the distance of the source location to the sensor nodes impact how much of the error will be observed.
	In the plot, post error starts with a high value for each test follows a decreasing trend throughout the test thanks to the increasing localization accuracy with more packets arriving and ends with an occasional peak.
	The peaks at the start of each test were caused by the delay until multiple nodes send their measurements with higher SPL values after a non-traffic source emerged.
	During this short transitional period, event detection model predicted false negatives, hence no correction was applied to the map.
	Similarly, the peaks in the error after the source disappears were caused by the delay until nodes' most recently received measurements contained lower values.
	These peaks in prior error last longer when bigger transmission intervals are used.
	These two problematic cases can be observed in sample maps provided at 4 discrete times in Figure <ref>.
	In the first map only road noise is shown, whereas in reality the test with a non-traffic source started at 11:02.
	Similarly, at 11:17, the source had already stopped making noise, but the presence of the non-traffic source still persisted in the map.
	Also, the occasional peaks between the tests are either caused by false positives in the event detection model or non-controlled noise sources such as birds with loud calls observed during the tests.
	Since our acoustic sensors are designed not to store audio recordings for privacy reasons we cannot perform post-analysis to differentiate between these sources and prefer to provide the cumulative error.
	
	
	
	The mean error correction results throughout all the trials are shown in Figure <ref>.
	The proposed system performs the best with the shortest transmission interval and gateway range.
	Its relative performance drops to 29% at the longest transmission interval and range.
	Although shorter transmission intervals cause lower PSR, the total number of packets successfully transmitted is still higher and improved system performance is observed as shown in Table <ref>.
	Similarly, a shorter gateway range gives better system performance.
	It should be noted that all the localization and error reduction results are reached using measurements from 8 sensors as a byproduct of cross-validation.

	The proposed system assumes full utilization of 8 out of 72 LoRaWAN uplink channels of US915 specification for city-wide coverage.
	This requirement can be relaxed without compromising the system performance using denser gateways leading to less channel occupation thanks to shorter packet airtimes.
	Although the system already meets the 1% maximum duty cycle limit asserted in some regions, denser gateways can also be used to adapt the system to the regions with less available channels and/or stricter fair usage limits.

	

	

§ CONCLUSION

	
	In this paper, we propose a LoRaWAN based dynamic map for urban noise monitoring.
	We develop a novel LoRaWAN acoustic sensor with edge AI capability.
	Realization of dynamic noise mapping with non-traffic sources is shown to be possible for when ML models are utilized to extract useful information from sparsely transmitted data.
	Using experimental data, the proposed system is found to stay operational under high packet collisions when sparse gateway deployment and frequent transmissions are preferred.
	The impact of channel model on this LoRaWAN IoT noise monitoring system is also analyzed.
	
	Our future work is planned to provide improvements to the localization and map error reduction performance using edge ML models while preserving the compatibility with LPWANs.
	
	IEEEtran
		

	-2plus -1fil
	


 Paper Title: Predicting Software Reliability in Softwarized Networks.






Predicting Software Reliability in Softwarized Networks


    Hasan Yagiz Özkan, Madeleine Kaufmann, Wolfgang Kellerer, Carmen Mas-Machuca 
Chair of Communication Networks 

Technical University of Munich

Munich, Germany 

yagiz.oezkan@tum.de, madeleine.kaufmann@tum.de, wolfgang.kellerer@tum.de, cmas@tum.de




















    30 July 2024
===============================================================================================================================================================================================================================================================================



§ ABSTRACT 

Providing high quality software and evaluating the software reliability in softwarized networks are crucial for vendors and customers. These networks rely on open source code, which are sensitive to contain high number of bugs. Both, the knowledge about the code of previous releases as well as the bug history of the particular project can be used to evaluate the software reliability of a new software release based on SRGM. 

In this work a framework to predict the number of the bugs of a new release, as well as other reliability parameters, is proposed. An exemplary implementation of this framework to two particular open source projects, is described in detail. The difference between the prediction accuracy of the two projects is presented. Different alternatives to increase the prediction accuracy are proposed and compared in this paper.






Bug History, Version Control System, Regression Model




§ INTRODUCTION



Delivering high quality software is of utmost importance for software vendors and platforms. For that purpose, they aim at reducing both, the number of undetected and unsolved bugs, to increase the software quality and dependability<cit.>. The software development process is a cyclic process that delivers periodically releases with new functionalities, solved bugs, etc. Each new release potentially has residual bugs (i.e., undetected and unsolved bugs from previous releases as well as from the added code). A software bug can be defined as an anomaly that spoils the normal operation of a software. Hence, bugs are big thread to the reliability of the software.



Furthermore, software development is nowadays being open source. Open source software development strategy offers many advantages in cost, service and support. As a result of these advantages the number of open-source projects and  the number of contributors increase consistently. Big open source projects are mostly based on the contributions from different programmers and/or teams, which develop different parts of the same software. Although it is an efficient and low-cost development process, it is prone to contain higher number of bugs than in closed software development processes<cit.>.


The problem for both, the developers/vendors of the new release as well as the clients considering its adoption, is how to evaluate the quality and dependability of new releases. For this purpose,  we propose in this paper a methodology to estimate the number of residual bugs of a new software release.  



Most of the open source projects,

adopt bug-tracking systems to efficiently manage any detected software bug <cit.>. These bug-tracking systems facilitate the bug detection and resolution by offering transparency on providing information about all the detected bugs, which can be processed by data-mining techniques <cit.>. These techniques can be helpful to create the bug history as well as understand the relation between the bug history and code changes of the different releases.



Software code metrics can be defined as the information about the properties of a source code like for example the complexity of the functions or size of the code. These code metrics have been used to predict quality, security vulnerability, and maintainability of a software <cit.>. This information can be easily gathered with help of online tools or programming libraries. Moreover, the history of the code metrics, which relates to the project releases can be calculated and considered in the residual bug estimation as proposed in this work. 











In this paper we present the following contributions: 

    
  * We propose a framework to predict the number of bugs in a new software release. This framework combines the bug history and the code metrics history of previous release(s) of a software project.
    
  * As some new projects do not have enough previous releases to make an accurate prediction. For this purpose we evaluate the usability of cross-project prediction and evaluate the impact of the prediction accuracy.
    
  * As the software development of some projects lasts several years (e.g., the first release of ONAP project was released in 2017 and new releases are still coming), aspects such as trends in the software development strategy or changes in the developer teams can impact significantly the bug estimation. Because of these changes, the usage of all previous releases can be unfavorable for the prediction. The impact of the number of previous releases on the prediction quality will be also investigated. 

    
 

After finding the best number of releases to consider and favourable metrics to feed the model, this framework can be used to predict the number of bugs in a new software release before its code freeze, which indicates the beginning of debugging process. 

This paper is structured as follows: Section <ref> summarizes the previous works that investigated the relation between bugs and code metrics and/or the effects of bugs on software reliability. In Section <ref> the framework to predict the number of bugs is discussed. An exemplary implementation of the proposed framework is explained in Section <ref>. The numerical results of this exemplary implementation are presented in Section <ref>. Finally, the paper concludes with some future work ideas in Section <ref>.



§ RELATED WORK




In this section works that combines the bug manifestation process and the code metrics are discussed by explaining the used methodologies and their outcomes. This section also introduces works that discuss the relation between bugs and software dependability.

In <cit.> the authors used different machine learning methods like linear regression, decision tree and multilayer perception to estimate the number of bugs in a project. The machine learning models are fed by the  code metrics like number of lines of code (LoC), number of commits. Based on data from 15 different projects from <cit.> the authors found that the linear regression model outperforms the other machine learning models and machine learning models can be successfully used to predict the number of bugs before the release of the software. As machine learning model they found that the linear regression model performs better than compared models and machine learning models can be successfully used to predict the number of bugs before the release of the software. Their model only consider the prediction after the release date. A prediction before release date might offer a possibility to the developers to provide more reliable software.


The authors of <cit.> predicted the number of bugs in a file with code metrics. They compared efficiency of four different code metrics to predict the number of bugs in a file. The results of their predictions were significantly higher than real number of bugs. They found that the depth of inheritance is better than the compared metrics for the bug prediction, they only considered one single metric for the prediction. In <cit.> authors  discussed the importance of choosing the correct metrics for the prediction of the number of bugs. They compared correlation based and wrapper based metrics for metric selection. They used different prediction models with multiple inputs for the considered models. They found that metric selection has an important effect on the prediction quality. The wrapper based metric selection found to be better than correlation based selection. However, they did not investigate correlation for different releases of the same project. This is an important issue because the similarity of the different releases of same project is expected to be higher and the metric selection based on that is expected to be more reliable. 

Instead of code metrics authors of <cit.> collected metrics depending on the judgement of the project managers of 31 different projects. These metrics includes information about design, documentation, scale of new code etc. Their method found more successful to predict the number of residual bugs of bigger projects compared to smaller projects. Although they could predict the number of bugs successfully the metrics they used is not easy to obtain and depends on personal comments of the project managers.

Instead of predicting the number of bugs some works focus on either a software is reliable or not. Reddivari and Raman <cit.> applied different machine learning models for software quality prediction. They made binary decision about reliability of a system 'faulty' or 'not faulty' and classified its maintainability into three groups 'light', 'medium' and 'heavy' depending on code metrics. The Random Forrest algorithm found as a reliable classifier for both reliability and maintainability prediction. Visagan et al. <cit.> predicted either a software is reliable or not depending in its code metrics. They found that projects with high complexity have poor reliability. The binary prediction might be not considered as superficial for dependability analysis.

SRGM models are employed to predict the bug manifestation after the release date in <cit.>. It is been shown that these models can be used to predict the number of residual bugs some time after the release date of the new release. They also showed that this information can be used to make an interpretation on the reliability growth of the software.



§ PROPOSED FRAMEWORK



Inevitably, any new release of a software project comes out with some bugs, that could not be found during its development and debugging period. These bugs, referred as residual bugs, have negative impact on the reliability of the new release. The goal of this framework is to provide a procedure to estimate the number of the bugs in the new release of a software project. The predicted value can be used to foretell the percentage of the bugs that are to be discovered and how reliable a new release is. Based on this estimation, developers/vendors can decide when the code is mature enough to be released and also users/customers can decide whether to  upgrade to the new release or keep the older version depending on their reliability requirements.

Based on existing literature, newly added, removed features and modifications of the existing features have been identified as main contributors of residual bugs <cit.>. The proposed framework relies on the code metrics of the new and previous releases as well as on the bug history of this software project. Code metrics are measures that gives insights into the internal properties of the release (e.g., Number of lines, number of functions), as well as the development process of a particular software project (e.g., number of commits, number of developers). The proposed analysis relies on the correlation between these code metrics and the detected bugs of previous releases. The overview of the proposed framework is given in Figure <ref>.



Most of the software projects follows a release cycle with dates that indicates: i) the start of the release (t_s), ii) start of the debugging period, so called code freeze date (t_f) and iii) release day (t_r). Between t_s and t_f new features are added to the software. At t_f, the debugging period starts and no more new features are allowed. Thus in this work the code from t_f is considered as the code that contains all the bugs. 


Let us give an overview of the main steps to estimate the number of residual bugs on release k given the bug history and the code metrics of previous releases: 

    
  * Bug History: Many of current open-source software projects (e.g., ONAP, ONOS) use trackers like JIRA, Bugzilla and Mantis, which allow users and developers to report any bug they experience. These bug reports include detailed information such as creation date, bug description and current status of the bug (if it is still active, it has been solved, etc.). The bug history in this framework represents the distribution of the bugs among the releases based on either the creation date or the affected release information in the bug report.

    
  * Code Metrics: The code metrics give information about the source-code features (e.g., number of lines of code) or development characteristics (e.g., number of commits) in a version control system. Most open-source software projects rely on version control systems like github and gitlab. These version control systems contain the history of the development process of the software. The code metrics of a particular software on a particular date can be found from the code on that date. Tools like SonarQube <cit.> and python libraries like pydriller <cit.> support code metric calculations. 
    
    Some of the considered metrics like lines of code or number of files, provide fundamental information about the state of the code on a day. In this case the code from t_f will be used to calculate the code metrics. On the other hand some code metrics like new lines of code aim to describe the development process, in other words only the changes made for the considered release. In order to calculate these metrics we take into account the changes made between t_s and t_f. 

    
  * Bug History and Code Metrics Correlation: The proposed framework utilises the code metrics and bug history of releases 1...k-1 to predict the number of bugs in the new release k of a software project. This is done by finding a meaningful relation between the code metrics and bug history, and converting this relation into a regression model. This regression model together with the code metrics of the new release are used to find the number of bugs in a new release. For that purpose, the code metrics that correlate better along the bug history are preferred over the ones with lower correlation. Thus a valuable set of code metrics can be chosen with the help of correlation values.
    
    New projects might not have fair amount of previous releases to find a valuable correlation between bug history and code metrics. In case a project does not enough releases to introduce a meaningful correlation, similar software projects with same language, similar release cycle or code review structure can be used to fill this gap. Feeding the model with older releases might not increase the quality of the prediction, because the code structure, experience level of developers and writing style might change with time. Therefore a correlation analysis can be done before including all their releases.   



    
    
  * Fit Regression Model: Regression models are widely used for prediction and forecast <cit.>. A regression model defines a response variable y related to one or list of covariates x_1, x_2, ... , x_n. The distribution of y values is related to x values and y value can be formulated as <cit.>; 
    
    y = f(x_1, x_2, ... , x_n) + ϵ

    where ϵ corresponds to the error. From the given formula, the prediction value can be defined as y = f(x_1, x_2, ... , x_n).  In the presented framework, regression model is based on the relation between the number of bugs in older releases 1 ... k-1 (response variable ) and the code metrics of these releases 1 ... k-1 (arguments). In this work fitting a regression model means finding a function that minimizes the error value ϵ. Therefore the code metrics that offers high correlation with bug history are decent candidates to be used as arguments in the regression model.
    
    
  * Prediction with Regression Model: The fitted model then can be used to make a prediction of the target variable given the arguments of the new release. In other words, only the code metrics of the release k are given to the regression model and regression model returns the predicted number of bugs of release k.
    






§ IMPLEMENTATION



In this section as an example one particular implementation of the proposed framework will be described. This implementation aims projects that use Jira for bug tracking and github for version control. The implementation of each step from Fig. <ref> will be discussed in detail. The projects 'Open Network Automation Platform (ONAP) <cit.> and Open Network Operating System (ONOS) <cit.> are examined as example.











 §.§ Bug History


Each software project using JIRA has a URL page. For example, the Jira page of the open-source project ONAP is ('https://jira.onap.org'), which  will be used as example. Since Jira is mostly used as project management tool, the type of tracked issues includes a wide variety of different types of issues (e.g., "Bug", "Milestone", "Improvement"). Some of them are also used for documentation of the software development steps (e.g., 'Milestone', 'Improvement' and 'Project Plan'). Since our purpose is to obtain the bug history, our implementation builds the bug history by filtering all the issues of type "Bug". 

As an example, two of the bugs with their associated information have been shown in Table <ref>.





After filtering all the bugs, they are appointed to their corresponding release depending on their "First Affected Release" field information. Unfortunately, not all the bugs are properly defined and many bugs reported for ONOS do not have affected version in JIRA. In this case, those bugs are associated to the release based on their "Create Date" as most of them are found during the debugging period and coming weeks after its release date (t_r). Even if development of the new release k+1 is started, majority of the bugs after t_r are related to newly released release k. Thus if the creation date of the bug is 14 days after t_r of release k, the first affected version as considered as release k. The dates and ditribution of the unlabeled bugs to releases are visualized in Fig. <ref>.










 §.§ Code Metrics


The proposed implementation calculates the code metrics by analyzing the  code. Therefore the code has to be downloaded from Github before code metric extraction. 















Also some metrics are calculated as sum of the other metrics like number of new and modified LoC.



  §.§.§ Downloading the repositories from Github


Two dates t_s and t_f are considered to extract the code metrics. It is assumed that the bugs appear because of the newly added features and changes made between t_s and t_f. Thus all the bugs of the release is assumed to introduced before t_f in other words the code from t_f is expected to have all the bugs of the considered release.

Some of the code metrics that considered are measures of the state of the code like number of functions, total complexity of the project and number of thousand lines of code (kLoC). The code state from t_f is enough to get those information. On the other hand having the code status from one day is not enough to find some of the considered metrics which measures the changes made in the code. New kLoC, number of new functions are some examples to these metrics, that cannot be found from one state of the code. Since the changes related to the release are made after t_s, the code state from this date is considered as the old state of the code and state from t_f is considered as the new version. Therefore the code states from last github commit before t_s and the last commit on t_f are downloaded. the commits added after t_f are considered as part of debugging process. The t_s and t_f dates should be found in the website or documentation of the considered projects and given by user.

Some of the code metrics like number of commits or number of contributors cannot be found in source-code instead they should be gathered from github. The number of commits per release is also extracted during the code download process. In this work the metric 'commit' represents the number of commits between t_s and t_f like other metrics that measures change.









  §.§.§ Extracting the code metrics


After downloading the source-code of different releases from the github, they are analyzed to extract the code metrics. In this work the code metrics will be divided into two categories. The terms complexity metrics and code size metrics will be used to differentiate these categories. The complexity metrics are calculated from each function separately and they are metrics related to their cyclomatic complexity like number of functions with complexity value higher than 15. The code size metrics are the metrics which defines the length of code like number of thousand lines of code or the changes made in the code like thousands of new lines of code and modified lines of code. Two different tools are used for the calculation of the metrics of these categories. The code size metrics are found with Count Lines of Code (CLOC) <cit.>. It gives the information about number of lines of code, number of files and also calculates the differences between two folders in this manner. Lizard <cit.>, which makes function level analysis, is used for complexity metrics. It calculates the complexity of the functions and also used to calculate the number of functions and number of new functions.



  
Code Size Metrics

The code size metrics describe the metrics which relate to the length of the code and quantity of the changes made in the code. We used CLOC tool <cit.> to calculate the code size metrics, because it calculates not only the lines of code but also the differences between the code in two different directories. The CLOC tool goes through all files and returns LoC, new LoC, modified LoC, number of files, number of new files etc.. This part takes a directory as input and returns the related metrics as output. 

The projects that are analyzed in this work are Java based projects with around 70% of all the LoC is written in Java. Therefore the code size metrics are extracted also specific to files written in Java. The languages like 'yaml', or 'xml' are excluded, because they mostly do not contain logical operations. Since they are mostly do not include complex operations, it has been assumed that they are mostly not the reason behind the bugs and even some of them like might be created automatically.




  
Complexity Metrics

Complexity related information are collected with the help of the 'Lizard' Python library <cit.>. The input of our implementation is the directory of the considered project and as output it gives the complexity metrics for this directory. In this work cyclomatic complexity is used to evaluate the code complexity, which is found useful by many research and widely used <cit.>. It represents the count of linearly independent paths in a piece of code. 

The counts of the functions with complexity values higher than specific thresholds (10, 15 and 20) are one of the extracted complexity related metrics. Additionally, the summation of the complexity numbers of all functions is considered. The main purpose of this tool is calculating the complexity of functions meanwhile it goes through all of the functions in the analyzed file. Thanks to this strategy the number of functions is also calculated.

In order to get complexity metrics for new and modified functions, the 'filecmp' Python library is used. First newly added and changed files in the project are found, then all the functions in the mew files are added to the count of new and modified functions. All functions in modified files are compared and if they are new or modified, they will also added to the count of new and modified functions.





 §.§ Bug History and Code Metrics Correlation


The code metric extraction returns 43 different code metrics. Since the number of metrics is high, some of them should be preferred over the others to predict the number of bugs in the new release. A meaningful relation between code metrics and bug history has been searched to choose the code metrics for prediction. Pearson correlation coefficient (PCC) is a measure for linear correlation to evaluate the correlation between two sets of data <cit.>. In order to evaluate the correlation between bug history and code metrics PCC is used. PCC gets a value between -1 and +1, where 0 means no correlation between two sets of value. Positive PCC value indicates positive correlation and negative PCC value means that correlation is negative. Higher absolute value of PCC indicates stronger correlation, where -1 and +1 point to perfect correlation. If the absolute PCC value is higher than 0.7, the correlation is considered as "high" correlation, whereas a value between 0.4 and 0.7 indicates "significant" correlation <cit.>. The correlation is weak if the PCC is below 0.4. The metrics with higher absolute PCC are expected to be more valuable for prediction. In this work 'pandas' library <cit.> is used to calculate the correlation between bug history and code metrics. 



 §.§ Fit Regression  Model


The experience from older releases is used to calculate the number of the bugs in the new release. The code metrics that has higher correlation with number of bugs has been chosen to fit the regression model. Since PCC gives information about linear correlation, linear regression model is preferred. The formula of the linear regression model is given in the following expression <cit.>:


    y_i = β_0 + β_1x_i1 ... + β_nx_in + ϵ_i,i = 1, ..., k-1


where β_0 is the intercept, β_1, β_2, ... , β_n are the slopes of the line and ϵ_i is the error value. In this work k-1 represents the number of older releases used to fit the regression model, y_i is the number of bugs in release i and x_in represent the nth code metric value for release i. Fitting the regression model aims at finding the optimal β values that minimize the error ϵ for all the older releases. In this work 'scikit-learn' library <cit.> is used to fit the regression model. Code metrics of the older releases and number of bugs of the same old releases are given to the regression model to fit.



 §.§ Prediction with Regression Model


After the model is fitted the code metrics of the new release is used to estimate the number of bugs in the new release. The calculated β values are express the fitted regression model. The code metrics values of the new release k (x_k1, ... , x_kn) are given to the fitted model to predict the number of bugs in release k (y_k). Since the β values are chosen to minimize the error ϵ_i for the older releases, y_k value is expected to be close to the real y_k value. 



§ RESULTS



The implementation described in Section <ref> is applied to two different open source software projects ONOS and ONAP. In this section the results for each step of the framework will be given.





 §.§ Bug History


Let us start with the ONAP project, whose first release Amsterdam was released on 16 November 2017. Since then, 10 more releases have been delivered. An ONAP release from its start t_s to its release t_r takes approximately 6 months. From its start to 17 October 2022, 11.666 bugs have been reported in ONAP Jira. These bugs have been assigned to their corresponding release based on the bug's affected release label in Jira.However, around 9.5% of these bugs could not be assigned as they do not have an information about the first affected release. Those bugs are assigned to the releases based on the issue creation date. The collected information for two example bugs are given in Table <ref>. 








As of 17 October 2022, ONOS project has 24 releases. An ONOS release from  t_s to t_r takes approximately 4 to 5 months. Similarly to ONAP, the bugs have been mapped to their corresponding release according to the collected bug information. In ONOS, 45% of all 2301 bugs (17 October 2022) do not have an affected release, and hence, those bugs are associated to the release based on their create dates. ONOS also has long time support (LTS) releases, whose bugs can be found for longer time compared to normal releases. It make the distribution process of the unlabeled bugs less convenient compared to ONAP. 

The number of bugs for the different releases of the ONAP and the ONOS projects are depicted in Figures <ref> and Figure <ref> respectively, where the blue bars correspond to the bugs with release information and the brown bars correspond to the associated bugs based on the bug creation date. It can be observed that for both projects, the number of bugs per release are higher for the first releases. However, there is a difference in the number and distribution of bugs without the release label. In ONAP, most of those bugs are associated to the first release, whereas for ONOS, those bugs are distributed along most of the releases. Also first ONAP releases contain the largest number of bugs without the affected release label but this trend is not observed in ONOS.









 §.§ Code Metrics


6 examples of the 43 code metrics are chosen as example. Those are the examples of development effort (commits), code metrics based on project size (kLoC), size of changes made in the code (new, removed kLoC), changes made in Java code (new kLoC Java) and addition of the complex functions (new complex Java functions). 

The number of bugs and 6 code metrics of the 6 ONAP releases are given in Table <ref>. Release 1 is chosen because it is the first release, release 3 is the most buggy release and 10 is the last and least buggy release. Other three release are intermediate points between those releases. Here it can be seen that the release 7 has the highest number of kLoC but it does not have the highest or lowest number of bugs. On the other hand, release 3 has the most number of commits and removed kLoC also the highest number of bugs. The release with the lowest number of bugs (release 10) is the one with the lowest number of commits. It can be said that there is a correlation between the code metrics and number of bugs. From this trend it can be said that if the number of commits and removed LoC is high, number of bugs in this release is expected to be high.




Same example code metrics for the 6 releases of the ONOS project are given in Table <ref>. Also first, last, most buggy (release 8) and least buggy (release 21) releases are chosen for ONOS. Other two releases are the releases between these releases. In ONOS also the release with the lowest number of bugs (release 2) has the lowest number of commits and removed kLoC. The release with highest number of bugs (release 8) has the highest number of bugs but it does not has the highest number of commits. This shows that code metrics can be used to predict the number of bugs in a release in ONOS project.



The kLoC value gives broad information about the size of the project and number of commits and new kLoC reflects the development effort for the related release. From Tables <ref> and <ref>, it can be said that the ONAP project is much bigger and more development efforts have been made for the given releases.  In connection with the development efforts the number of bugs of the ONAP releases are also higher than the ONOS releases.



To understand the connection between code metrics and bug history better, the change of bug history and commits that corresponds to the development efforts is shown in Figure <ref>. It can be observed, that ONAP is closer to a linear dependence than ONOS. 









 §.§ Bug History and Code Metrics Correlation

The correlation between the bug history and the code metrics have been performed for ONAP and ONOS as shown in Figures <ref> and  <ref> respectively. 

As observed in Figure <ref>, the expected number of bugs increases proportionally with number of commits. In order to evaluate the correlation between the number of bugs and each code metric, the Pearson correlation coefficient (PCC) is used.

Let us focus on the correlation of the code metrics with the bugs for ONAP, which are shown in the last row of the Figure <ref>. Six out of nine metrics shown "high" correlation, and in particular metrics like commits, sum of new, modified LoC and number of complex function have a correlation coefficient higher than 0.9. Those "high" correlation values motivate the use of code metrics to predict the number of bugs in a new ONAP release.



However, the correlation of ONOS is different as shown in Figure <ref>. In this case, the PCCs for code metrics and the number of bug are lower than the ONAP project. Only the metric commits has "high" correlation and three other metrics have "significant" correlation. Since the correlation values are lower than ONAP project, the prediction accuracy is expected to be also low.






 §.§ Regression Model



Linear regression model, whose formula is given in Formula <ref>, is used to predict the number of bugs. Fitting the regression model aims at finding the optimal β values that minimize the ϵ values.





The number of commits, number of new LoC, sum of new and modified Java LoC, number of new functions and number of complex Java functions values are used to feed the regression model as they offer higher correlation values and they give information about different aspects like development effort, complexity and Java based changes. The prediction values are evaluated with an error value which calculated with given formula:


    e = |E[#Bugs] - #Bugs|/#Bugs


where e represents the error value, E[#Bugs] is the predicted number of bugs and #Bugs is the real number of bugs.

Figure <ref> shows the distribution of the error value of the linear regression for different adjustments. In the first case referred as base linear regression model (BLR), all the previous releases are considered to feed the model (e.g., to predict the bugs of the fifth release, all releases up to release four are used to fit the linear regression).

Since all the chosen code metrics have positive correlation with bug history of ONAP, the second case referred as "linear regression with positive coefficients" (LR-PC) considered only positive slope β values. Also a linear regression model without intercept value (LR-woI) β_0 = 0 (third case) is trained because the changes made in the code is considered as the reason of the emerging bugs and without any change no bug is expected. The drop in the median value of error can be seen in the Figure <ref> 

In order to get better insight of the figure, Table <ref> summarizes the mean, median, maximum and minimum error values for each case. Both median and mean values are lower for LR-PC and LR-woI compared to BLR model. A "linear regression model without intercept and positive slope (LR-PC+woI) offers even lower error values. The median value drops from 0.499 to 0.12 which shows that the prediction can become closer to the real value with finding the correct structure of the linear regression model. These results show that the LR-PC+woI can be used to predict the number of bugs in the new release of the ONAP project.



Similarly to ONAP, all code metrics except for the number of complex Java functions, have been used to fit the regression model to ONOS releases. The reason to exclude the number of complex Java functions is due to its negative correlation with the number of bugs and using a positive slope is only meaningful if all the correlations are positive. Instead sum of new, modified and removed LoC of Java is used. 

The box plot of the error value for the predicted number of bugs with different compositions for ONOS is given in Figure <ref>. It can be observed that there are more outliers than in ONAP. The not labeled data and irregular release dates might cause the very high errors.

The mean and median values of the prediction error are given in Table <ref>. The lowest error value is reached with using positive slopes and no intercept for both projects. On the other hand using positive slopes alone or no intercept does not give lower error value. The higher values show that the lower correlation value effects the prediction accuracy negatively. 



 §.§ Cross-Project Prediction































Sometimes a project does not have enough previous releases to fit a regression model and achieve a realistic prediction of the number of bugs. As an example, the ONAP project started in November 2017, 3 years later than ONOS project. Can this model be used to predict the number of bugs in a new project with the help of older project? In order to answer this question, a regression model is fitted to all the ONAP and ONOS releases which were released before the code freeze date of the predicted release of the ONAP project. As example, the first 11 ONOS releases are used to predict the first ONAP release and the regression model is fitted to fitst ONAP release and first 13 releases ONOS releases that released before second release of ONAP to predict the second ONAP release. The results from the Table <ref> shows that the predicted values can give a rough idea about the number of bugs in the release. TheSince the linear regression model with positive slope and without intercept was better for in-project prediction the results of these model will be discussed. The error value is 0.081 for the first release, 0.481 for the second and 0.11 for the third release. The cross-project strategy can be used for the prediction of the number of bugs in the first releases of a new software.































 §.§ Prediction Accuracy with Different Number of Considered Releases


From release to release, the experience level of the contributors, the release procedures or priorities of the teams can change, which may impact the number of bugs. Over time, these aspects may affect the similarity between the releases and hence, the objective of this study is to evaluate how many of the previous releases should be considered to achieve the best estimation of the number of bugs of a new release. 

In order to perform this evaluation, the first task is to evaluate the correlation of the different metrics for the different number of releases. Based on this correlation, the change in the correlation value for different number of releases will be given.



As first task, we evaluate the PCC for a different number of releases. The median and mean PCC for bug history and two code metrics, which offer higher correlation value, commits and new and modified number of Java code is given are shown in Table <ref>. The number of previous releases is the number of releases used to get the correlation coefficient. Let us give as an example the case of considering 2 previous releases. In this case, for release x, the code metrics of releases x-1,x-2 are considered.  
The median and mean values are calculated from all the correlation coefficients values that are gathered using the same number of releases.

From this Table <ref>, it can be seen that the median values are high for lower number of releases, it is because those releases are coming from a shorter time period and similar to each other. On the other hand, the mean value is lower because a smaller shift in one release causes a lower correlation value. This is because of the low mean value using 2 previous releases might give lower prediction accuracy. After 3 previous releases, both mean correlation coefficients becomes higher than 0.7 which is the limit for high correlation. Although the mean values are comparably higher for more releases the median value does not follow the same trend. The median value for the best correlated metrics drops from 0.979 to 0.974. Thus using more than 3 and less than 7 previous ONAP releases seems reasonable from the correlation values. 



The error value for prediction of each ONAP releases with different number of previous releases used to fit the regression model is given in Figure <ref>. Linear regression model with positive slope values and without intercept is used for prediction. There are more results for latest releases because for those releases more  previous releases can be given to the model. After using more than 3 previous releases none of the prediction errors reach to 0.4, on the other hand after 7 previous releases all the error values are higher than 0.1. It can be said that neither using all the releases nor using only the last one or two releases gives the optimal prediction performance. 



The mean and median error values for regression model, which feed with different number of releases is given in Table <ref>. In order to make a fair comparison prediction error for the last 4 releases is also given because the error values is comparably higher than 4th, 5th and 6th releases as it can be seen at the Figure <ref>. The best value is 0.141 of the prediction with 9 previous releases but it only has one sample and the error for this sample is not lower than prediction with 4, 5 or 6 previous releases which can be seen in the last column of the Figure <ref>. After 9 previous releases the lowest number is reached with using 4 previous releases which followed by 5 previous releases. If we compare this results with the results in Subsection <ref>, we can see that the mean value of the 4 previous releases is lightly higher but the mean value is comparably lower. 
















































































































§ CONCLUSION



This paper proposes a framework to predict the number of bugs in a new software release. First each step of the proposed framework has been described. Then an exemplary implementation of this framework has been discussed. Two open-source projects ONOS and ONAP were analyzed with the presented implementation. The prediction has been improved with different methods and cross-project prediction is also tested to predict a new software with low number of old releases. 

The results shows that the number of bugs can be predicted at the end of the process of the software. The trustworthiness of this prediction depends on the project's properties. When ONOS and ONAP projects are compared, it can be seen that the correlation between code metrics and bug history of ONAP project is relatively higher, which leads to better prediction accuracy. One reason for that can be the number of bugs, which does not have any effected release information in issue tracker, is higher in ONOS. Other possible reasons are sliding release times of ONOS project and different type of releases like normal and long time support. The results also show that the bug history of a similar project can give a rough prediction on the number of releases of the first releases of the software. The optimal number of releases that should be used for a better prediction is also investigated. 

As future work this framework can be taken forward to predict the number of bugs in the su-bmodules of a software project. Also it can be tested with different machine learning models. The distribution of the unlabeled bugs can be improved with further investigation of commits that solves the bugs. Bug reports also includes information about the priority of the bugs, the framework can be tested with the bugs of different priority. A software reliability growth model can be feed with the predicted number of bugs and the expected reliability of the software on release date can be foreseen.



§ ACKNOWLEDGMENT


This work has received funding from the "DFG" under grant numbers MA 6529/4-1 and KE 1863/10-1.








1

visagan2019building
A. Visagan et al. Building a Data Mining Based Software Reliability Estimation Model,1em plus
 0.5em minus 0.4em1st Int. Conf. on Advances in Information Technology (ICAIT), 2019.

 
 falcao2020relating
F. Falca o et al., On relating technical, social factors, and the introduction of bugs,1em plus
 0.5em minus 0.4emIEEE 27th Int. Conf. on Software Analysis, Evolution and Reengineering (SANER), 2020.
 
 yang2017analyzing
 G.Yang et al., Analyzing emotion words to predict severity of software bugs: A case study of open source projects,1em plus
 0.5em minus 0.4emProc. of the Symposium on Applied Computing, 2017.

vizarreta2020dason
P. Vizarreta et al., DASON: Dependability assessment framework for imperfect distributed SDN implementations,1em plus
 0.5em minus 0.4emIEEE Trans. on Network and Service Management, 2020.

 mamun2017correlations
M. A. A. Mamun et al., Correlations of software code metrics: an empirical study,1em plus
 0.5em minus 0.4emProc. of the 27th Int. workshop on software measurement and 12th Int. Conf. on software process and product measurement, 2017.

rajendren2021predicting
S. Rajendren et al., Predicting number of bugs before launch: An investigation based on machine learning,1em plus
 0.5em minus 0.4emIEEE 45th Annual Computers, Software, and Applications Conf. (COMPSAC), 2021.

toth2016public
Z. Tóth et al., A public bug database of github projects and its application in bug prediction,1em plus
 0.5em minus 0.4emInt. Conf. on computational science and its applications, 2016.

 gupta2015developing
V. Gupta et al., Developing software bug prediction models using various software metrics as the bug indicators,1em plus
 0.5em minus 0.4emInternational Journal of Advanced Computer Science and Applications (IJACSA), vol. 6, 2015.

 osman2018impact
H. Osman et al., The impact of feature selection on predicting the number of bugs,1em plus
 0.5em minus 0.4emarXiv preprint arXiv:1807.04486, 2018.

 
 fenton2008effectiveness
N. Fenton et al., On the effectiveness of early life cycle defect prediction with Bayesian Nets,1em plus
 0.5em minus 0.4emEmpirical Software Engineering, 2008.

 reddivari2019software
S. Reddivari et al., Software quality prediction: an investigation based on machine learning,1em plus
 0.5em minus 0.4emIEEE 20th Int. Conf. on Information Reuse and Integration for Data Science (IRI), 2019.

 vizarreta2017empirical
P. Vizarreta et al., An empirical study of software reliability in SDN controllers,1em plus
 0.5em minus 0.4em13th Int. Conf. on Network and Service Management (CNSM), 2017.

 zimmermann2009cross
T. Zimmermann et al., Cross-project defect prediction: a large scale experiment on data vs. domain vs. process,1em plus
 0.5em minus 0.4emProc. of the 7th joint meeting of the European software engineering Conf. and the ACM SIGSOFT symposium on The foundations of software engineering, 2009.
 
nagappan2005use
N. Nagappan et al., Use of relative code churn measures to predict system defect density,1em plus
 0.5em minus 0.4emProc. of the 27th Int. Conf. on Software engineering, 2005.

 sonar
https://www.sonarqube.org

 
 pydriller
https://github.com/ishepard/pydriller

fahrmeir2013regression
L. Fahrmeir et al., Regression Models, Methods and Applications,1em plus
 0.5em minus 0.4emSpringer Berlin, Heidelberg, 2013.

 onap
https://www.onap.org/

 onos
https://opennetworking.org/onos/

cloc
https://github.com/AlDanial/cloc

 lizard
https://github.com/terryyin/lizard

  ebert2016cyclomatic
C. Ebert et al., Cyclomatic complexity,1em plus
 0.5em minus 0.4emIEEE software, vol. 33, no. 6, pp. 27–29, 2016.

eid2013linear
H. F. Eid et al., Linear correlation-based feature selection for network intrusion detection model,1em plus
 0.5em minus 0.4emInt. Conf. on Security of Information and Communication Networks, 2013.

 yu2017research
Z. Yu et al, Research on software project risk assessment model based on fuzzy theory and improved,1em plus
 0.5em minus 0.4emIEEE 2nd Advanced Information Technology, Electronic and Automation Control Conf. (IAEAC), 2017.

pandas
https://pandas.pydata.org/

scikit
https://scikit-learn.org/
 










































